Saturday, 30 June 2018

Appropriate way to store files? (Rest, API, SQL)

Hey,I'm developing a REST API which accesses another API (for the sake of simplicity I will refer to it as the 'remote API') to retrieve data and store it into a database. In this context the data represents a file, which is represented as an byte array: the remote API is written in Java, where it parses a file into a byte array and serves the byte array as the response to the GET request of the REST API. The current implementation looks like this:REST API makes a GET request for a specific file.Remote API receives and parses the query (where it finds information about the desired file).Remote API parses the requested file as a byte array.Remote API sends the byte array as a response back to the REST API.REST API receives the response.REST API stores the byte array into my postgreSQL db.Later on the REST API should serve the stored data (decoded) to my frontend, written with react.js.Each file is around 5MB and represents ~ 5'000'000 characters. Given the above implementation, accessing a single stored file takes around 3 seconds. The response looks like this:{"data": [{ "id":1, "name":"file0", "content":{ "type":"Buffer", "data":"Content of the file..." }, "createdAt":"2018-06-29T10:17:17.829Z", "updatedAt":"2018-06-29T10:17:17.829Z" }] } Based on this, I have a few questions:What is the standard in the industry to store files?Where do you prefer storing files (refer to the question below)?Is it bad practice to store 5MB files into a single column in a db (currently I'm storing the content of into a single column for each file, of type bytea)Also, I believe the read process of a stored file takes no time, however the display of the files content does. I tried returning only the content of the stored file, without any other information (non-JSON), which lead to Chrome starting to download automatically the returned data (as you would download anything). This happened immediately, that's why I believe, the retrieving from the db isn't an issue.Am I wrong on this?Given that I would return the content, without any other information as shown above, how would I access the returned data on my front end (e.g. to display it in a table)? (since I'm not returning JSON, how would I access the data?)If you have any other suggestions, I would appreciate your help :)

Submitted June 30, 2018 at 12:00PM by Fasyx

No comments:

Post a Comment