Current best method for storing uploaded documents (2024)?

[we5inelgr]

Hi all,

What is currently the best method, in terms of security as well as scalability and least complexity, to store user uploaded documents on a shared hosting platform?

Is it to store the uploaded documents in a secure folder(s) location with a reference pointer (file path) in the database?
Or store the documents in the database itself (blob datatype)?
Or use a nosql "document store" version of the database?
The documents uploaded will be:

Mix of sensitive information (ex. containing a living person's date of birth) as well as historical, non-sensitive information​

Varying in size from 1 page or image to several dozen​

Varying in document type, mainly from .pdf, image files (.png, .jpeg, etc), .doc or .txt text files (there will be no audio or video file types)​

The number of documents stored in the first year is estimated between 100 and 500, with about 1000 to 1200 additional each of the next couple of years.

If/when the site outgrows a shared hosting environment, other hosted solutions will be explored.

Other info:

PHP version 8.3.2​

MySQL version 8.3.0 (InnoDB type used)​

Thanks.

 

[EkBass]

There is billion things to consider when building a system like this. How many users, how many user levels, local hosting or external hosting, knowledge of programming and servers....

I once created a pretty simple server for images. Me or any of my family could upload images easily from their phones to my local server.

Server uploaded the image, it was named by the timestamp + suffix. SQL then had database where was included filename, uploader, date, time and tags.

There was simple folder structure where images was saved. If i remember right, after every 500 images a new folder was created and next 500 images was stored there instead. This way, number of image files in one folder was not increased infinity.

This system handled few tens of thousands images just fine until we got bored and stopped using it. Unfortunately i dont have it any more. I could have give it because editing it to use what ever files would not have been hard task.

You can put pdf to sql, but im not sure why to do that. If someone can hack to your computer and study folders, he/she can propably do so with the sql installed on server just aswell. Organised folder structure does the job well enough.

 

[we5inelgr]

There is billion things to consider when building a system like this. How many users, how many user levels, local hosting or external hosting, knowledge of programming and servers....

Thanks for the reply.

This will not be for public consumption and will be for authenticated users.

The number of users will be very small. Initially just 1, potentially up to maybe 50 at most, years from now.

Seems like using a filesystem to "store" the files is probably the way to go here.

 

[EkBass]

Seems like you are on a good route. For a tip, think a bit forehead of for what the usage is. Tags, categories and such are much easier create from start than after couple of years start editing whole system for them. Good luck with your project.

 

[we5inelgr]

Seems like you are on a good route. For a tip, think a bit forehead of for what the usage is. Tags, categories and such are much easier create from start than after couple of years start editing whole system for them. Good luck with your project.

Good advice. And, thanks for the reply!

 

[coatimundi]

Will you have to hide certain docs from certain users, or can everyone view everything?

 

[Antero360]

Mix of sensitive information (ex. containing a living person's date of birth) as well as historical, non-sensitive information

You might want to be extremely careful of what you're going to store.

 

[Antero360]

Mix of sensitive information (ex. containing a living person's date of birth) as well as historical, non-sensitive information

You might want to be extremely careful of what you're going to store.

 

[coatimundi]

You might want to be extremely careful of what you're going to store.

Exactly. PII is a slippery slope. There have been lawsuits recently due to data breaches. If it doesn't need to be stored, don't store it.

 

[IMMaximum]

Exactly. PII is a slippery slope. There have been lawsuits recently due to data breaches. If it doesn't need to be stored, don't store it.

I agree, this could be risky, make sure, like coatimundi said, not to store anything that you don't need to, only the necessary parts you should store!