Web Connection
Uploaded Storage Scenario
Gravatar is a globally recognized avatar based on your email address. Uploaded Storage Scenario
  Harvey Mushman
  All
  Dec 26, 2016 @ 08:40am

In the past, I always tried to seperate my upload files into logical groups... where the groups might be Client-#1, Client-#2, etc. and then further seperated into subfolders for photos, static documents, expiring document...etc.

But I also keep references to these files in my application tables with meta data that points back to who uploaded, when and where the system accesses it.

I think I did this because I had several disks mounted so it seemed logical, web uploads were seperate from OS and application stuff. Or perhaps it had to do with the size of backup media... I really don't remember anymore.

But rethinking it this morning as I'm working on writing a Backup function into my newest application, I'm wondering if any of this makes sense anymore... The cost of huge drives is cheap and with SSD's speed is not an issue.

I guess I'm still thinking that photos vs all other types of uploaded files (Word, PDF, Excel) are the only seperation that makes sense. (And only because of folder preview functions.)

Mind you, customers will be uploading these photos and other files so virus protection and size restrictions will be enforced. And as far as a hacker looking at someone elses file, having a common path is less secure but the filename is still sys(2015) generated so guessing names seems unlikely. And besides, the files are not considerd sensitive data, in fact they are for the most part all public domain.

...KISS comes to mind (Keep It Simple Stupid)

Any thoughts?

Gravatar is a globally recognized avatar based on your email address. re: Uploaded Storage Scenario
  Rick Strahl
  Harvey Mushman
  Dec 26, 2016 @ 12:10pm

The problem with keeping things in your app folders is that it can get in the way of backups. If data is big you may not want to backup your 'data' with your application code.

My Weblog is a good example. I have thousands of pictures on my blog that are associated with posts. These pics and other attachments amount to about 5 gig of or so. So now everytime I back up I have to push that data around with the site while data backups (the SQL Server DB) are backed up simply as DB tables.

Personally I prefer to keep data separate from code or logic related items whenever possible. You can still keep the data 'near' each other, by way of an application parent folder with subfolders for binaries, Web site and data. That's typically how I manage separation. This gives you the option of backing up everything all at once, or the individual pieces without interference.

But - I also think that data should logically be thought of as separate from the application always since often you have the ability to switch data sources as needed. For example on my blog I might choose to later store images on a CDN or something like Azure Blog storage which would be completely offline.

In the end - do whatever works best for your use case, but always think in the back of your head that the data might get really large and that you want to be able to manage that data in a meaningful way. It's never a problem when there are a few megs of data, but once those morph into Gigabytes of data it beceomes much more difficult to reason and move it around easily.

+++ Rick ---

© 1996-2024