How We'll manage when our SharePoint on-premises a site data increase 5GB per day?

Question

We have a SharePoint 2013 Web Application, Under this Web Application We have A site Collection. On this site collection around 5 GB documents/others data upload per day( Data related to one topic so that upload under the signal site). so if we calculate monthly increment data this is almost 125 GB per month, end of the year data will be 1 TB+. How we'll manage this type of data with best performance of this site.

Please suggest me the Best approach, how we'll manage this data with out compromise Site Performance. This question save others SharePoint-er lives also so please share your best experience. Thanks In Advanced.

Answer 1

Site performance shouldn't really be affected by the size of the documents on document libraries. It's not like SharePoint reads all documents when user accesses the site (unless you have some "latest X documents Content Query Web Part on the front page crawling all doclibs). Obviously you need to pay attention to how documents are browsed - so having list view that shows hundreds of documents (=without paging) will be slow - especially on IE.

Also pay attention to search crawl performance if searching for document content is required.

Your biggest challenge will be database size, as the bigger the DBs get, the more time it takes to backup (and restore) them. In SharePoint, you solve this by adding multiple content databases and not placing all site collections to one database. Usually 200GB is the recommended maximum DB size, but reason for that is DB maintenance related (backup/restore times for example). Supported DB size limit is 4TB, so theoretically you could have one database per year or even more, if that structure would work for you.

In you case what this would mean, is that you'd need to implement your solution so that it supports adding new site collections later on. Can you e.g., have site collection per "month", or site collection per "topic", site collection per "user", or some other way.

Obviously the end-user experience is the tricky part. If decide to have one site collection per "topic", and you create dedicated content database for that "topic" site collection, what happens when your defined maximum DB limit (say 200GB) is reached? You create new site collection for that topic and create new dedicated content database for that site collection. How will end users know or navigate between these site collections? How will they find a document as it could be in any of the topic's site collections? Search maybe? Is is OK to just have navigation links on top for user to navigate between the "topic" related site collections?

Without knowing the "use-cases" of the service and documents, I cannot give detailed tips, but hopefully this will at least help you get started.

I'm not even mentioning Remote Blob Storage, RBS, as an option as it will not get you around 200GB DB limit and will instead add more complexity without bringing much benefit.

Answer 2

you can check if audit is activated , if activated, you can purge logs perhaps