Dispelling the Myths of Shredded Storage in SharePoint 2013: Part 2

Read Part 1: The Impact of Shredded Storage on SharePoint 2013

It is important to separate fact from fiction when it comes to shredded storage in SharePoint 2013, a topic that created a tremendous amount of buzz in the SharePoint community.

It is true that shredded storage, for collaboration scenarios, will reduce network and storage I/O associated with saving edits to an existing document. However, it completely misses the mark when it comes to performance related to file upload and download.

Thus, it is now time to dispel the myth that shredded storage serves as a replacement for Remote BLOB Storage (RBS) and show how you can optimize Metalogix StoragePoint to make the most of shredded storage in SharePoint 2013.

Remote BLOB Storage (RBS) provides plumbing to allow third-party providers to offload binary large objects (BLOBs) from SharePoint Content Databases to external storage locations. The primary benefit of RBS is to reduce the size of unstructured data (BLOBs) stored in SQL Server databases while providing support for commodity storage.  Third party providers like Metalogix StoragePoint have extended this basic BLOB offloading capability to provide a long list of enterprise capabilities including support for a wide variety of storage devices, compliant storage, archiving, and enhanced backup/restore capabilities.

Recently I have heard statements from within the SharePoint community and from other companies that RBS is no longer needed due to the introduction of shredded storage.  This couldn’t be further from the truth.

In part 1 of this series we discussed the primary benefits of shredded storage.  Microsoft’s goal in implementing shredded storage was to reduce the I/O associated with saving document changes.  Rather than store entire copies of files SharePoint 2013 shreds files into smaller chunks allowing for incremental changes to documents to be saved to the SharePoint Content Database.  As a result network and storage I/O is greatly reduced making the process of saving edits to a document very efficient.  Additionally SQL transaction logs associated with document edits are smaller making log shipping more efficient (in fact addressing log shipping challenges for Office 365 was one of the drivers for introducing Shredded Storage).  But what about the impact to uploading (new) and downloading existing documents for SharePoint?

In my experience, most SharePoint farms have a much higher ratio of downloads and uploads versus edits to existing documents.  The fact remains that while shredded storage greatly improves the I/O characteristics when saving incremental changes to SharePoint it has a net negative impact on uploads and downloads speeds.

With Shredded Storage in place, the core value of that RBS provides still exists.  Does Shredded Storage reduce the size of SharePoint Content Databases (SQL Server) by removing BLOBs from SQL Server databases?  Does shredded storage allow you to leverage commodity or complaint storage devices?  Does shredded storage address backup challenges with growing SharePoint environments?  The answer to all of these questions is a resounding “NO!”

Optimizing RBS with Shredded Storage

In SharePoint Server 2013, Shredded Storage and RBS coexist without issue.  As previously discussed, the result of Shredded Storage is a single file broken down into smaller “chunks” and stored within the SharePoint Content Database.  With RBS in place the smaller “chunks” will be externalized rather than a single file.  Regardless of shredding the end result is the same: BLOBs are stored outside of SharePoint Content Databases.

There are, however, considerations when optimizing the performance of RBS with Shredded Storage.  While Shredded Storage cannot be “turned off” in SharePoint Server 2013, it can be optimized or disabled altogether by changing the chunk size of the file shreds.  The default chunk size is set to 64KB however you could set the chunk size to 2GB (the maximum allowable file size in SharePoint) effectively disabling Shredded Storage.  When performance testing Metalogix StoragePoint with Shredded Storage, we found that setting the chunk size to 20MB will yield the best upload and download performance.  Changing the chunk size is quite simple and requires a bit of PowerShell script.

[void][System.Reflection.Assembly]::LoadWithPartialName(“Microsoft.SharePoint”)
$service = [Microsoft.SharePoint.Administration.SPWebService]::ContentService
$service.FileWriteChunkSize = chunk size in bytes
$service.Update()

You will need perform an IISRESET and restart the SP Timer Service on all machines in the farm.

As you have seen over this two-part series, there is a lot of misinformation currently floating around about Shredded Storage and RBS in SharePoint 2013. The reality is that neither replaces the other. In fact, Shredded Storage and RBS complement each other. Shredded Storage reduces network and storage I/O when saving document edits. And RBS reduces Content Database size, improves upload and download speed, and accelerates backup/restore operations. Following the guidelines above will help you get the most out of RBS and Shredded Storage.


2 thoughts on “Dispelling the Myths of Shredded Storage in SharePoint 2013: Part 2

  1. Excellent article and so pertinent to our timing. We have Storage Point and are implementing SharePoint 2013, so your article solved several questions that I had (especially regarding Metalogic’s solution).
    However, I have one still unanswered question? I see the requirements to implement Shredded storage as SharePoint 2013, SQL Server 2012, and either Hyper-V or Server 2012. Is Microsoft leveraging an API in SharePoint or is shredding available with SQL Server 2012 and Server 2012 alone? In essence, what combination is required for Shredded storage to be implemented?

  2. Hi Jim!. Shredded Storage is always on in SharePoint 2013. you cannot disable the shredded storage feature. The only configuration option that you have is to change the chunk size (default is 64KB). You can effectively disable shredded storage by setting the chunk size to 2GB. Note that the process of shredding a file happens within the SharePoint API/application and not at the SQL Server tier. Other than storing the chunks SQL Server does not have much to do with the shredding process.

Leave a Comment

Your email address will not be published. Required fields are marked *