SPC11 Fail-over Demo Reveals Importance of Remote BLOB Storage

After much fanfare, SPC11 has come and gone leaving us with lots of great information to digest.  For me, one of the more compelling parts of the show was the keynote address where Richard Riley, Director on the SharePoint team, demonstrated a high availability scenario with SharePoint 2010 and the CTP of SQL Denali.  The demonstration included a 40-second fail-over of a 14TB database while simulating over 7,500 concurrent user connections. It was certainly an impressive display of raw computing power with EMC VNX storage being at the center of it all.

If you missed the SPC11 keynote you can watch a recording here: http://www.mssharepointconference.com/pages/keynote.aspx.

While failing over a 14TB SharePoint database live on stage was both impressive and daring, I feel a few important details may have been lost in the overall demonstration.  Let’s first revisit thehigh-level environment configuration.  Microsoft has done us a favor by publishing the full documentation for the fail-over test.  You can find it here: http://www.microsoft.com/download/en/details.aspx?id=27573

At a high level, the SharePoint Server farm contained the following components:

  • EMC VNX 5700 with 300TB of storage
  • NEC Express 5800/A1080a server with 1TB of RAM
  • SharePoint Server 2010
  • SQL Denali CTP

** Note that all servers in the farm were virtualized and ran on the NEC server.

The test simulated the following:

  • A single 14TB content database containing document archive content
  • 7,500 concurrent users
  • 108 Million Documents in an Archive Scenario (note that Microsoft’s supported limit is 60 million items per content database)
  • Fail-over of the SQL Server due to a network outage

What stands out most in this environment is the massive amount of storage required to support a 14TB document archive in SharePoint.  This demonstration leaves many people asking the question, “This is impressive but can I afford that?”  You may already be aware that Microsoft released updated database sizing guidelines in conjunction with Service Pack 1 for SharePoint 2010.  As part of the updated guidance, SharePoint now supports content databases larger than 200GB in size.  Going beyond 200GB for both collaboration and document archive scenarios requires special consideration for the disk subsystem.  I previously wrote about these considerations in a previous post you can review here.

At a high level, databases larger than 200GB in size require at least .25 IOPS/GB minimum, with 2IOPS/GB being recommended.  This is the very reason for over-provisioned storage in the case of the 14TB high-volume fail-over test demonstrated at SPC11.  The result is 300TB of over provisioned SAN storage required to support two (2) 14TB content databases (there are two copies of the database in the fail-over scenario).  At a cost in the millions of dollars, this is hardly a cost-effective solution for managing large volumes of SharePoint content.  So you have to ask, “Is there a better way?”   The answer is most certainly “YES!”

Many organizations have dozens if not hundreds of TBs of content sitting on file shares and these organizations have a need/requirement to move much of this content into an environment that allows for true content lifecycle management.  SharePoint is the logical choice for much of this content due to the extensive features provided at a relatively low cost.

As you can see, when SharePoint grows in size the underlying infrastructure and storage costs can increase exponentially.  Microsoft, in conjunction with partners like Metalogix, provides a better way to scale your SharePoint environment while reducing overall storage costs.  SharePoint 2010 provides Remote BLOB Storage (RBS) interfaces that allow partners like Metalogix to provide the ability to externalize unstructured  data (i.e. Office documents, PDF, TIF, JPG, etc.) from the SharePoint content databases.

Leveraging RBS and Metalogix StoragePoint will allow your organization to grow SharePoint without the need for expensive over provisioned storage. For more information on RBS and Metalogix StoragePoint

Leave a Comment

Your email address will not be published. Required fields are marked *