Category: SharePoint 2007

Metalogix Releases StoragePoint File Share Librarian with StoragePoint 3.1

The StoragePoint team announced the released version 3.1 on 10/26/2010.  This is the latest release in the award winning StoragePoint product line and contains the new File Share Librarian Module (along with a list of performance updates and hot fixes).  The StoragePoint team created teh File Share Librarian module in response to customer requests for a high speed migration capability for file share data.  Traditional, full fidelity migration tools provide an extension set of features but sacrifise performance to provide those features.  You can expect to see somewhere in the neighborhood of 5GB per hour for a traditonal file-share-to-SharePoint migration tool.  As you can see we aren’t going to be setting any land speed records anytime soon with that level of performance.  In many file share migration scenarios customers want to move large quantitites of data from the file system into SharePoint and then disable file system access to the content.  StoragePoint File Share Librarain is the answer.  

 By leveraging the RBS/EBS capabilities of StoragePoint the File Share Librarain uses a “shallow” copy migration to catalogue file share content into SharePoint.  To help understand this concept you need to first understand the workflow involved with inserting content into SharePoint with StoragePoint deployed.  Files uploaded into the SharePoint web front end server (either through the user interface or some programatic means) are handing off to StoragePoint for processing.  StoragePoint will then store the file on some external storage device (the most common scneario is a file share).  For migration scenarios, rather than deal with the overhead of physically moving the file into SharePoint to have it then be immediately externalized by StoragePoint, the File Share Librarain module leaves the file in-place and simply catalogues the file to make it available in SharePoint. 

The benefits of the File Share Librarian module are many but most notable the ability to process millions of files in a short period of time.  The speed of your migration is no longer bound to the physical size of your file share.  Below you will find an additional list of features for the File Share Librarian along with links to resources where you can learn more about its capability.  Note that the File Share Librarian will work with both SharePoint 2007 and SharePoint 2010 environments.

  • The File Share Librarian product page: http://www.storagepoint.com/product.aspx?tab=1
  • Overview Demo: http://www.storagepoint.com/Images2/File%20Share%20Librarian.wmv
  • Team Share to Team Site Demo: http://www.storagepoint.com/Images2/Team%20Shares-to-Team%20Sites.wmv
  • File Share to My Site Demo: http://www.storagepoint.com/Images2/File%20Shares-to-MySites.wmv
  • Request a 3.1 /w File Share Librarian Trial: http://www.storagepoint.com/contact.aspx?action=trial
    • Single and Dual Access Modes.  Single Access Mode treats the cataloged file share 100% like a normal BLOB store, assuming that you are going to shut off end user access to the content once the file share is cataloged.  And as you’re probably already guessing, Dual Access Mode is there for that transitional period between end users accessing content via the file share to end users accessing content exclusively via SharePoint.  It would be nice to just say that on Monday the file share is gone and you need to go to this URL to get to your content.  You can send that email out until you’re blue in the face and 50% of your end users will open help desk tickets or call yelling and screaming Monday morning wanting to know where there content is.  Not to say that you can’t operate in Dual mode for some undetermined period of time, but why would you want to.  Let me put it this way…we didn’t put this feature in Librarian to encourage folks to operate this way…we put it in there to do our little part to keep the IT staff sane while end users become comfortable with change and/or get out of their own way.
    • Simulation Mode.  Run the Librarian in this mode if you want to detect and have an opportunity to cleanse invalid folder or file names…or shorten path and file names to get within SharePoint URI length limits.  You can let us do this stuff for you (…there is an option), but we’re going to be punitive about it (i.e. an ampersand goes bye-bye, it doesn’t get converted to and).
    • Scheduling Options.  Schedule the Librarian job to run on pretty much any interval you can come up with.  The initial run will catalog the entire file share, with subsequent runs cataloging only the changes (i.e. new files added, old files removed, file updates/changes, etc.).
    • Exclusions.  Exclude file share content by sub folders, filename or filespec, created date, last modified date, and last accessed date.
    • Dynamic Container Creation.  Librarian will create a dynamic container structure (i.e. site collections, sites, libraries, and folders) based on the starting Destination container you pick when creating the Librarian configuration.  If your Destination is a Web Application or Content Database, it will start creating site collections.  If you start with a Library it will start creating folders.  You get the idea.
    • Permissions Promotion.  As previously stated, the Librarian takes the effective permissions of the folders to be cataloged and maps those to Owner (Full Control), Member (Contributor), and Visitor (Reader) roles on the SharePoint containers it creates.
    • Picture and Asset Library Support.  If the file share content is being cataloged into Picture or Asset libraries, Librarian will create thumbnail and preview images for supported image types and make sure those are cataloged as well.  This removes the need to upload the images into SharePoint, where SharePoint would natively create the thumbnail and preview images itself.
    • My Site Support.  Pretty much everybody has that drive that’s mapped to them whenever they login to Windows.  It’s their own personal networked “My Documents”.  We’ve talked to lots of companies that would like to transform those personal drives to personal sites in SharePoint, but many times we’re talking about 10’s of thousands of users and 10’s – 100’s of terabytes of data.  Not something you’re going to easily represent in SharePoint.  By simply pointing Librarian at the root of those personal drives and picking a My Site Host as the destination you would be well on your way to accomplishing that move with little effort.  You would get some use case-specific options to let you tell Librarian where to find the actual personal drive folders (…even if they are not immediately under the root of the parent file share), where to store the content in the personal site (i.e. Personal or Shared Documents or a custom structure), whether or not to create the User Profile and Personal Site if it doesn’t already exist, and so on.  As a test, we took 48,000 user folders, 48 million documents representing 6TB of content and represented it in SharePoint as 48,000 My Sites and 48 million list items in a single 150GB content database…in just over 6 days.  And no, this wasn’t on some kind of fantasy hardware.  The SQL box was dual quad core Xeon processors with 24GB of memory and the app server the job ran on was a single quad core process with 4GB of memory.  While the job was running (16 threads…you can control that and change it in-flight anytime you like) the app server was running about 80% CPU utilization with the SQL box running under 10%, vast majority of the time running 5-7% (…it was taking a nap).  I think the data speaks for itself.  We’re going to conduct some larger-scale tests in the coming months and publish a whitepaper with detailed results, including farm configuration.

    StoragePoint 3.0 Webinar Series Announced

    Yesterday we announced a new webinar series where we will review and demonstrate the features of StoragePoint 3.0.  StoragePoint 3.0 is a highly anticipated release that provides significant advancement over our  already robuts BLOB remoting capabilities for SharePoint 2007 and SharePoint 2010.  New features like Intelligent Archiving, Multiple Storage End Points, and advanced reporting capabilities significantly reduces the total cost of ownership of SharePoint deployments while providing improved performance, reduced backup/restore time frames, and flexible storage options .  StoragePoint 3.0 is a must have for organizations deploying any level of document management or enterprise content management solutions on SharePoint 20007 or SharePoint 2010.  To register for an upcoming webinar please visit our Event Brite page at http://storagepoint.eventbrite.com.  For more information on StoragePoint visit www.storagepoint.com, blog.storagepoint.com, or www.youtube.com/user/storagepoint.

    EMC Certifies StoragePoint Centera Adapter

    Organizations are often tasked with storing content in a compliant way.  Whether you are dealing with SEC, HIPAA, SOX, or DOD compliance scenarios, SharePoint’s native storage architecture fails to meet the level compliance that organizations require.  Enter StoragePoint.  By leveraging supported APIs and interfaces, StoragePoint alters the native SharePoint storage architecture allowing you to separate the BLOB (Binary Large Object) from the content metadata.  The metadata is stored within SharePoint Content Databases while the BLOB can be remoted to a variety of storage locations.  For complaince scenarios organizations often turn to EMC Centera to provide WORM (Write Once Read Many) storage.  StoragePoint provides a certified storage adapter for storing SharePoint content on EMC Centera.  EMC recently certified our StoragePoint adapter.  Check out the press release.  http://finance.yahoo.com/news/Metalogix-Releases-bw-3469966683.html?x=0&.v=1

    Merge Content Databases After StoragePoint Externalization

    It’s not uncommon to have multiple SharePoint content databases for a single SharePoint application.  The separation of site collections into different content databases is typically done in an attempt to work around content database size recommendations provided by Microsoft (see Microsoft’s Plan for Software Boundaries document for more information).  In many cases organizations have one content database per site collection.  After deploying StoragePoint into your existing SharePoint farm the next logical step is to run the externalize job(s) which will relocate your existing content outside of the content database.  Once completed you will be left with very small content databases that can now be merged (note that you will have to run the DBCC_ShrinkDB script several times to reclaim your unused database space or rebuild table indexes before running dbcc_shrinkDB.  For more information consult the StoragePoint Administrator’s Guide).  Merging SharePoint content databases can be done using the stsadm command. Execute the steps below to merge your content databases.

    [more]

    Prerequisites

    1. **** Important!  Microsoft recommends applying the April Cumulative Update before you merge content databases.  There are known issues with this merge database command prior to this update.
    2. Even though your content databases will be very small after relocating BLOBs with StoragePoint’s externalize job, you must make sure you have enough free space to merge content databases.  The general rule is that you must have at least three times the size of source site collections database size.  *** Note:  Do not use the value returned for the StorageUsedMB property when running the stsadm -o enumsites -url webappurl to determine your database size.  With StoragePoint deployed this property will reflect the total space used by content in SharePoint but not the actual size of the content database. 
    3. In order to execute the following STSADM command you must be a member of the Farm Administrators group and be an Administrator on the local computer.  Additionally you need to have Full Control permission for any site collection being moved.  For SQL Server, you must be a member of the db_owner database role.

    Steps to Merge Content Databases

    1. From a SharePoint server in your farm, open a command prompt and navigate to %COMMONPROGRAMFILES%\Microsoft shared\Web server extensions\12\Bin
    2. Type the following STSTADM Command
      STSADM –o mergecontentdbs –url <url of web application> –sourcedatabasename <source db name> –destinationdatabasename <dest. db name> –operation2 Note that the URL is the URL of the web application that contains the site collection.  This is not the URL of the site collection itself.
      Operation2 = full database merge

    3. Restart IIS by running iisreset/noforce from a command prompt.

    BlueThread Announces StoragePoint for SharePoint 2010

    LAS VEGAS, Oct. 19 /PRNewswire/ — (at the Microsoft(®) SharePoint(®)
    Conference); BlueThread Technologies(®), Inc. (BlueThread) announced
    StoragePoint(®) for SharePoint 2010. Attendees at the conference can preview a
    demonstration of the 2010 version at the BlueThread booth.  StoragePoint
    enables organizations to realize at least a 95% reduction in the size of their
    SharePoint content databases by relocating content BLOBs (Binary Large
    Objects) out of the SQL database onto virtually any Cloud-based or on-premise
    storage platform. It improves SharePoint manageability, scalability,
    performance and security without any sacrifice to functionality or user
    experience.

    [more]

    “We wanted to provide our clients with unprecedented SharePoint 2010 BLOB
    storage flexibility,” says Rob D’Oria, BlueThread’s CTO and co-Founder. “With
    StoragePoint 2010, you will be able to create policy-based storage profiles
    which manage content BLOBs at a very granular level.  It also leverages both
    SharePoint 2010 EBS and RBS provider interfaces. Clients can choose to use
    them independently, side-by- side or leverage our out-of-the-box the EBS to
    RBS migration capability.”

    D’Oria continues, “Organizations don’t have to wait for 2010 in order to
    realize the benefits of StoragePoint. WSS 3.0 and MOSS 2007 customers can
    install StoragePoint within their existing SharePoint deployment while taking
    comfort that a no-hassle upgrade to 2010 is already in place.”

    Mark Wiley, Chesapeake Energy’s IT Supervisor for Document Management &
    Collaboration said; “Our current StoragePoint deployment reduced our content
    databases from 450 gigabytes to less than 20, a reduction of over 95%.  Not
    only did it dramatically improve SharePoint manageability, it surprisingly
    increased performance as well.  As an early adopter of Microsoft technologies,
    we are looking forward to SharePoint 2010 and leveraging StoragePoint.”

    “BlueThread is a great partner and StoragePoint is a perfect complement to our
    SharePoint 2010 ECM solution,” said Ron Cameron; President and CEO of
    Knowledgelake, a BlueThread partner. “StoragePoint enables our customers who
    are scanning millions, or even ten-of-millions of documents with our software
    to manage them simply and easily within SharePoint.”

    Customers can download a 30 day, no-cost trial version at StoragePoint’s
    website: www.storagepoint.com

     

    About BlueThread Technologies, Inc.
    A Microsoft Gold ISV, BlueThread Technologies(®), Inc. (www.bluethreadinc.com)
    is an Ann Arbor, Michigan-based company that specializes in Enterprise Content
    Management (ECM) and system performance solutions exclusively for the
    Microsoft(®) SharePoint(®) platform. With decades of ECM and years of
    SharePoint experience, we develop highly-scalable solutions and tools which
    are deployed at some of the largest organizations around the globe

    Fixing “Unable to connect publishing custom string handler for output caching”

    I was troubleshooting another issue on a client’s production server and ran across the following errors in the event log (Application Log).  The error was occurring on a KnowledgeLake Viewer web service.  After some quick searching I found Namwar Rizvi’s post on how to fix the problem.  It appears that SharePoint output caching is trying to access the web service that is called by a web part which is causing the error to occur.



    Event Type:        Error

    Event Source:        Office SharePoint Server

    Event Category:        Publishing Cache

    Event ID:        5785

    Date:                9/2/2009

    Time:                9:29:56 AM

    User:                N/A

    Computer:        <WebFrontEnd>

    Description:

    Unable to connect publishing custom string handler for output caching.  IIS Instance Id is ‘50319175’, Url is ‘http://xxxx.com/_layouts/klviewer/WebResource.axd?d=’.

     

    For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

    Enabling Zip Files in SharePoint Multiple Document Upload

    This pertains to Windows Vista and Windows 7 only.  If you are running Windows XP then stop reading.

    If you are running Vista or Windows 7 and you are attempting use the “Upload Multiple Documents” feature of MOSS 2007 or WSS 3.0 to upload zip files you have probably noticed that zip files do not show in the upload explorer window.  The “Upload Multiple Documents” feature is an ActiveX control that installs with Office 2003/2007 so if you aren’t running either Office versions then the option won’t show up in SharePoint.  The problem with zip files and the ActiveX control has to do with how Vista deals with zip and CAB files. By default Vista is configured to show zip and CAB files as folders allowing the user to drill into their contents within Windows Explorer.  For this reason they do not show up in the ActiveX control that handles multiple file uploads for SharePoint.  To fix this problem you need to delete the following registry keys.  WARNING! Backup your registry before making any changes!

    HKEY_CLASSES_ROOT\CLSID\{E88DCCE0-B7B3-11D1-A9F0-00AA0060FA31}
    HKEY_CLASSES_ROOT\CLSID\{0CD7A5C0-9F37-11CE-AE65-08002B2E1262}

    Now deleting the above registry keys is a bit more involved than running RegEdit, finding the keys, and deleting them.  Both keys above are protected keys and are own by the SYSTEM account.  In order to delete the keys you need to change ownership of the keys, delete sub keys first, and then delete the root key.  To do this you need to download the PSExec utility from Microsoft that is part of the Sysinternals tools.  The download is located here http://technet.microsoft.com/en-us/sysinternals/bb897553.aspx .  Once you have downloaded the tools and extracted the PSExec utility from the zip file, you need to then start a windows command prompt as the local machine administrator (if you are not logged in as the administrator then run the following from the RUN dialog:  runas :user/localhost\administrator CMD ).  Once the command window launches navigate to the directory where you unpacked the contents of the zip file that you downloaded earlier and execute the following command: psexec -i -d -s c:\windows\regedit.exe. At this point you are now running RegEdit as the SYSTEM account and will be able to change ownership of registry keys.  It is important to note that you have to delete child keys before deleting the root key.

    Bookmark and Share

    When Not To Crawl Content

    It is generally accepted that searching for content in MOSS or WSS 3.0 requires the content to first be crawled by the SharePoint Search Service.  However in traditional Enterprise Content Management (ECM) scenarios this typically doesn’t make a lot of sense.  If evaluate how most organizations manage content for the purposes of managing that content, you will quickly see why crawling content doesn’t make a whole lot of sense.  A typical ECM related business process involves the capture (data stream or scanning of content), categorization, processing, and archival of content.  In many cases significant time, money, and effort is expended in these business processes.  So if you spent significant resources to capture and categorize content then why would you rely on a search technology that is better suited for unstructured, full text queries to retrieve your content?  In most (and I say “most” because there are exceptions to this rule) ECM scenarios users are not conducting broadly scoped searches for content.  User’s search critieria is very targeted and specific.  For example an accounting user might want to search for an invoice for a specific vendor based on vendor id and/or invoice number.  A slightly broader search might be executed where the same accounting user is looking for all invoices from a specific vendor for the 2008 calendar year.  In either case the search is targeting.  Attenmpting to crawl this content doesn’t result in a favorable outcomes.  For starters crawling content in SharePoint doesn’t occur immediately after content is added and incremental crawls can take long periods of time to execute depending on how much content was added since the last incremental crawl was executed.  In many EMC scenarios users are required to immediately validate the content once it’s archived to SharePoint but requiring the content to first be crawled doesn’t support this process due to the latency by which items are made available for searching. 

     The performance challenges with crawling large volumes of content in SharePoint are well documented.  If you are not familiar with SharePoint limitations I would recommend reviewing Microsoft’s TechNet article title Plan for Software Boundaries (Office SharePoint Server) located here http://technet.microsoft.com/en-us/library/cc262787.aspx.  If you have ECM scenarios where users are conducting targeted searches in SharePoint, I would suggest evaluating existing search utilities that leverage CAML (Collaborative Application Markup Language) or developing your own.  In large volumen scenarios it makes sense to exclude the content from the SharePoint crawl all together.  I have personally experienced extremely unfavorable crawl performance as a result of larger content volumes in SharePoint even when the underlying SharePoint server infrastructure was optimal.

    Bookmark and Share

    SPQuery Doesn’t Search Sub Folders

    If you are using the SharePoint object model to query a SharePoint list you will find out that SPQuery doesn’t retrun items in the subfolders. In  order to execute a search query against a list and return results regardless of subfolder location you need to add “Scope=’RecursiveAll'” attribute to the SPQuery object’s ViewAttributes property.  Since the ViewAttributes property is a string you need to make sure that this attribute doesn’t already exist.  If it exists you must remove it and then add it back to the ViewAttributes property.   

     

    SPListItemCollection

    foreach

    (SPListItem item in itemCollection){
         //do something
    }
    SPQuery query = new SPQuery();
    query.ViewAttributes += ” Scope=’RecursiveAll'”;
    itemCollection = list.GetItems(query);

    Bookmark and Share