So You Were Forced to Use the dreaded TFS Collection /Recover Command, Now What?

Since we have used Recover on a production database and lived to tell the tale I thought I would share our experiences. If you read this post you will know that one of my client’s got themselves into a world of hurt where we needed to restore a nightly backup that was not detached.  I know, I know, detached backups, or using the TFS Database Backup Power Tool to backup and Restore, are the way to go.  Well, now THEY know that too   Nonetheless, sometimes you may find yourself needing to recover a TFS Team Project Collection (TPC) database, and if you’ve read the MSDN documentation you’ll know this is not an ideal situation. The Recover command is very lossy, BUT you get your data back. And in our case it was worth the risk.

So here is the backstory…  Someone deleted a Test Plan with a month’s worth of data in it, and if you know MTM you know there is no “undelete”. Restoring a backup was our only hope. BUT our nightly backups are SQL backups of the entire SQL Server instance, so undetached (we are addressing this NOW). The TFS Backup Power Tool does not detach the databases before backing them up, but it automates something a bit more complicated to allow you to restore single collections from a full backup. Plucking one TPC out of what we had, and attaching it to the TFS instance was just not an option.  You cannot attach a collection that thinks it is already attached.  Trust me, I know. And unfortunately we did not have extra hardware sitting around to allow us to restore the entire thing to a different instance to detach it properly.  So here is what we did:

1. Restore the backed up TPC from the nightly backup into our dev TFS environment
2. Used the TFSConfig /Recover command, followed by TFSConfig /Attach to get it attached in dev
3. Used the TFSConfig /Recover command to get the TPC into the proper state
4. Detach the hosed TPC from production
5. Restore that detached version of the TPC to production
6. Attach the backup to production (we actually hit an interesting bug in TFS 2010 at this point, so the attach was quite harrowing and involved an emergency hotfix to our TFS sprocs, I may blog about later.)

Now, I would love to say everything was perfect but the recover command did blow away some things that we had to get back into place before people could use the TPC again.  What we lost:

1. All the security setting ever!
   • Collection level groups and permissions
   • Team Project (TP) level groups and permissions in every TP in the TPC
   • Permissions around Areas and Iterations in every TP in the TPC
   • Permissions around Source Control in every TP in the TPC
2. SharePoint settings  (in every TP in the TPC). Settings on the SharePoint server themselves will be fine of course but you will probably see a “TF262600: This SharePoint site was created using a site definition…” error when you try to open the portal site that was once attached to those TPs. You will need to fix this in 2 places.
   • Go to TFS Admin Console, select the TPC you just restored and make sure the SharePoint Site settings for the TPC are correct. It will probably be set to “not configured” now.
   • Open team explorer (as an Admin user), and for each TP go to “Team Project Settings | Portal Settings” and verify everything there is correct. Ours were just plain gone so we had to enable the team project portal and reconfigure the URL.
3. SSRS Settings – this will probably be fine if you restored the database as-is but we also renamed it as part of the restore, and so had to update the Default Folder Location through the Admin Console for the TPC in order for this to work again.

So word to the wise, make sure you understand what the settings above are for all of the TPs in your TPC BEFORE you perform a Recover command because chances are you will have to manually set them all back up. And please, PLEASE backup your TFS databases properly.