So this week, we had a little bit of fun, by which I mean a day that started with panic and scrambling when someone accidentally deleted a Test Plan (yes, a whole test plan) in MTM in production. A well established test plan with dozens of test suites and over a hundred test cases with a month’s worth of result data no less... Some important things of note:

• test plans are not work items, they are just a “shell” and so are a bit easier to delete than they should be (in my opinion)
• there is no super secret command-line only undelete like there is for some artifacts in TFS, so recreate from scratch or TPC recovery are your only options here to get it back
• when you delete a test plan, you lose every test suite you had created.  Thankfully, not test cases themselves, those are safe in this situation.  Worst case, a plan can be created, although it is tedious and can be time consuming.
• when you delete a test plan, test results associated with that test plan will be deleted*. Let that sink in – ALL OF THE TEST RESULTS FOR THAT TEST PLAN, EVER, WILL ALSO BE DELETED.  ::this is why there were flailing arms and sweaty brows when it happened::

So at this point, you may be thinking it’s time to update your resume and change your phone number, but hold up. You may have some options to recover that data, so buy some donuts for your TFS admin(I like cinnamon sugar, BTW).  I should mention, there may be a lot of other options but these are the three I was weighing, and due to some things beyond my control we had to go with #2.

1) Best Case Scenario: restore your DETACHED (this is required) team project collection database from a backup, cause you’re totally taking nightly backups and using the TFS Power Tool right? You lose a little data depending on how old that backup is, but it may be more important to get back your test runs than have to redo a few hours of work.

2) Second Best Case Scenario: If you cannot lose other data, and are willing to sacrifice some test run data, then restore the TFS instance from a traditional SQL backup to a separate TFS instance (so, NOT your production instance), open up your test plan in that secondary environment, and recreate your test plan in production.  Not ideal, but if you didn’t have a ton of test runs this may be faster and you don’t sacrifice anything in SCM or WIT that was changed since the backup was taken.

3) Worst Case Scenario: if your backups were not detached when you did your last backup, cry a little, then use the recover command to re-attach them. The gist is to use the TFSConfig Recover command on the collection to make it “attachable” again, then attach it to your collection. I have written a separate post on this because it can be complicated…

Once you are back up and running, make sure rights to managing test plans is locked down!  It might not be obvious that you can even do this, or where to find it, since it is an “Areas and Iterations” level permission. But do it, do it now!  This permission controls the ability to create and delete Test Plans, so be aware of that. But for the most part, anyone with authority and knowledge to delete entire Test Plans, considering what they contain, should be the only person creating them.  If everyone needs the ability to create/delete these willy-nilly, then you are doing it wrong, in my opinion anyway.

I am still in the midst of getting this back up and running so will update once we’re finished. There is an MSDN forum post out there regarding one bug I seem to have uncovered, if anyone wants to look at it and maybe fix my world by answering it I am sure I’ll be able to add some more tips and tricks by then.

Now, I am no @ZainNab, the guru of “Tips and Tricks”, but I occasionally run across features that have been staring me in the face for YEARS and yet somehow went completely unappreciated, sometimes unnoticed.  And then one day it hits me and OMG my life is easier, and I want to tell everyone.  Sure, it’s a bit embarrassing to admit sometimes given that I worked at Microsoft for 5.5 years focusing on the Visual Studio tools, but who hasn’t done that?  Not you? Really?  I am skeptical…  There are after all, a bajillion commands to try and remember. For real, if you don’t believe me, look at the entire book that Sara Ford and Zain wrote about it. It’s worth every penny and Amazon has a great deal on it, pick up a copy!

So, back to my point. I was sitting in MTM, looking at a fairly daunting list of PBI based test suites, thinking “now which PBI’s were the ones where I had test cases to run again?”  I started thinking about writing a query, but that only helps is YOU are assigned to the test case, it doesn’t really help with test RUN assignment. Then it all came flooding back.  Wait, there’s this FILTER button to sort that out.  And conveniently it’s right there in front of my face ::face palm::  I felt a little better when no one else admitted to noticing it was there either. Maybe they were just being nice to me.  Either way, in case you didn’t notice it, check it out. Before:

After, I have MUCH fewer test suites that I have to look at:

That’s my Microsoft Test Manager tip o’ the day!  I won’t be posting them every day like Zain has been doing on his blog around Visual Studio 2010 for the past couple of years, of course I also don’t mainline 5 hour energy like he does   I will do them whenever I can.  Hope this was helpful! Feel free to post any tips of your own or shoot me a note if you have other questions or comments.

So there is a fantastic little conference gaining steam in the Midwest called Quest, which is all about Quality Engineered Software.  If you’ve never heard of it, you should seriously check it out next year regardless of your role.  As I have always said, Quality is NOT the sole responsibility of the testers, and this conference has something for everyone.  I was fortunate enough to be introduced to the local QAI chair who runs the conference the first year it ran (2008), which lucky for me also happened to be in my back yard.  I was with Microsoft at the time, and we had opted in as the biggest conference sponsor, cause let’s be real - who on earth in QA ever thought “Yeah, Microsoft has some awesome testing tools”.  ::crickets::  Right.

At the time VSTS (remember THAT brand? ) was still new-ish, and the testing tools were focused almost entirely on automated testing. Yeah, I know, TECHNICALLY there was that one manual test type but let’s not even go there.  I know a few, like literally 3, customers used the .MHT files to manage manual tests in TFS, but it wasn’t enough. The automated tools were pretty awesome, but what we found was that MOST customers were NOT doing a lot of automation yet. Most everyone was still primarily doing manual testing, and with Word and Excel, maybe SharePoint. We had a great time at Quest talking to testers and learning about what they REALLY need to be happy and productive, we got the word out on VSTS and TFS, and started planning for the next year.  I was able to be part of Quest as a Microsoftie in early 2009 as well, when the 2010 tools (and a REAL manual test tool) were just starting to take shape, and then the conference spent a couple of years in other cities.  Fast-forward to 2012 when Quest returned once again to Chicago.

I was no longer a Microsoftie, but if you’ve ever met me you know that working a booth and talking to as many people as possible about something I am passionate about is something I rock at, and enjoy! So I attended Quest 2012 again this year, this time as a guest of Microsoft.  I worked the Microsoft booth doing demos and answering questions about both the 2010 tools and the next generation of tools, and WOW did we get some great responses to them.  Particularly the exploratory testing tools.  I am pretty sure the reverse engineering of test cases from ad-hoc exploratory tests, and 1-click rich bug generation that sent ALL THE DATA EVER to developers gave a few spectators the chills. I certainly got a lot of jaws dropping and comments like “THIS is a Microsoft tool?!” and “I wish I had this right now!”. It was pretty great.

I was also fortunate enough to also get to attend a few pre-conference workshops, keynotes and a session or two.  I have to say, WOW, the conference is really expanding, and I was very impressed with the quality of the speakers and breadth of content.  As a born again agilista, I was so pleasantly surprised to see an entire TRACK on Agile with some great topics.  I was able to attend “Transition to Agile Testing” and “Test Assessments: Practical Steps to Assessing the Maturity of your Organization“ and learned quite a bit in both sessions.  One disappointment, there is even more FUD out there in the QA world than what I see in the developer world when it comes to Agile, what it actually means and how it SHOULD be practiced.  I’m not about being a hard core “to the letter” Scrummer or anything, but I also am not about doing it wrong, calling it Agile, and blaming the failure on some fundamental problem with Agile.  There are lots of Agile practices that can be adopted to improve how you build, test and deliver software, without going “all in”, and that was something I kept trying to convey whenever I spoke up.

I heard “Agile is all about documenting as little as possible”, “Agile lacks discipline”, “Agile is about building software faster”, and all of the usual suspects you would expect to hear.  No, it’s about "documenting only as much as is necessary; there is a difference!  Agile requires MORE discipline actually.  People on Agile teams don’t work faster, they just deliver value to the business SOONER than in traditional waterfall models, which sure, can be argued is “faster” in terms of time to market.  The only thing that will make me work faster would be a better laptop and typing lessons.  I still look at the keyboard, I know :: sigh::   I am highly considering doing a session next year on Mythbusting Agile and Scrum, to help people understand both the law and the spirit of Agile practices.  Overall it was great to see that the QA community is also embracing Agile and attempting to collaborate better with the development side of the house. We just need the development side to do the same   I also met at least a dozen certified Scrum Masters in my workshops as well, which was great to see! 

One of my favorite parts of the conference was of course getting to catch up and talk tech with Brian Harry.  He was the first keynote presenter of the conference, and spoke on how Microsoft “does Agile”, the failures and successes along the way, and even spent some time talking about his personal experiences as a manager learning to work in an Agile environment. I.LOVED.THIS. Yeah, I’m a bit of a Brian Harry fan-girl, but it really was a fantastic talk, and I had many people approach me in the booth later to comment on how much they enjoyed it. My favorite part was Brian admitting that at first, even HE was uncomfortable with the changes. It FELT like he was losing control of the team, but he eventually saw that he had BETTER visibility and MORE control over the process, and consequently the software teams.  It was brilliant.  So many managers FEAR Agile and Scrum for just those reasons. It’s uncomfortable letting teams self organize, trusting them to deliver value more often without constant and overwhelming oversight by project managers, and living without a 2 year detailed project plan - that in all actuality is outdated and invalid as little as a week into the project.  Wait, WHY is that scary? Sorry, couldn’t let that get by.

And so off I go again, into the software world, inspired to keep trying to get through to the Agile doubters and nay-sayers, and to help teams to adopt Agile practices and tooling to deliver better software, sooner.

Due to very popular demand to hold a VS 11 session out the the burbs, we are repeating the session held at the Aon Center in February, and are tweaking it a bit. Topics to be covered will include (but are not excluded to):

1. ALM Ranger Guidance
2. TFS Service Preview, a.k.a. TFS in the Cloud – what is it all about?
3. New Agile Planning Tools
4. Client Feedback Tool
5. Story Boarding tool
6. Team Explorer Changes (the code review feature is pretty hot!)

We may add some more items to that list, or refine it a bit, so be sure to check back closer to the meeting for more specifics.  And certainly let me know if you have any special requests!

Location: Microsoft Office - 3025 Highland Pkwy, Ste 300, Downers Grove, IL

When: Wednesday May 23rd, 6:30PM dinner followed by presentations and demos

Register here!  Please do register, as the security desk REQUIRES a list of folks to allow into the building at least 24 hours in advance.   And do keep in mind that we do our best to order food based on the number of attendees. IOW, if you need to cancel PLEASE let us know so we can adjust the food order so as not to waste our limited funding, well and of course food. Let’s NOT be wasting food.

Speakers

Prasanna Ramkumar is a Senior Consultant for Magenic Technologies and a VS ALM Ranger. He has extensive experience in implementing custom solutions using Microsoft development technologies for Magenic’s clients and provides ALM consulting to them using TFS. He has led and mentored several client projects using Scrum and is well versed in Agile methodologies. As a Ranger, Prasanna has been creating the hands on labs for the upcoming TFS11 Project Guidance and is actively reviewing other projects guidance.

Jim Szubryt is the TFS Product Manager and ALM Team Manager for the Enterprise Workforce at Accenture in Chicago. Jim’s TFS Team supports 1,300 developers in the global development centers. The ALM Team provides ALM guidance and assessments of the internally developed applications. Jim is also in the VS ALM Rangers program and has worked on the CodedUI guidance, TFS11 Upgrade guidance and TFS11 guidance on Teams. Prior to Accenture Jim worked at Magenic Technologies where he implemented TFS for clients and worked on a wide range of development projects.

Angela Dugan is the ALM Practice Manager for Polaris Solutions. Prior to joining Polaris, Angela Dugan was a technology evangelist with Microsoft focusing on Visual Studio and TFS group for over 5 years, and a software developer and architect for a small consulting firm in the western suburbs of Chicago for 8 years before that.

So if you’re reading this you are probably finishing up my 3 part story about updating a process template from Agile 4.2 to Agile 5.0 on a TFS 2010 server.  This is the last installment where I embarrass myself further by sharing one more stumbling block that I encountered along the way.  So now we have all of our awesome tools installed, we downloaded Hakan’s script, we got our work item definitions imported and updated, and finally added our trusty old Quality of Service Requirement to the new Requirements Category in the process template.  Everything was working beautifully until I went and tried to link a QoS Requirement to a Test Case. Cue Sad Trombone again…

This was certainly not handled in any script, and I couldn’t find any documentation of it on MSDN, so hey, maybe this is something actually NEW in terms of guidance   As soon as I saw this I knew what was happening.  I was pretty sure that somewhere there was some XML specifying what work item types were allowable in that dropdown, and my guess was QoS Requirement was not one of them.  I would have thought it was covered in the updated TestCase.xml used by Hakan’s script, or that maybe it was using the “Requirement Category” from Cateogires’xml but that would have covered QoS Requirement.  I double checked and it was not.  Here is the xml included with the script, note only “User Story” is allowed here:

I went ahead and made a little tweak so that QoS Requirements were allowable for the “Tested User Story” functionality and re-imported the TestCase work item definition using the Power Tools.  Essentially all I had to do was add my work item type to the TypeFilter node in the XML:

And now when I click “New” or “Link To” from a TestCase work item, I have access to my Quality of Service Requirements, HUZZAH!

Now, I am sure this is intentional. I assume in most cases you really only want “User Story” type work items to be linked in this particular tab, but for our purposes this is what we are looking to do.  I was a little curious as to why Hakan’s update script did not include the User Story work item type definition…  But hey, at the end of the day I demystified some more of the “magic” going on behind the scenes in TFS.  I am currently digging in a bit more to figure out if it makes sense to add User Stories to these upgraded Team Projects as well, since there are some very different fields and metadata being collected on them.  As I mentioned before, these are mostly inactive projects I am “experimenting on” so I’d love to hear and feedback or opinions on what you have done with your own projects.

OK, one last pro tip before I go. How often do you get an error dialog from TFS or VS, and you want to Google or Bing it, but now you have to type in all of the text by hand and hope you don’t miss a letter or number?  For me, daily!  *Sometimes* you can copy and paste the test, sometimes there is even a tool or link to let you copy it, but often times you are on your own. I totally ran into this on accident the other day.  So in OneNote you have a great screen capture tool that will work in any app, even on the desktop.  Make sure you have opened the OneNote app at least once, and seriously if you haven’t you’re crazy cause it’s the only tool I use for taking and sharing notes.  Hit Control+S, drag the cross hair around what you want to capture and let go. Copy the image to your clipboard and paste anywhere.  Cool huh? It gets better.  I noticed if you right click an image, you get the option to “Copy text from picture”. I saw that and thought, “no way that works!”, and lo and behold it does.

You’re welcome

That’s it for now, hope you learned something in reading about my adventures in process template upgrading.

Part 1 – Process and Tools

Part 2 – Field Mismatches

So, I am NOT calling myself the absolute guru of Team Foundation Server work item tracking or process template upgrades just yet, but I did learn a ton during the process and wanted to share my experience in case you can glean some wisdom from it.  Now I leveraged a LOT of content written by other VERY smart people from the TFS product teams so I try to be sure to give credit where credit is due. I experienced some bumps and bruises along the way because I was following at least 5 different sources on the internet for the upgrade and they did not all contain the exact same info.  Since the same things might trip you up, give the post a quick run-through all the way to the end.  I know it’s a lot to sort through, but you’ve probably played Mine Craft for hours at a time so I am sure you can handle this. Also, I broke it into 3 parts since there is so much to cover and it WAS getting a bit ridiculous, even for me  

We (Polaris Solutions) have a TFS 2010 instance that has a lot of legacy TFS 2008/Agile 4.2 Team Projects on it. I have been working on getting all of those projects upgraded to be more in line with the newest Agile 5.0 templates so that we can take advantage of some of the great features of TFS 2010 like Test Case management, hierarchical work items, new relationship types, and more. Now my life was made easier by the fact that this particular instance was not running SSAS/SSRS or SharePoint, and the original Agile 4.2 templates were unaltered, so really I was only concerned with making basic updates to the process templates and didn’t have to deal with reporting, merging customizations, or enabling the agile planning workbooks. But I still ran into some snags…

So, first thing is first, dump that chump command line and get a more functional one. You’ll spend some time at the command line so might as well not have it be painful.  I seriously owe Scott a big box of Sugar free brownies or something for this. Rocked my world!  If you’re not a fan of the command line, download the TFS 2010 Power Tools which will give you a lovely GUI interface for doing things like downloading your process templates and exporting/importing work item definitions and global types.  I actually switch between them depending on the task.  Also, the Power Tools just got updated in December of 2011, and some of the new features will bring a tear to your eye.

Next, download Hakan’s update script. Make note that you must be upgrading from a virgin Agile 4.2  process template to a 5.0 one to be able to just run it and be more or less done. If you have some customizations the script will still come in handy but you may need to tweak it, or reverse engineer your customizations back into the resulting template. Hope you documented all those changes, hehe. But don’t panic, the first thing his script does is back up your original template before changing it. Like I said, he’s a smart dude.  Here is what you will see when you grab the zip file and unpack it:

It includes a batch script to execute all of the necessary backup and import operations, a new Categories definition, 2 new link types (TestedBy and SharedStep), and new (SharedStep, TestCase) and updated (Bug, Scenarios, Task) work item definitions. Even if you decide not to use his script, these files could come in handy for manual updates as well, so I suggest grabbing it nonetheless.

Now honestly, I’m not one for blindly running ANY script, even one written by someone as awesome as Hakan, so I dug through his script line by line, looked at the artifacts it used, and compared it to the documentation on MSDN for updating old process templates to leverage new functionality.  I realized that the first couple of steps could be skipped altogether since I had some TFS 2010 projects in that collection already, and so things like categories and link types already existed.  You will see in a later post that this bit me in the butt Uploading any of the template artifacts using Hakan’s script won’t hurt anything so long as you haven’t modified the ones on the server, import = overwrite in this case.  I did a lot of it by hand at first, using my spiffy new command line tool and the power tools just so I would understand better what was happening.  Once I was comfortable with that, I updated another project using Hakan’s script. Much faster process and way fewer errors, shocking right? Hehe.  I did run into a couple of errors that the script could not handle; one that I address in my next post that has to do with some “friendly name” value changes between 2008 and 2010, and another having to do with something that may or may not be a standard practice which I address in my third post.   For now my process is to start with Hakan’s script to get the imports done quickly, then make manual tweaks to bring the template the rest of the way in line to what we need. 

Now if unlike me you do also have SSRS/SSAS and SharePoint installed and need to turn on some other features, check out Aaron’s post and Allen Clark’s post on how to finish up the upgrade. Like I said, mine was a little less involved but those articles are widely held as THE ones to use as a guide if you find yourself in this situation as well. Hopefully I sprinkled in some tips and tricks to help you out as well

One last thing you absolutely should have any time you need to work with process templates is a good compare tool.  I ran across 2 tools used for visualizing process template artifacts that saved my butt!  The first is the “Team Project Manager Tool” on CodePlex. Gives you a quick and easy way to look the XML in your template, do quick comparisons, but also includes tools to help you visualize/manage build templates, security and even source control. I love this tool SO MUCH!

And while in this instance I did not need it as much, the TFS Rangers Integration Platform Mapping Tool is a pretty neat tool and came in pretty handy too. When I had an instance where I wanted to quickly compare two WIT definitions and see how the fields mapped and where there were differences, it had some great visualization capabilities:

Well that’s it for my process and tools bit.  I have 2 more posts that cover issues I ran into with field mismatches, and making changes to allow for QoS requirement work items to take the place of User Story work items. They will be posted shortly!

Part 2 – Field Mismatches

Part 3 – Swapping in QoS Requirement for User Story