Setting up remote execution of Coded UI Tests Via MTM

Question

I have searched high and low for clear set of instructions for my case. I have even followed instructions from Testing for Continuous Delivery with Visual Studio 2012 book.

The query is: I would like to run my coded ui tests from Microsoft Test Manager.

• I have created Coded UI Tests in VS2012 that work
  • This is a separate solution from the web application all together
• I use TFS2012 to manage the web app solution
• The test controller has been setup on the same server as the web application
  • The test controller has been configured to work with the tfs default project collection
• The Test/Lab Manager has been setup on my PC
  • I have added a standard virtual environment
• I have associated a test case to a coded ui test

The next step is where I am stuck...All of the forums and the book state that I need to create a deploy - build - test process. Since my coded ui test solution separate from the web application does this change anything, if not, how do I go about doing this?

If my understanding is correct once my coded ui test solution is built,

• I need to deploy it to some directory in the virtual environment created via Lab Manager
  • This will drop the coded ui test dlls into some directory
• I can than use Lab Manager to run the test which will use vs2012 on the virtual environment to run the coded ui test from the dlls

Please correct me if I'm wrong, and provide steps that I can use to finish of the set-up for my case.

Thank you all in advance!

Answer 1

If you are not looking to automate your tests everytime your web application is built and deployed then the steps are far easier.

• Create a Build for your Automated Testing Solution.
  • In Visual Studio go to your Team Explorer and click on Builds
  • Click on New Build Definition
  • General Tab
    
    • Fill in Build Definition Name
    • Leave Queue Processing Enabled
  • Trigger Tab
    • I choose Continuous Integration so that it would build every time it was checked in. But pick the one that fits your need the best.
  • Source Settings Tab
    • You should only need one line with Status Active
    • Source Control Folder pointed at the directory your Automated Test Solution is
    • Build Agent Folder $(SourceDir)
  • Build Defaults Tab
    • Select your Build Controller
    • In VS 2013 there is an option for Copy build output to the server I found that to be the easiest. If you dont have that option or you have a network share you can use then I would use that. It just needs to be accessible by what ever user you have running your Test Controller and Test Agent.
  • Process Tab
    • The default .xaml file should have all the pieces you need.
    • You should be able to leave everything as defaults with the exception of
      • Build -> Projects (Choose your solution for your Automated Test Solution)
      • Test -> Advanced -> Disable Tests (True) You dont need them to run everytime you build this.
      • Advanced -> Create work item on failure (False) You dont need to change this but most people dont need a ticket everytime a build fails on this type of solution. If you want one you can leave it at True.
  • Retention Policy (You can leave it be, or adjust however many builds you want to save)
  • Finally Run your build for the first time.
• Assigning the Build to your Test Plan
  • Open MTM and go to the Plan Tab inside Testing Center
  • Click on Properties
  • Make sure that the Automated Runs section has a Test environment selected for it.
  • In the Builds Section set your Filter for Builds to the builds with the name of the build you just created
  • Next to Build In Use: click Modify and choose the latest build.
  • Close and Save your Test Plan
• Link CodedUI Test to MTM Test case.
  • Open your Automated Testing Solution in Visual Studio
  • Go to Team Explorer and click on Work Items
  • Click on new Query and create a query to find your Test case and hit run (I found querying for the ID worked the fastest)
  • Open your Test case by double clicking on it
  • Go to the Associated Automation tab within your Test Case
  • Click on the ... button and select your Test Case.
    • If your test case does not show up it may be because either your solution is not open and Built, or you may have multiple TFS projects open and the project you are working out of is not alphabetically first. If you have multiple TFS Projects open I find it best to remove them completely so that they do not show in the Connect to Team and Projects window.
  • Finally save your Test case.

At this point you should be able to run it in MTM.

If you update your Automated Test Solution and have it rebuilt remember to go back to your Test Plan in MTM and assign the latest build. I have not figured out how to have it automatically take the latest build.

Answer 2

You need to create an environment for running your automated tests. Once you have done that you should associate your build and environment to the test plan of your AUTs. Also use the right .testsettings which point to the environment you have configured. And bazinga you are good to go.

You should be able to find many blog posts on this. Here is one I liked - http://www.deliveron.com/blog/post/Running-Automated-Tests-from-Microsoft-Test-Manager.aspx