Is there Anyone who Successfully Implement UI Test Automation Regime using Microsoft UI Automation?

Question

I am looking for an Automated UI test framework/ software tool. In the past I have been using TestComplete, and although it's a good piece of software, but the concept of GUI test automation was deemed to be sufficiently difficult that I wrote a few posts to complain about it.

One of the problems with third party test automation tool is that you have to learn new language in order to be productive on it, not to mention that the tooling support is poor. I am now planning to look into Microsoft UI Automation that comes with .Net 3.0 and the White Framework. But before I do that, I want to know what's the outcome there.

Anyone has any experience to share on this? Have you create a sustainable and successful test suite using UI automation on your application?

Edit: This seems like a very hard question. I would setup bounty for this if I don't receive any answers within these few days.

Answer 1

Cold Response... Better I answer a dummy answer, just in case there is no good answer to this.

Answer 2

This is an extremely late response (you probably already shipped), but I tested a WPF application with it.

I had a lot of flakiness while I was using the beta1 version of VS2010, and encountered some bugs. Using the release version, I had fewer problems.

It was challenging because I was learning WPF, VS UI automation, and WPF/Windows accessibility all at the same time. The fact that it is hard to create bullet proof automated UI tests added to this difficulty.

After the VS2010 RTM, I still had intermittent issues with a native/custom page that we wrapped. I think this was partially due to a parenting issue, and partially due to a threading/process issue.

The biggest headache I encountered was when I was trying to test against some control we used from the WPF toolkit, in particular the DataGrid control. It had some known UI automation bugs, and I had a devil of a time working around them.

I also encountered some challenges where the UI automation framework was caching data behind the scenes. I would have to re-instantiate controls in order to get updated data on them (particularly visibility, when a control was poorly parented).

If I had a pure WPF or pure Win32 app, that didn't inherit UI from other applications, wrap things in WebBrowser controls, use custom controls, etc, then I'd probably use it again. If you haven't made a decision yet, and your app uses any of those, then I'd see if I could use something else. Maybe a simpler set of scripts to do more limited integration testing, and try to cover the rest with unit tests, using mock objects.

I haven't used it within the last 6 months, though, so your mileage might vary.

Answer 3

We use the White test framework here with great results.

The framework uses win32 messaging to find the controls and interact with them. It's fairly slow on large forms with a lot of controls however, that's the only drawback I've encountered. We automate the test running using buildbot and nunit-console too.

Answer 4

I wrote an ironruby-based gem called Bewildr that wraps the MS UI Automation framework. It has been successfully used to automate MS WPF apps at various companies including the BBC. Here's a step-by-step guide to using it: http://www.natontesting.com/2011/08/27/step-by-step-example-of-bdding-a-wpf-app-with-cucumber-rspec-ironruby-and-bewildr/

Answer 5

I have been using MS UI Automation framework using C#.net and i found this extremely simplem and helpful.No issues were observed as such but it lacks report generating support so you need to write your own logic for that based on your specific test case.

Overall satisfaction : 8 / 10.

Answer 6

I have no experience with Microsoft UI Automation, but I'm using AutoIt (http://www.autoitscript.com/autoit3/) to do a little bit of GUI automation. I'm using it to test my Visual Studio add-in. Not really an answer to your question, but it could be interesting for you.

Regards,

Sebastiaan

Answer 7

Ive used a lightweight version of UI automated testing for the .net framework using a similar approach to this:

http://msdn.microsoft.com/en-us/magazine/cc163864.aspx

I did encounter issues using tabs and deciding which panels were going to be used, but that eventually led to a redesign of the form (so it pretty much found issues in design too!)