User Interface Testing

Question

We are working on a large project with a measure of new/modified GUI functionality. We've found in the past that we often introduced new problems in related code when adding new functionality.

We have non-technical users perform testing, but they often miss parts and allow bugs to slip through.

My question: Are there any best practices for organizing the UI testing of a WinForms project? Is there any way to automate it?

Thanks!

Answer 1

There are GUI testing tools that will click buttons and stuff for you but they're pretty fragile in my experience.

The best thing to do is to keep your UI layer as thin as possible. Your event handler classes should optimally be only one or two lines that call out to other more testable classes. That way you can test your business logic in unit tests without having to actually do a button click.

Answer 2

You can automate GUI testing using White framework.

Also consider using TDD friendly design, i.e. use MVP/MVC pattern.

I would highly recommend you to read documentation from Microsoft patterns&practies teams.

Especially have a look at the Composite UI application block and CompositeWPF.

These projects specifically designed to give you best practices in GUI apps development including test driven UI.

Answer 3

Keep the GUI layer as thin as possible. Michael Feathers article, The Humble Dialog Box, is a classic. Also check out Martin Fowler's Passive View. I have also heard that the "automatic button clickers" are fragile, and that it's easy to spend more time maintaining the test than you spend maintaining the code.

Answer 4

In the event that someone finds this useful:

List of GUI testing tools found on Wikipedia.

Answer 5

The following book is an introduction to the subject. There are as many ways as there are developers out there..

http://pragprog.com/titles/idgtr/scripted-gui-testing-with-ruby

Answer 6

There are many tools and libraries available that can automate WinForms testing, ranging from open source solutions like White to the expensive commercial solutions such as HP QuickTest Pro. There is also the UIAutomation namespace in .NET if you want to roll your own automation framework. But the real cost of automation is in the time and specialised skills it requires to implement. Maintainability is also one of the most important aspects of automated test design; you dont want to expend excessive resource keeping the automation assets current with your application. There are also lots of factors influencing the decision to automate which will be specific to your specific application and organisation.

Your best bet will be to do some more research on the subject and check out some of the specialised testing sites such as http://www.sqaforums.com.

Answer 7

I found this quick and dirty way to test web page layouts in various browsers. It's called browsershots.org. Our client requires support in 5 browsers right now and that takes about a week for full regression testing. This service will deliver screenshots of some 70+ browsers and versions. I print them out and hold up the pages to the light. If they don't line up, there must be a layout problem.

Answer 8

I can't really help with organization or best practices, but an NUnit extension appears to be available for forms testing, NUnitForms.

Answer 9

I used a trial download similar to this product (http://www.tethyssolutions.com/product.htm) and this product (http://www.mjtnet.com/macro_scheduler.htm) years ago and I was happy with the results. These are pretty cheap solutions and some of these macro recorder products can actually be used for automated testing.

Answer 10

A new method available now is using Ruby, via the Ruby gem called win32-autogui. This provides a framework for testing Windows GUI apps. Combine it with Ruby tools RSpec and Cucumber, and it makes for quite a powerful testing framework.

Answer 11

How to organize UI testing depends on how you design the test cases.

Automating Windows Forms application in unit test level can use TDD framework, such as NUnit; or use BDD framework, such as NSpec.

Automating Windows Forms applications in functional testing level can use White, CodedUI, or even directly using Windows Automation API 3.0 (UI Automation and MSAA).

Since all these stuffs mentioned above are technologies rather than solutions, it's better to build some fundamental test automation/scripting frameworks based on these technologies before you start to write automated tests.