Which stage of Agile (SCRUM) should we start creating automation tests?

Question

A little background of me - I'm a manual tester for almost 2 years within an Agile environment using SCRUM (1-2 weeks sprints). So I'm wanting to introduce automation testing in my work using Selenium WebDriver (with Java).

My question is during when should I test the functionality manually and when should I convert them for automation testing?

I have been reading and getting different approach, such as:

1. When a new sprint is starting, convert the user stories to automated scripts from the previous sprint, OR;
2. Convert the user stories within the same sprint.

Any advice/s would be very much appreciated. Thank you in advance.

Answer 1

Test automation (and all other testing) should be part of the definition of done. This in order to make a potentially shippable product. Can you ship if it wasn't tested?

Testing should also be a whole team approach, so test-automation is not the testers responsibility. Start thinking about testing as soon as possible in the process.

Test automation is so important in Agile because:

Organizational Agility is constrained by Technical Agility

In other words, when you are slow in making changes to your product, then it doesn’t matter how you structure your teams, your organization or what framework you adopt, you will be slow to respond to changes.

https://less.works/less/technical-excellence/index.html

If you postpone testing until another iteration you will always be lagging behind. Making it harder to change the direction of the product as it is harder to refactor and safe-guard the external behavior of the product. Having any repetitive manual testing is key in slowing you down, automate-it!

A lot of testers will tell you that you should not start testing end-to-end until the product interface has stabilized. Don't wait, instead make good use of PageObjects and make sure your tests are maintainable and make it a developer responsibility to create and fix them.

Answer 2

The key thing is that you not mark a story complete unless you have written automated tests for that story.

So 1 seems to be out, as you are writing tests for a task completed in a previous sprint. what if the tests fail?

Answer 3

Ideally you should write automated tests in the same sprint that the code is written. Code shouldn't be considered "done" until automated tests have been written, and you must get code to a "done" state by the end of a sprint.

You should start the process on the first day of the sprint by working with the developer to understand the code, and to help them understand your needs as a tester. For example, if you are building web pages you can help them understand the need to add unique identifiers for every page element that you need to interact with.

Remember that in scrum, your job isn't to write tests. Your job is to work with the team to complete the sprint goals. That means communication and collaboration, which should happen very early in the sprint. You can begin working on test designs and test plans well before the code is ready to be tested.

Answer 4

If you are going to test automatically you might as well create the tests upfront. This will help you define what behavior you are expecting and stimulates you to think like a client, it should make your software more usable in the end. And you will benefit from the test immediately as you are implementing the functionality.

Answer 5

You can do either, but typically you want to target regression testing with automation tests. That would mean I would do manual until your sure it's solid enough to be a regression test. This may be in the middle of a sprint for some functionality and may be in a future sprint for other functionality.

Answer 6

As was stated in another answer, when testing occurs should be a part of the definition of done. However, I do disagree with some of that answer, so I wanted to expand with experiences that I've encountered.

In a truly Agile environment, everyone is able to do everything. There wouldn't be someone 100% dedicated to testing, you would also develop skills to help with some basic UI work, or something else. However, we rarely ever live in an ideal world.

What I would recommend would be to do a bit of a hybrid approach. For your Definition of Done, I would say manual testing should be part of the Sprint the work is coded in. You know it works, and any bugs can be immediately reported, possibly fixed, before the Sprint is over so you can plan the next one. By focusing on manual testing, you become familiar with what the code is supposed to do. In the beginning of the next Sprint when you might not have as much to do, you can set up your automated tests that can run as part of the build process to prevent regression errors.