Helping testers who don't have specialized domain knowledge

Question

I am a team lead on a project that develops specialized engineering software. My developers generally have advanced degrees and the domain knowledge needed to work with the software we develop, but my testers don't have the necessary background knowledge. I don't have much flexibility in determining the composition of my team, so I can't hire new testers with an engineering background. What steps can I take to help my test team understand the software better -- or alternatively, how can I make it easier for them to do "black box" testing for which they won't need that knowledge?

Answer 1

If your test team does not have the necessary background knowledge, they are not going to be able to test your product adequately.

(This is a special case of a more general problem: If a team member does not have the necessary background knowledge, he is not going to be able to do his job adequately.)

You're going to have to figure out how to explain to your management and your HR people that your test team members need this background knowledge to do their jobs. This will probably have the highly-undesirable (from management's perspective) side-effect of costing money, in the form of (a) requiring the company to spend more money on their salaries, to be able to hire people with the requisite knowledge, or (b) requiring the company to spend money teaching/training them (and then quite possibly having to pay them more to retain them, now that they have the knowledge).

You would not ever, for example, ask a non-pilot to take a new airplane up for first flight test. He needs that knowledge. You MIGHT ask a non-pilot to try to test a new piece of avionics equipment, but you'd be smart to stick him in a simulator or send him up with an instructor so he gets some idea of what the pilot is up against in the Real World. (Pilots HATE complicated avionics. I didn't really understand why until I flew a sim scenario, and learned firsthand. They have their hands full with the airplane and the mission, and anything that adds workload is THE ENEMY.)

Having your developers write the test procedures is not the answer. Even if you want to argue that the developers are most familiar with the product, and most able to do it, you also need to test the paths that the developers AREN'T familiar with. Example: Certain functions on a certain airplane are required to be done in a very specific order. A certain operational pilot, during a VERY high-profile certification test, deviated from the procedure, and demonstrated a previously-unknown MAJOR issue, that caused the certification test to fail instantly -- and with very good reason. The airplane REALLY should not have done what it did. (I'm being deliberately vague here. I have my reasons.) A good test engineer, having been told "Do it like this", is GOING to deviate, to see what happens if he doesn't "do it like this". That's what test engineers do.

Note: When asked why the pilot deviated from the procedure, the lead answered by asking "Do you know 'pilot'?". Wry smiles all around the room.

Answer 2

Your developers are going to have to write very detailed and explicit testing instructions until the testers learn the domain and the app to the point where they can write their own. Make testing training videos from screen recordings just like they would for user training.

You may find some interesting bugs getting discovered because your testers don't think like engineers or programmers.

If this isn't feasible, someone in your firm needs to address the current business testing model and make room for more qualified people.

Answer 3

Your testers do not need to think like a developer - for black-box testing they need to think like a user of your software, and they should have additional knowledge about how to test software in general. So assumed your testers are smart enough to learn those things in a reasonable amount of time, train testers the same way you train your users, and make them learn testing basics. For the first part, if you are developing a professional product, I guess you already have a standard procedure. For the letter, there are books, courses, and lots of other material available at the web.

And if the usage of your software needs a fully skilled engineer, you will need your testers to have equal skills, at least some basic skills from that field. For example, when you develop a CAD software, you need people trained in technical drawing, if they are users or if they are testers. Probably your testers don't have to be as good as your users in the domain, but some knowledge of the domain will be essential. But maybe you could see this also as a chance to improve your software in a way to lower the entry skill level?