Don’t trust your test cases! There is a reason for that

Once upon a time, I liked my tests, now I don’t. But why, you may ask? Let me tell you a fictional story about a tester and his routine daily work. And who knows, maybe you recognize somebody in this story…

“Imagine that you came to a new project and your new Scrum team was created from scratch. Sounds promising, especially if all of you are very good developers. Yes, yes, you read it right – developers. Didn’t you know that we’re all developers? There is no splitting by roles in Scrum: everybody should be able to take any tasks, everybody is a developer and a tester at the same time.

So, new team + new product = new worries & headache. Let’s skip those formal procedures that every team usually starts doing at the very beginning of the product development and move on to the quality assurance part. Despite that fact that Scrum prefers working software over comprehensive documentation, test cases are needed otherwise the testing process will be chaotic and unuseful (except some cases when you’re gonna run exploratory testing). Boring but true!

It’s not a secret that you put your soul in test cases you create. It’s like your brainchild, your best creature and the embodiment of the best practices you gained during your professional career. That’s why we usually do our best while developing test cases. It makes sense also from that point of view that good test cases save our time, time of those people who’ll come after us and who have no idea what a hell is going on with a product.

What are the attributes of a good test case then? What criteria should a test case meet in order to be a good one? Well, first of all, it should be easy to understand by people who have never seen it before. It should be easy to run and should help to detect if something goes wrong with a functionality it checks (Damn it! Do such test cases exist?). Based on the criteria above you create test cases, run them, review them and modify, if necessary. You love your test cases and you rely on them, it’s unavoidable. When something goes wrong, you know that you’ll run your test cases and find the problem. Eventually, you know your test cases so well and trust them so much that you don’t need even to have a look at them while testing… Bang!

And here lies the trick. The test cases you’ve been working on for a long time blunt your vigilance. What does that mean? It means – the more you run your test cases, the more chances that you run them not properly, forgetfully, carelessly, whatever… You get used to running specific test cases and that’s why you might not run them 100% precisely as initially designed. You can call it human factor or habituation, or tiredness but there is always a chance to deviate from a test case, and this chance increases together with the number of test runs you’ve done.

Eventually, test cases which supposed to help with controlling the quality of the software, don’t do that. And you don’t even notice that often, the end user does. And you can’t believe that test cases you cherished aren’t good enough, you can’t blame them. And you shouldn’t!

What is the solution in this case? Don’t trust your test cases! Work with them like this is the very first time you see them. Read every step like you’ve never read it before. Don’t trust your test cases from the good point of view and keep tracking every detail. While working with test cases, behave like somebody else created them, not you did…”

I used to like my tests but now I don’t. From the good point of view of course. And you know, quality of the software is getting better after all…

Like this post?

Subscribe to updates from my blog, if you don't want to miss more interesting future posts and materials

Please check your email and confirm subscription

Pin It on Pinterest

Share This