Once more about writing asserts…

┬áSpoiler: I’m not going to re-invent a wheel or say extraordinary things in this post. All of the following is nothing more than well-known but passed through the prism of my experience.

Internet is overflowed by articles how to write good unit (and not only) tests. Most of them are based on the The Art of Unit Testing book which I also strongly recommend to read if you haven’t read it yet. It has lots of examples and explains main characteristics of unit tests.

I am not going to repeat the content of the above-mentioned book, really! All coincidences are random. I just want to share my approach to writing tests (not unit ones!) and maybe you find something useful.

Why not unit tests?

Well, despite I strongly believe that Test Automation guys should easily read developer’s unit test code, I think we shouldn’t write unit/integration tests for production code. There are several obvious reasons behind this opinion but… to each his own: developers should test developers code, we should assist and test almost all the rest. However, with all similarities unit tests are different. They test the lowest and most detailed level of software where abstraction is transformed into a business logic.

That’s why (I also believe) unit testing requires a specific mindset inherent to developers. And just to finish this chapter – all the above-mentioned doesn’t mean that we shouldn’t write unit tests at all. The simplest example is our own test tools, libraries and pet frameworks that everyone and each of us is writing or used to write. We act as developers in such case, and what kind of developer does not write tests for his code?

What a hell is a good Assert?

Do you remember the AAA pattern for writing unit tests? It’s a very nice and practical way, and the first step to making our tests more readable. Let me skip the explanation of the Arrange and Act parts as they’re more specific for each tester and say a couple of words about the Assert one instead. I distinguish 2 main characteristics of a good Assert:

  1. It should show clearly in code what it’s going to assert;
  2. It should explain clearly what is failed and why in the test run output/logs.

Let’s see how a typical assert looks like. Such asserts appear in many, many tests all over the world and they’re often written in rush (and obviously, not enough attention):

How much useful and meaningful information can you get from the line above without seeing the whole test class (or even a couple of classes)? Some “status” should be equal to the enum Status.Connected and that ideally means that something is connected somewhere. Doesn’t sound good if we care about tests readability and maintainability.

Another question is what will we see in the test output if the assertion above fails? Let me tell you – apparently, it’ll look:

“Expected True but was False”

In the best case scenario, if you would be running this test from IDE, you could click on the associated line of code, navigate to the assertion line, then walk through the variables and find the root cause. In the worst one, when you don’t have access to the test code and just push the “Start testing” button once per day, you’ll need to disturb other responsible people asking for help, re-run tests after investigation, etc. Wonderful pastime, isn’t it?

How can the above-mentioned assert be improved in order to fit the characteristics of a good Assert? My approach is pretty simple – always write your tests like a book and assert like a sentence in this book. For example,

What we’ve reached by refactoring code like that:

  • anyone can understand what that assert checks, even people without programming or testing skills;
  • when that assert fails, the reason is clearly shown in the test run output. For example, “Ubuntu_7 server is not pingable by the 192.168.1.7 address”.

Do these improvements satisfy the characteristics of a good Assert? I guess yes. But you have to understand also that much work should be done in a test and in a class in order to have everything readable and have assert-friendly test code, like the shown above. Of course, the example above can be and should be improved but you’ve got the idea.

What does NUnit offer?

I didn’t mention any language or framework at the beginning. But as you might have noticed, all examples were for C# / NUnit. I chose this combination because NUnit Constraints model provides really convenient way of asserting different (if not all) cases. The other unit test frameworks (like xUnit, JUnit, etc.) are more modest and do it in a more traditional way (comparison). Several examples:

All the asserts above are read like sentences from a story and this really cool because everybody is able to understand what a hell is going on. I’d say that such asserts are self-documented.

You can explore more about the Constraints model and available Constraints.

The Eternal Question: how many asserts are enough?

The old-fashion approach assumed having 1 assert per test for several reasons. First of all, because every single test should check a small piece of functionality or flow and stay away from the rest (the other tests will do). It makes our life easier and helps to isolate issues (that don’t work properly) much faster.

Second of all, if one of the several asserts fails, execution of a test is stopped and we have no idea about the rest of functionality under test. Even if a bug that was found by the failed assert is fixed, who can guarantee that we don’t have bugs in features/flows covered by the other assets in the same test?

Many of you might say: “But we know cases when we must put several asserts in a test to check things properly!” Yes, of course! Nobody stops us from putting as many asserts as possible but we have to remember a rule of thumb – each test should deliver value in verifying that something works or doesn’t.

Nowadays we are not affected by that limitation anymore. Actively developed frameworks (like JUnit 5, NUnit 3.7+) provide the Multiple Asserts feature that I’ve explained several months before. Let’s have a look how some of the above-mentioned asserts fit inside the multiple asserts block:

If you remember, all asserts in the Multiple block will be executed and reported, even if any or all of them fail. Only unhandled exceptions threaten our tests but it’s a different story.

What’s next?

Your tests are always a subject of continuous improvement. Tests are written by humans and analyzed by humans (at least for now). That’s why we should keep them readable and maintainable, and of course, our tests should actually test what they were designed for. Keep things simple and they will be simple to use…

Like this post?

Subscribe to updates from my blog, if you don't want to miss more interesting future posts and materials

Please check your email and confirm subscription

Pin It on Pinterest

Share This