Test Infected - Test Ignorance

Introduction

In this part of the Test Infected series, I will talk about the ignorance of tests and how we can achieve more ignorance.

The short answer: if it doesn’t contribute to the test, hide/remove it!

The reason I wrote this post is because I see many tests with an overload of information embedded, and with some more background information people may increase the ignorance of their tests.

Ignorance

Test-Driven Discovery

Sometimes people write tests just because they are obliged to do so. Only when someone is looking over their shoulder they write tests, in any other circumstances they don’t. It’s also crazy to see people abandon their practices (TDD, Merciless Refactoring, …) the moment there is a crisis or a need for a quick change or something else that causes stress to people.

The next time you’re in such a situation, look at yourself and evaluate how you react. If you don’t stick with your practices in those situations, then do you really trust your practices at all?If you don’t use your practices in your stress situations, abandon them, because they wouldn’t work for you (yet).
This could be a learning moment for the next time.

Now, that was a long intro to come to this point: what after we have written our tests (in a Test-First/Last approach)?

Test Maintenance

In my opinion, the one thing Test Ignorance is about, is Test Maintenance. When there’s some changes to the SUT (System Under Test), how much do you have to change of the production code and how much of the test code?

When you (over)use Mock Objects (and Test Doubles in general), you can get in situations that Gerard Meszaros calls Overspecified Software. The tight-coupling between the tests and the production code is causing this Smell.

But that’s not actually the topic I want to talk about (at least not directly). What I do want to talk about are all those tests with so much information in them that every method/class/… is Obscured.

People read books about patterns, principles, practices… and try to apply them to their Production Code, but forget their Test Code.

Test Code should be as clear than the Production Code.

If your method in production has 20 lines of code and people always lose time to read and reread it… (how many times do you reread a method before you refactor?), you refactor it to smaller parts to improve usability, readability, intent…

You do this practice in your production code; so, why wouldn’t you do this in your test code?

I believe one of the reasons people sometimes abandon their tests, is because people think they get paid for production code (and not because of their lack of discipline). It’s as simple as that. But remember that you get paid for stable, maintainable, high quality software and you can’t simply deliver that without tests that are easily maintainable.

“Ignorance is Bliss” Patterns

“I know this steak doesn’t exist. I know that when I put it in my mouth, the Matrix is telling my brain that it is juicy and delicious. After nine years, you know what I realize? Ignorance is bliss.”

– Cypher (The Matrix)

Now, when you understand that your test code is just as much important than your production code, we can start by defining our Ignorance in our tests.

There are several Test Patterns in literature that support this Test Ignorance, so I’ll give you just the concepts and some quick examples.

This section is about readability and how we can improve this.

Unneccessary Fixture

The one place where you could start and where it is most obvious there’s a Test Smell, is the Fixture Setup. Not only can this section be enormous (I’ve seen gigantic fixtures) and hard to grasp, there’re also hard to change and so, to maintain.

Look at the following example. We need to setup a “valid” customer before we can insert it into the Repository. In this test, do I really need to know what all the different items are to make an invalid customer. Do we need all of them? Maybe it’s just the id that’s missing, but that could be autogenerated, or maybe the address doesn’t exist, …

Only show what I need to make the test pass.

We can change the example with a Parameterized Creation Method as an example of the One Bad Attribute Test Pattern. In the future, we could also parameterize the other properties if we want to test some functionality that depends on this information. If this isn’t the case, we can leave these initializations inside the Creation Method for the customer instead of polluting the test with this unnecessary information.

Now, if we want to act as a fully Test Infected person, we can also Test-Drive these Creation Methods. Next time you analyze the code coverage of your code, include the test projects and also make tests for these methods! This will increase your Defect Localization. If there’s a problem with your fixture and not the test that uses this fixture, you will see this in your failed tests and know that you have a problem with the Fixture and not the test itself.

Also note that this newly created method is only accessible within this test class. If we want to write tests with the same Fixture, we can extract this logic in its own class.

In either way, we have made our intentions clear to the test reader. I always try to ask the following question to the test: “Do you really care if you know this?”.

Again, the test can be made clearer if we send the argument that makes the customer invalid with it, so we know what’s the cause why the customer isn’t inserted. If we move the “id” somewhere else, we won’t know what causes it and would made the test more Obscure.

I see some reasons why a Test Fixture can be big:

• The Fixture has a lot of “setup-code” in place because the SUT is doing too much. Because the SUT is doing all these steps, we must build our Fixture with a lot of info and behavior. Otherwise, the SUT will see the Fixture as invalid.
• The Fixture is the smallest possible for exercising the SUT and the SUT is a Complete Abstraction, but needs nonetheless a Fixture that needs some lines to setup before the Fixture is valid.
• The Fixture contains some unnecessary information that doesn’t contribute the result of the test but is embedded in the test anyway.
• …

So, there are a lot of different possibility why a Fixture can be big and the solution is for all these situations the same: make the Fixture as small as possible + only add the information to the test, relevant to the result of the test . Contribute or get out.

Now, if you move ALL the Fixture code somewhere else (so extracting too much), you also have a problem. Test readers will now see some Magic Fixtures in place that act as Mystery Guests which can result in Fragile Tests.

Obscured by Eagerness

Sometimes, I encounter tests that are “Obscured by Eagerness”. A Test can be “obscure” for different reasons. One can be because we want to assert too much in a single test, another can be because we want to “set-up” too much in a single run, and yet another can be because we combine tests in a single test run by exercising multiple actions on the SUT.

To summarize:

• Eager Assertion: assert on too many state and/or behavior in a single run.
• Eager Fixture: set up unnecessary fixture (see previous section).
• Eager Exercises: exercise multiple actions on the SUT to combine tests.

I’ve seen people defend tests with more than 20 assert statements because they still tested a “single unit” outcome. Sometimes you have functionality that looks like you have to write 20 assert statements or more, but instead of writing those statements you should ask yourself: What are you trying to test?

By explicitly asking yourself this question, you often come up with surprising results.

Because the assert-phase of the test (in a Four Phase Test) is important to verify the state of the test (failed or succeed), I always try to start by writing this phase first. It forces you to think about what you trying to test and not what you need to set up as Fixture. By writing this phase first, you’re writing your test from bottom to top and only define what you really need. This way (like writing tests for your production code), you only write what you need.

Previous snippet is a perfect example of how we can abuse the Assert-Phase. By placing so many asserts in a single spot, we obscure what we really trying to test. We need to test if the message is serialized correctly; so instead of manually getting each element, why not assert on the whole xml?

We create an expected xml string and verify if this is the same as the actual serialized xml string.

Conclusion

Writing tests should be taken as serious as the production code, only then we can have maintainable software solutions where developers are eager to run tests instead of ignoring them.

The next time you write a test, try to think firmly about the test. What should I know, what do I find important to exercise the SUT, what do I expect… this way you can determine what items are important and which aren’t.

I sometimes “pretend” to be the test case:

“Do I care how this Fixture is set up?”
“Must I know exactly how to assert all these items?”
“Have I any interest of how a ‘valid’ object looks like?”
“What do I really want to test and what information only pollutes this?”
“Do I care that these actions must be executed before the exercise of the test?”
…

Tests only need to know what they need to exercise the SUT, nothing more, but equally important: nothing less!