The reason test fixtures exists
When the project has a good code design, there are many places where the code can be unit-tested. This means that even the smallest part of the system is testable. When that part requires dependencies on external resources, then test fixtures can help to simulate that resource. The important note here is that the test fixture is only there to help with writing a meaningful test, and to facilitate test scenarios. Test fixtures that are too deeply embedded or need a complex setup to work, are not helping the test. This is one of those cases where code coverage cannot tell what is wrong with the test.
When test fixtures are too deeply involved
When a test fixture is too deeply involved in the test, the setup becomes too much work and there is not enough test coverage to compensate. They might not be easy to spot, but they usually linger among tests with ‘arrange’ pieces that are about behind-the-scenes methods and assertions that only interact with the test fixture. An example:
This test might succeed or provide code coverage, but what is really being tested here? I see these tests all the time, and they are mostly the result of tests being written after the fact. There are several things wrong with this test. The functionality of the service is not properly being tested and the only thing being asserted is a fixture we have set up. What happens if the message service contacts other ‘delete message’ operations on the repository? The test is not strong enough to handle those and is at the same time too involved in the behind-the-scenes workings of the service.
A solution to this would be to use a fully working, in-memory repository and service result values to assert upon.
👀 Note that the test does not act on specific method names, but rather on the data. This change from method-to-data to arrange-to-assert is an important part of this improvement.
Overuse of test fixtures
What I also often see is the overuse of test fixtures. Every input is stubbed out by a replacement that is constructed in such a way that the test has nothing left to verify. This artificially created environment where the unit tests run adds nothing of value to the project, as they do not test anything of value. Mostly, these type of tests only tests the test fixture itself, instead of the service under test.
This overuse of test fixtures is rather common in poorly-maintained test suites. They are the result of quick and dirty work on a system that was not testable in the first place. What the overuse of test fixtures in the code is trying to say, is that there are too many dependencies, too much scattered functionality, and no real points of contact to know what is going on.
💡One of the things to never create a test fixture for – unless you are writing a logging system, of course – is a logger. Even unit tests need to delegate their logging to the test output.
Conclusion
Testing is not something that gets the attention or investment it deserves. Even new Microsoft technology is sometimes not fit for testing. “How can I test this?” is an important part of the project and is often forgotten when new technology or systems are introduced. Writing good tests and spotting bad ones should be common knowledge. Listen to what the tests are trying to say, since the bad ones are often the loudest.
Thanks for reading,
Stijn
Subscribe to our RSS feed