Test assertions
A test is formed from several different parts. The focus on preparing the functionality under test and executing it is commonplace, primarily due to their initial occurrence and significance in the implementation process. However, the third and final phase — assertion — is frequently overlooked. This phase is arguably the most crucial in determining the success or failure of the test. Consequently, the art of crafting effective test assertions is essential in test writing.
One of the biggest problems with assertions is that they are often written so we cannot determine the reason for a test failure without looking at the code. A good assertion makes sure that we have an idea of where the implementation went wrong, purely by looking at the failure message. Usually, this is about giving context.
Unquote
Unquote is an assertion library that provides a meaningful reduction of a given complex calculation. This is great when the complexity of the test lies in the interaction with the implementation, not in the input. Unquote will provide several reduction steps in the test failure so that the end result failure becomes clear by looking at the inputs. It does this by inspecting a F# quoted expression. Adding this to your test is fairly easy. Look at how a possible test failure is enhanced in the following restaurant test by adding test <@ ... @> :
Unquote will show the reduction steps on how we get to the test failure. Look how the Error becomes visible in the reduction. Together with the logging of FsCheck, this becomes a very context-loaded assertion message that shows the correlation between the input and the output of the implementation.
FsCheck
Unquote is great when we want to focus the attention on the implementation, not the input. The reason for this is that with complex or larger inputs, Unquote’s output will be too specific. The previous example demonstrated this problem by giving us the guest names and the full reservation dates. In the cases where the complexity lies in the input and how it relates to the output, FsCheck labels are better suited for the task. They can add additional context purely based on the kind of inputs that were generated.
👀 Look how a simple label on the current reservation quantity on a specific date can help a lot in defect determination:
The great thing about FsCheck is that it will ensure that we have the most simple input in order to let the test fail. Here is another example with an even bigger input or without good domain-related shrinking:
Conclusion
Tests are all about providing proof with meaningful results. Unquote can help when the test inputs are small and we want to verify a certain functionality or combination of functionality. This can be very useful when verifying inverse, round-trip, or other kinds of test properties. FsCheck labels are more suited if we want to give meaningful parameters with complex inputs. These will help us to understand the input better and find the defect quicker.
Note that in both cases, the actual ‘Act’ part of the test does not alter. This is an important factor as we do not want to change meaningful interactions with the implementation for the sake of assertions. The assertions should be flexible enough.
Thanks for reading!
Stijn
Subscribe to our RSS feed