The feature gap in Azure Functions testing
In my opinion, the biggest gap in functionality that is missing is the ability to properly test an Azure Function. At the time of writing, Microsoft has not provided us with a way to unit test custom isolated Azure Functions middleware components, let alone the functionality to properly integration test your application. Only with code reflection are we able to initiate the required models to call the public methods, but there is no way to initiate the startup code from your test suite.
In-process Azure Functions are the same and maybe even worse, as there are no runtime files to initiate from your test suite.
You have to actually run the entire application in a separate process from the test process or deploy the application, before you can write your test. That is why locally defined self-contained testing is so important for Azure Functions applications. It brings test development back to the local environment where we can have quick test feedback and bring back the motivation to write tests that this feature gap has taken away from us.
Self-contained Azure Functions application
Azure Functions applications can be run locally with the Azure Functions Core tools, which you can install on your system. This tool will be called from the test suite. To fully understand this post, you must imagine yourself to be very lazy. Imagine that you want to write an integration test, a test that verifies if the Azure Functions app handles your request well. But all you want to do is run the test, starting from your favorite IDE. Everything else should be automated. This mindset brings test automation to the surface. You see your application as just another test fixture you need to set up (and teardown). Because of this, we need this CLI tool to run our Azure Functions application.
To write this as a test fixture, I usually follow this kind of pattern:
👀 Notice that the Process uses the func Azure Functions CLI tool. In Arcus, we have created an extra check with a clear message if this tool is not available on the system, so that any test developer knows they have to install the tool first. In that same mindset, there is a WaitUntilTriggerIsAvailableAsync function that makes sure that the entire test fixture is only used once the Azure Functions application is fully started. This removes any ‘retry assertions’ and improves defect localization in case something is wrong during startup.
Local and remote testing goes hand in hand
Running your Azure Functions app locally is something that people can agree on, but running it on your build server during pull request checks may cause some problems. This is especially when the app is accessing stateful services (think parallel pull requests). For Arcus, we have made sure that those tests can run in parallel, but for Invictus, we have chosen another path and run those integration tests against real-life deployed Azure Functions applications on a ‘Test’ release environment. The integration test should be aware of this:
With patterns like these, you completely hide the host system where the tests are run and you help any test fixtures based on the type of host system. If the TestHost is set to Remote, we can make sure that the TemporaryAzureFunctionsApplication is never set up later on. The test itself will never know if it tests against a remote or local system, simplifying your test, increasing test ignorance and improving motivation for Azure Functions test development. It also makes sure that you can quickly see if your changes to the Azure Functions application work. This means quick test feedback without the need to develop your code.
💡 You can use the MY_PROJECT_ENVIRONMENT environment variable to determine against which environment the integration tests are run, and load the specific appsettings.{env}.json file accordingly. See this post for more information on this topic.
Conclusion
Tests are fun to write, but sometimes you have to set up a nice workspace yourself. This post goes over the missing gap in testing Azure Functions applications. The major problem is that there are no ‘hooks’ available for robust testing, which means we have to come up with our own. By running the Azure Functions application locally and relying on a deployed application remotely, we are placing both developer experience and reliable testing first. The best of both worlds.
Thanks for reading!
Stijn
Subscribe to our RSS feed