Organizations are increasingly moving their existing integration solutions to the cloud. The resulting solutions include components such as Logic Apps and Azure Functions. In order to implement the CI/CD process, organizations often use Azure Pipelines, a platform that provides pipelines for building, testing and deploying solutions, and allows you to choose between classic and YAML pipelines.
- Classic pipelines allow you to visually author your CI/CD pipelines.
- YAML pipelines provide a pipeline-as-code approach, allowing you to use modular templates. Thus, pipelines can reuse these templates to simplify their configuration. Consequently, this reduces deployment failures and increases development speed.
It is common for integration environments to contain many independent solutions, each requiring dedicated pipelines. Moreover, most integration solutions follow similar patterns. Therefore, YAML templates are a good option to simplify the configuration. But to be successful, a pipeline template must meet a number of criteria:
- Minimize solution-based configuration to reduce complexity.
- Limit template size to reduce complexity.
- Allow for versioning to increase flexibility.
- Minimize pipelines creation effort to increase development speed.
- Reduce effort to support new PaaS components.
General Structure
CI/CD pipelines use separate build and deployment stages. The first stage builds all components and creates an environment agnostic package. The second stage then deploys this package to a given environment. Azure DevOps must retain the generated packages for later use. This can either be for a rollback or a manual deployment. Yet single YAML pipelines cannot retain their packages at the time of writing. Therefore, we use separate CI and CD pipelines. This leads to a separation of component configuration: CI pipelines contain the build configuration, whereas CD pipelines hold the deployment configuration. Merging the CI and CD configuration into a third template reduces this complexity. This structure is shown below.
CI.yml and CD.yml contain the pipeline definitions and both extend Solution.yml. A push first triggers a CI pipeline run. Once completed, this run triggers the CD pipeline. Solution.yml contains the component configuration and calls Template.yml, which in turn calls task templates.
Solution Specific Configuration
CI.yml and CD.yml files only include basic information such as target environment, name and trigger. They also define the config parameter. This parameter contains the base path for the solution and signals the downstream template that it should either act as a CI or a CD pipeline.
The Solution.yml file passes on the config parameter to Template.yml and then adds two new parameters. First, it defines the variable groups for build-and-release stages. It also creates the components parameter. This parameter merges the CI/CD configuration of all components. The example below shows a case with one Azure Function.
The above approach centralizes and condenses the pipeline configuration for each solution. This increases maintainability. Moreover, automated scripts can use the above files as a template for creating new pipelines. Pipeline variables cannot contain objects, and so this approach uses parameters.
Generic Pipeline Definition
The generic Template.yml file manages all pipeline stages. Here, each stage either builds or deploys the components from Solution.yml. The entire file looks as follows:
The template starts by defining the four parameters from the Solution.yml file. It then introduces the environments parameter. This parameter contains the environment configuration for all pipelines, defining the name, service connection, dependencies, and deployment conditions. After initializing the base path of the current solution, the file determines whether it acts as a CI or as a CD pipeline.
CI Pipelines generate a single Build stage. This stage first loads the variable groups from the buildVariableGroups parameter. It then generates a job for every component. The compiler uses the type of each component to dynamically load a task template with the corresponding build tasks.
CD pipelines generate identical Deploy stages for each environment. The stage first links the deployment variable groups for the right environment and by default also loads the Infra variables group for the right environment. The stage conditions depend on the existence of a custom environment condition. The condition also includes a mechanism for performing manual deployments to a single environment. Next, the stage defines deployment jobs for all components. Each of the jobs loads a task template containing the deployment tasks for a component. Again, the component type determines what template to use.
Task template
Each component type requires a seperate build and deployment task template. The files follow the same naming scheme: <component.type>.build.yml and <component.type>.deploy.yml. Both files contain the component parameter. This parameter includes the component-specific information from Solution.yml. In addition, the deployment task template defines the environment parameter. This parameter contains the environment name and the service connection to use.
Both templates use the parameters to include component and environment-specific information in the template. In addition, the build task template uses the basepath variable from Template.yml. The component type also contains a version, providing a versioning strategy for task templates.
Conclusion
In conclusion, the pipeline design of this post satisfies the stated criteria:
- It minimizes solution-based configuration
- It allows for automated pipeline creation, using scripts.
- It provides a mechanism for performing on-demand deployment to a given environment.
- It includes a versioning strategy for build and deployment tasks
- Extension with new components does not require modification of existing files.
Therefore, this approach improves maintainability and reduces overall complexity.
Subscribe to our RSS feed