How GitOps practices support modern cloud development

Infrastructure as code, or IaC, enable Devops teams to build and deploy infrastructure using similar approach and tools as with other software applications deployments, such as Git for collaboration and version control and CI/CD processes for repositories, testing, integration and code pipeline automation.

Modern cloud solution development and deployment is driven through Devops practices, or more precisely, “GitOps” for infrastructure workflow automation.

GitOps for designing, changing and deploying cloud infrastructure

GitOps is a branch of the Devops practice or culture and offers a way to automate and manage infrastructure that has been defined as code. Generally, GitOps is defined as common denominator of IaC, Git for merge requests – as the change mechanism for all infrastructure updates and CI/CD for workflow and provision automation.

GitOps applies especially towards working with container-based infrastructure environments, such as Kubernetes, and is in a way the next generation in configuration management, similar to earlier tools like Chef, Puppet and Ansible that handled on-premise and cloud-based server and virtual machine-based configurations.

Base cloud infrastructure is now increasingly provisioned as container platforms or serverless, gradually moving away from virtual machines or servers as hosts for running applications. Instead, most modern applications are built as Docker images that are provisioned inside pods running inside container clusters. This is where GitOps plays an important role.

Building IaC through declarative programming

Hyperscalers like AWS and Azure early realized the restrictions, complexity and error-prone practices of deploying and maintaining servers and infrastructure through manual processes and, therefore, developed their own vendor-specific tools for defining infrastructure.

This includes Azure ARM/Bicep and AWS CloudFormation that are YAML or JSON based declarative programming tools and methods for describing and provisioning infrastructure in the subsequent cloud platforms.

However, as many organizations want to be able to switch clouds or support multicloud environments, other IaC alternatives emerged that avoid dependency on single vendor environments. These include for example Terraform and Pulumi that are also declarative languages but support most cloud environments by supporting different vendor-specific syntaxes, enabling cloud and devops engineers to reuse the same code and templates, with different syntaxes, to run on multiple cloud platforms.

Origo’s devops team mainly leverages Terraform as a cloud agnostic way to develop cloud solutions for its customers, enabling maximum flexibility and adaptability when changes and update needs arise.

An important element of a tool like Terraform is managing different code states in the development and deployment process. The different states describe the levels of the code pipeline, from development to the actual running cloud environment and include:

• Desired state – describes the desired state of the cloud environment as described in the code.

• Known state – declared in .tfstate files and serves as a mapping mechanism between the desired state and the actual resources deployed in the cloud environment.

• Actual state – is the actual state of objects in the running environment.

When applying or updating the infrastructure, Terraform refers to the .tfstate files that perform the actual change on the environment and, by best practices, are stored on a secure storage account that the CI/CD pipeline has access to when doing automatic updates on the infrastructure.

Git and CI/CD pipelines

As with many other collaborative projects, Git serves as the primary tool for team collaboration and version control and eventually merges code, when merge requests are performed, to centralized code repositories in CI/CD, or source control management, platforms like Azure DevOps or Github Enterprise.

The CI/CD platforms store the desired code stages in different repositories, commonly segmented into dev, test, staging and production stages, and leverage release pipelines to automate the deployment process to the actual running cloud environment. For this, the pipeline needs to be connected to the actual cloud environment using service principals enabled by vendor-specific plugins.

Once changes are made on a code repository, supported by a release pipeline, the pipeline automatically picks up the changes and deploys them into the cloud environment. Before a particular cloud deployment is initiated through a release pipeline, a build process is performed that describes a plan or actions that will be performed, and it is possible to set up a test framework at this stage for the code if desired.

Eventually, this process creates a fully automated process for deploying IaC to the desired cloud environment

Flexibility and adaptability is important

Although this is a very short description of how modern cloud development and deployment is leveraged through GitOps practices, it should give a high-level overview of the actual process. Obviously, many different tools or toolchains can be used for this purpose, depending on the particular Devops team preferences and know-how or development strategy. There is no one-way of doing GitOps as it’s a methodology or set of best practices and each team needs to find its own approach that best suits their needs and projects.

To create a complete process or journey of cloud solutions, the Origo Devops practice utilizes many of the recommended best practices and tools to support its processes. Following best practices is vital and being flexible and willing to make changes to tools, methods and processes, when desirable. This will help to maintain dynamic and agile devops processes and address the continuously changing landscape of cloud infrastructure development and provisioning.