Why You Should Deploy Immutable Infrastructure on the Cloud

Even as DevOps and cloud adoption have increased significantly in recent years, the use of immutable infrastructure has also become pretty popular. This article outlines the core benefits of deploying immutable infrastructure on the cloud.

Immutable infrastructure refers to a state where the infra components like VMs, servers and containers are never modified once deployed. In case any changes are required, the existing components are destroyed and replaced by new ones.

In the real world, we often make changes in security patching and while deploying a new application version, as well as OS-specific changes. All these changes are made ‘in-place’ or with the help of configuration management tools like Chef, Ansible, Puppet, or SaltStack. That makes the whole infrastructure unstable and prone to configuration drift. This is where the concept of immutable infrastructure comes in.

Under this approach, a golden (master) image of a server is created that has all the required software stacks preconfigured. The same image is used to provision new servers. If any changes are required in the configuration of the servers, a new image is baked, tested, and rolled out across environments.

Some of the benefits of this approach are listed below.

• Creates consistency across environments (dev/stage/prod), as all the servers are provisioned with the help of a particular golden image.
• Makes infrastructure more stable because everything is immutable, as ‘in-place’ changes are not permitted. Additionally, eliminates the risk of configuration drift.
• Simplifies the deployment process because all the components are preconfigured on the server. Also makes rollback safer since the same process used to deploy the new version is used to roll back to older versions.
• Makes debugging much faster during production issues since all the servers, provisioned using golden images, are properly versioned and well documented.
• Enhances security as the images are baked while keeping compliances in mind and validated after thorough testing.Finally, embraces Agile software deployment in practice by implementing DevOps principles.
• Terraform, Vagrant, Docker and Packer are some of the open source tools available for creating and managing immutable infrastructure on the cloud.

The challenge of using immutable infrastructure

One of the core practices of DevOps is ‘continuous delivery’, wherein every single change is promoted from development to the production environment automatically. This sounds pretty fascinating and rather suitable for releasing a new version of an application, but in the real world, bringing it into practice is quite challenging, primarily when the enterprises fall under one of the following criteria.

Heterogeneous environments: Here, applications are deployed on multiple environments (i.e., dev, stage, preprod or prod) or on more than one cloud (i.e., AWS, GCP or Azure).
Infrastructure auditing: Lots of security tickets are received from SecOps routinely, which require immediate attention to fix the requested vulnerabilities.

Monolith applications: The applications deployed on the cloud still follow the monolith design pattern, and decoupling them is nearly impossible at the moment.

Complex tooling: The toolchain designed to promote changes from development to production environments is quite complex and requires manual intervention while making changes.

Since there are no clearly defined design practices to automate the deployment of immutable infrastructure, enterprises still prefer to manage the infrastructure in traditional ways, with the help of configuration management tools like Chef, Ansible, Puppet or SaltStack.

Solution: Automate immutable infrastructure deployments

Now that we have covered the core concepts of immutable infrastructure and the barriers to bringing it into practice, let us try and understand the patterns, processes, and practices needed to deploy it in any cloud environment.

Figure 1 illustrates the logical representation of the entire workflow, where the different phases of a pipeline are mentioned. The pipeline starts its execution by fetching the vanilla (base) machine image from the respective cloud repository. Then it pulls the latest application code commits, builds, and compiles. Upon successful validation, it creates a golden image from a running server, and publishes it to cloud repositories. And, finally, the golden image is deployed to the respective cloud environments.

The table above lists some excellent open source tools that can be used to implement the design workflow illustrated in Figure 1.

Deploying a demo application on AWS cloud

Now, let’s implement the deployment design illustrated in Figure 1 and use it for launching an application instance on AWS.

The first step is to set up a Jenkins server and configure AWS credentials on it. Then, install Packer and Terraform binaries on the Jenkins server to launch the application instance in the AWS environment. Next, create a Jenkins job using the declarative pipeline given below:

pipeline {
    stages {
        // Pull latest code
        stage(‘Code Check-in’) {
            steps { 
                checkout([$class: ‘GitSCM’, branches: [[name: ‘*/main’]], userRemoteConfigs: [[url: ‘https://github.com/vijayg92/immutable-infrastructure’]]])
                sh(‘ls -ltr’)
            }
        }
        // Validate Packer template
        stage(‘Template Validation’) {
            steps { 
                withCredentials([str
ing(credentialsId: ‘AWS_ACCESS_KEY_ID’, variable: ‘AWS_ACCESS_KEY_ID’), string(credentialsId: ‘AWS_SECRET_ACCESS_KEY’, variable: ‘AWS_SECRET_ACCESS_KEY’)]) {
                    sh script: ‘cd packer/webserver/ && /opt/tools/packer validate -var-file=vars.json app.json’, returnStatus: true
                }
            }
        }
        // Build Golden Image
        stage(‘Build Golden Image’) {
            steps { 
                withCredentials([string(credentialsId: ‘AWS_ACCESS_KEY_ID’, variable: ‘AWS_ACCESS_KEY_ID’), string(credentialsId: ‘AWS_SECRET_ACCESS_KEY’, variable: ‘AWS_SECRET_ACCESS_KEY’)]) {
                    sh script: ‘cd packer/webserver/ && /opt/tools/packer build -var-file=vars.json app.json’, returnStatus: true
                }
            }
        }
        // Validate Terraform Templates
        stage(‘Validate’) {
            steps { 
                withCredentials([string(credentialsId: ‘AWS_ACCESS_KEY_ID’, variable: ‘AWS_ACCESS_KEY_ID’), string(credentialsId: ‘AWS_SECRET_ACCESS_KEY’, variable: ‘AWS_SECRET_ACCESS_KEY’)]) {
                    sh script: ‘cd terraform/webserver/ && /opt/tools/terraform plan -auto-approve’, returnStatus: true
                }
            }
        }
        // Deploy golden ami to AWS cloud
        stage(‘Deploy’) {
            steps { 
                withCredentials([string(credentialsId: ‘AWS_ACCESS_KEY_ID’, variable: ‘AWS_ACCESS_KEY_ID’), string(credentialsId: ‘AWS_SECRET_ACCESS_KEY’, variable: ‘AWS_SECRET_ACCESS_KEY’)]) {
                    sh script: ‘cd terraform/webserver/ && /opt/tools/terraform apply -auto-approve, returnStatus: true
                }
            }
        }
    }
}

Finally, trigger the pipeline job, and wait for it to finish. If everything goes well, as shown in Figure 2, the application instance should be up and running in the AWS cloud (Figure 3).

The benefits of immutable infrastructure are not just restricted to faster deployments, safer rollbacks, enhanced security and scaling, but allow service owners like SysOps, SecOps, SREs, developers, etc, to move away from older, time-consuming ways of managing servers.