You can leverage Packer, Ansible, and Testinfra to automate the creation and testing of golden AMIs, ensuring your infrastructure is configured correctly. Here’s how!
This article covers a step-by-step process of building a golden AMI using Packer, Ansible and TestInfra. It also demonstrates how to integrate these tools to master immutable infrastructure.
In recent years, immutable infrastructure has become the most popular approach to deploying applications in the cloud. It enables developers to create secure, scalable, and repeatable infrastructure. Immutable infrastructure involves creating machine images, also known as golden AMIs, that are pre-configured and ready to run applications. These machine images are then deployed as-is and never modified, ensuring reliable and consistent infrastructure.
Before we begin, let’s quickly understand the basic terminologies and get familiar with the tools we are going to use.
- Immutable infrastructure: This approach to infrastructure management emphasises creating and deploying infrastructure in a non-changing state. In other words, an immutable infrastructure cannot be modified once created. Instead of modifying the existing infrastructure, a new version is created every time changes are made. Immutable infrastructure has several benefits, such as easier testing, more efficient deployments, and better security.
- Golden AMI: It is an Amazon machine image containing the operating system, configuration, and applications required to run an application or service. It is the foundation for creating instances in AWS. Ensuring that the AMI is up-to-date, secure, and consistent is crucial.
- Packer: This tool creates identical machine images for multiple platforms from a single source configuration. It allows you to automate the process of building machine images, which can be used for deployment, testing, and development. Packer supports multiple builders, including Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure.
- Ansible: This open source automation tool enables you to automate applications and systems’ deployment, configuration, and orchestration. It is designed to be simple, lightweight, and extensible.
- Testinfra: It is a framework that enables you to write unit tests for infrastructure. It provides a simple syntax for defining test cases, which can be used to verify the functionality and configuration of infrastructure components.
Now that we have a basic understanding of the immutable infrastructure concepts, let’s dive in and understand how to deploy Apache web server on an Amazon Linux 2 machine by building a flawless golden AMI using Packer, configuring and deploying the web server with Ansible, and testing its functionality with Testinfra. Before we begin, make sure you have the following:
- Access to AWS console to launch EC2 instance.
- Packer, Ansible, and Testinfra packages on the local machine.
- Basic understanding of Ansible, Packer and Python.
Steps to building a golden AMI
Define Packer template: First, we’ll create a Packer template that defines the AMI we want to bake in. Create a ‘webserver.pk.hcl’ file that installs ‘httpd’ package, configures a virtual host ‘helloworld.example.com’, and runs the web server on an Amazon Linux 2 AMI.
variable “app_role” { type = string }
variable “app_version” { type = string }
variable “instance_type” { type = string }
variable “root_volume_size” { type = string }
variable “aws_region” { type = string }
variable “aws_access_key” { default = “${env(“aws_access_key”)}” }
variable “aws_secret_key” { default = “${env(“aws_secret_key”)}” }
locals {
timestamp = regex_replace(timestamp(), “[- TZ:]”, “”)
ami_name = lower(“${var.app_role}-${var.app_version}”)
}
source “amazon-ebs” “webserver” {
region = “${var.aws_region}”
ami_name = “${local.ami_name}-${local.timestamp}”
ami_description = “Golden AMI To Deploy WebServer”
ssh_username = “ec2-user”
ssh_wait_timeout = “10000s”
instance_type = “${var.instance_type}”
source_ami = “ami-052f483c20fa1351a”
launch_block_device_mappings {
device_name = “/dev/sda1”
volume_size = “${var.root_volume_size}”
volume_type = “gp3”
delete_on_termination = true
}
tags = {
Name = “${var.app_role}”
}
}
build {
sources = [“source.amazon-ebs.webserver”]
provisioner “file” {
source = “test_apache.py”
destination = “/tmp/”
}
provisioner “ansible” {
playbook_file = “webserver.yaml”
}
provisioner “shell” {
execute_command = “{{.Vars}} sudo -E -S bash ‘{{.Path}}’”
inline = [
“python3 -m pytest /tmp/test_apache.py”,
]
}
}
Define Ansible playbook: In this step, we will define an Ansible playbook that installs ‘http’ package, ‘pytest-testinfra’ module, configures virtual hosts, and finally starts the ‘httpd’ service on the Amazon Linux2 machine during the baking process.
- name: WebApp Playbook hosts: all become: true tasks: - name: install_httpd yum: name: httpd state: present - name: set_listener lineinfile: path: /etc/httpd/conf/httpd.conf regexp: ‘^Listen\s+80$’ line: ‘Listen 0.0.0.0:80’ - name: create_vhost copy: content: | <VirtualHost *:80> ServerName helloworld.myexample.com DocumentRoot /var/www/html ErrorLog /var/log/httpd/error.log CustomLog /var/log/httpd/access.log combined <Directory /var/www/html> AllowOverride All Require all granted </Directory> </VirtualHost> dest: /etc/httpd/conf.d/helloworld.conf - name: create_webroot file: path: /var/www/html state: directory owner: apache group: apache mode: ‘0755’ - name: create_index_html copy: content: ‘<h1>Hello, World!</h1>’ dest: /var/www/html/index.html owner: apache group: apache mode: ‘0644’ - name: start_httpd service: name: httpd state: started enabled: true - name: install_testinfra ansible.builtin.pip: name: pytest-testinfra executable: pip3
Define the Testinfra test cases: Once the templating part is done, we will need to write a simple test case using Testinfra to ensure the AMI has ‘httpd’ package successfully installed, configured, and running fine. In short, the AMI should be in the desired state. Here is an example of a sample test case for ‘httpd’ service:
import testinfra def test_apache_is_installed(host): assert host.package(“httpd”).is_installed def test_apache_is_running(host): assert host.service(“httpd”).is_running assert host.service(“httpd”).is_enabled def test_apache_virtual_host(host): virtual_host = “”” <VirtualHost *:80> ServerName helloworld.myexample.com DocumentRoot /var/www/helloworld ErrorLog /var/log/httpd/helloworld-error.log CustomLog /var/log/httpd/helloworld-access.log combined <Directory /var/www/helloworld> AllowOverride All Require all granted </Directory> </VirtualHost> “”” assert host.file(“/etc/httpd/conf.d/helloworld.conf”).exists assert host.file(“/etc/httpd/conf.d/helloworld.conf”).contains(virtual_host) def test_apache_index_html(host): assert host.file(“/var/www/html/index.html”).exists assert host.file(“/var/www/html/index.html”).contains(“<h1>Hello, World!</h1>”)
Bake the golden AMI: Now, we can start baking the AMI once we are done with all the steps mentioned above. So let’s run the following command, which will initiate the baking process:
$ packer build -var-file=templates/webserver.pkrvars.hcl templates/webserver.pkr.hcl amazon-ebs.webserver: output will be in this color. ==> amazon-ebs.webserver: Prevalidating any provided VPC information ==> amazon-ebs.webserver: Prevalidating AMI Name: webserver-2.4-20230520103749 amazon-ebs.webserver: Found Image ID: ami-028a473e64001bfb2 ==> amazon-ebs.webserver: Provisioning with Ansible... amazon-ebs.webserver: Setting up proxy adapter for Ansible.... ******************************************************* amazon-ebs.webserver: changed: [default] amazon-ebs.webserver: amazon-ebs.webserver: PLAY RECAP ************************ amazon-ebs.webserver: default : ok=8 changed=7 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 amazon-ebs.webserver: ==> amazon-ebs.webserver: Provisioning with shell script: /tmp/packer-shell1563525277 amazon-ebs.webserver: ============================= test session starts ============================== amazon-ebs.webserver: platform linux -- Python 3.7.16, pytest-7.3.1, pluggy-1.0.0 amazon-ebs.webserver: rootdir: /tmp amazon-ebs.webserver: plugins: testinfra-7.0.0 amazon-ebs.webserver: collected 4 items amazon-ebs.webserver: amazon-ebs.webserver: ../../tmp/test_apache.py .... [100%] amazon-ebs.webserver: amazon-ebs.webserver: ============================== 4 passed in 0.20s =============================== ==> amazon-ebs.webserver: Stopping the source instance... amazon-ebs.webserver: Stopping instance ==> amazon-ebs.webserver: Waiting for the instance to stop... ==> amazon-ebs.webserver: Creating AMI webserver-2.4-20230520103749 from instance i-000ad46d12682b5c2 amazon-ebs.webserver: AMI: ami-0a78efa7e31b4d102
Deploy the golden AMI: The last step is to deploy the AMI in pre-production or production upon a successful run. Infrastructure changes can be deployed through CI/CD automation tooling like Jenkins, CircleCI, CloudFormation, or Terraform. Such automation ensures the infrastructure is consistent and easily replicated across multiple environments.
Building golden AMIs is an important step in mastering immutable infrastructure. By following the steps outlined in this guide, you can use Packer, Ansible, and TestInfra to automate the process of building and testing your AMIs and ensure that your infrastructure is always configured correctly. By doing so, you can increase the reliability and security of your infrastructure and reduce the risk of downtime or security breaches.
