In this 24th article in the DevOps series, we will learn how to set up HAProxy as a load balancer for multiple Nginx Web servers using Ansible.
HAProxy is free, open source, highly available, load balancer software written by Willy Tarreau in 2000. It is implemented in the C programming language. It is known for its high performance and is extremely reliable and secure. It supports both Layer 4 (TCP) and Layer 7 (HTTP) based application load balancing, and is released under the GPLv2 licence. Nginx is a Web server created by Igor Sysoev, and is also written in the C programming language. It can be used as a reverse proxy, mail proxy and as an HTTP cache. It was first released in 2004 and uses the 2-clause BSD licence.
Setup
The HAProxy and Nginx installations use CentOS 7 (x86_64) as their base operating system. A single instance is launched using KVM for running HAProxy. A couple of CentOS VMs are provisioned to install and configure Nginx. The centos users in all the VMs are given sudo access using the visudo command. SELinux is disabled for the exercise.
The host system is a Parabola GNU/Linux-libre x86_64 system and Ansible is installed using the distribution package manager. The version of Ansible used is 2.6.0 as indicated below:
$ ansible --versionansible 2.6.0config file = /etc/ansible/ansible.cfgconfigured module search path = [‘/home/guest/.ansible/plugins/modules’, ‘/usr/share/ansible/plugins/modules’]ansible python module location = /usr/lib/python3.6/site-packages/ansibleexecutable location = /usr/bin/ansiblepython version = 3.6.5 (default, May 11 2018, 04:00:52) [GCC 8.1.0] |
The Ansible inventory, files and playbook directories are created on the host system as follows:
ansible/inventory/kvm//playbooks/configuration//files/ |
The inventory/kvm/inventory file contains the following:
[front]haproxy ansible_host=192.168.122.113 ansible_connection=ssh ansible_user=centos ansible_password=password[web1]nginx1 ansible_host=192.168.122.248 ansible_connection=ssh ansible_user=centos ansible_password=password[web2]nginx2 ansible_host=192.168.122.147 ansible_connection=ssh ansible_user=centos ansible_password=password[web:children]web1web2 |
The ‘front’ group contains the HAProxy instance information. The couple of Nginx Web servers are grouped together under a ‘web’ group, and can also be accessed individually. You can test connectivity from Ansible to the CentOS guest VMs using the following Ansible commands:
$ ansible -i inventory/kvm/inventory haproxy -m pinghaproxy | SUCCESS => {“changed”: false,“ping”: “pong”}$ ansible -i inventory/kvm/inventory nginx1 -m pingnginx1 | SUCCESS => {“changed”: false,“ping”: “pong”}$ ansible -i inventory/kvm/inventory nginx2 -m pingnginx2 | SUCCESS => {“changed”: false,“ping”: “pong”}$ ansible -i inventory/kvm/inventory web -m pingnginx1 | SUCCESS => {“changed”: false,“ping”: “pong”}nginx2 | SUCCESS => {“changed”: false,“ping”: “pong”} |
Nginx
We will first set up the Nginx Web servers. The EPEL release RPM is installed and the HAProxy server IP address is added to /etc/hosts file on the instances. The YUM package manager is used to install Nginx, and then Port 80 is allowed through the firewall. We then start the Nginx Web server and wait for the server to listen on port 80. The Ansible playbook to install and set up Nginx is as follows:
---- name: Setup Nginx web serverhosts: webbecome: yesbecome_method: sudogather_facts: yestags: [web]tasks:- name: Install EPEL Releasepackage:name: epel-releasestate: present- name: Add HAProxy server to /etc/hostslineinfile:path: /etc/hostsline: “{{ hostvars[‘haproxy’].ansible_host }} haproxy”state: present- name: Install Nginxpackage:name: nginxstate: present- name: Allow port 80shell: iptables -I INPUT -p tcp --dport 80 -m state --state NEW,ESTABLISHED -j ACCEPT- name: Start Nginx serversystemd:name: nginxenabled: yesstate: started- name: Wait for server to startwait_for:port: 80 |
The above playbook can be invoked using the following command:
$ ansible-playbook -i inventory/kvm/inventory playbooks/configuration/frontend.yml --tags web -vv -K |
The -vv represents the verbosity in the Ansible output. You can use up to four ‘v’ s for a more detailed output. The -K option prompts for the sudo password for the guest CentOS user account.
You can now open the Nginx Web server URLs (http://192.168.122.248 and http://192.168.122.147) in a browser to see the default Nginx home page as shown in Figure 1.
HAProxy
The YUM package repository needs to be updated before proceeding to install HAProxy. The Nginx server IP addresses and hostnames are added to the /etc/hosts file on the instances. The default /etc/haproxy/haproxy.cfg directory is backed up and a new haproxy.cfg file is created, whose file contents are shown below:
globallog 127.0.0.1 local2chroot /var/lib/haproxypidfile /var/run/haproxy.pidmaxconn 4000user haproxygroup haproxydaemonstats socket /var/lib/haproxy/statsdefaultsmode httplog globaloption httplogoption dontlognulloption http-server-closeoption forwardfor except 127.0.0.0/8option redispatchretries 3timeout http-request 10stimeout queue 1mtimeout connect 10stimeout client 1mtimeout server 1mtimeout http-keep-alive 10stimeout check 10smaxconn 3000listen haproxy-monitoring *:8080mode httpoption forwardforoption httpclosestats enablestats show-legendsstats refresh 5sstats uri /statsstats realm Haproxy\ Statisticsstats auth admin:passwordstats admin if TRUEdefault_backend app-mainfrontend mainbind *:80option http-server-closeoption forwardfordefault_backend app-mainbackend app-mainbalance roundrobinoption httpchk HEAD / HTTP/1.1\r\nHost:\ localhost{% for item in groups[‘web’] %}server {{ hostvars[item][‘inventory_hostname’] }} {{ hostvars[item][‘ansible_default_ipv4’][‘address’] }}:80 check{% endfor %} |
The rsyslog software will be used to collect the HAProxy logs. The /etc/rsyslog.conf configuration is updated to load the UDP Syslog Input Module (imudp) and to run a UDP server on Port 514. A /etc/rsyslog.d/haproxy.conf configuration file is created, the contents of which are as follows:
local2.=info /var/log/haproxy-access.loglocal2.notice /var/log/haproxy-info.log |
The firewall is updated to allow Port 8080, and the rsyslog server is restarted. The final step is to start the HAProxy server and wait for it to listen on Port 8080. The complete Ansible playbook to install and configure HAProxy is given below:
- name: Setup HAProxyhosts: frontbecome: yesbecome_method: sudogather_facts: yestags: [haproxy]tasks:- name: Yum updateyum: name=* update_cache=yes state=present- name: Install HAProxypackage:name: haproxystate: present- name: Add Nginx servers to /etc/hostslineinfile:path: /etc/hostsline: “{{ hostvars[item][‘ansible_default_ipv4’][‘address’] }} {{ hostvars[item][‘inventory_hostname’] }}”state: presentwith_items: “{{ groups[‘web’] }}”- name: Backup default haproxy.cfgcommand: mv haproxy.cfg haproxy.cfg.origargs:chdir: /etc/haproxy- name: Create new haproxy.cfgtemplate:src: ../../files/haproxy.cfg.j2dest: /etc/haproxy/haproxy.cfgmode: 0644- name: Update /etc/rsyslog.conflineinfile:path: /etc/rsyslog.confregexp: “{{ item.regexp }}”line: “{{ item.line }}”with_items:- { regexp: ‘#\$ModLoad imudp’, line: ‘$ModLoad imudp’ }- { regexp: ‘#\$UDPServerRun 514’, line: ‘$UDPServerRun 514’ }- name: Create /etc/rsyslog.d/haproxy.confcopy:src: ../../files/haproxy.confdest: /etc/rsyslog.d/haproxy.confmode: 0644- name: Allow port 8080shell: iptables -I INPUT -p tcp --dport 8080 -m state --state NEW,ESTABLISHED -j ACCEPT- name: Restart rsyslogsystemd:name: rsyslogstate: restarted- name: Start HAProxy serversystemd:name: haproxyenabled: yesstate: started- name: Wait for server to startwait_for:port: 8080 |
A sample execution of the above playbook is as follows:
$ ansible-playbook -i inventory/kvm/inventory playbooks/configuration/frontend.yml --tags haproxy -KSUDO password:PLAY [Setup Nginx web server] *******************************TASK [Gathering Facts] **************************************ok: [nginx1]ok: [nginx2]PLAY [Setup HAProxy] ****************************************TASK [Gathering Facts] **************************************ok: [haproxy]TASK [Yum update] *******************************************ok: [haproxy]TASK [Install HAProxy] **************************************changed: [haproxy]TASK [Add Nginx servers to /etc/hosts] **********************changed: [haproxy] => (item=nginx2)changed: [haproxy] => (item=nginx1)TASK [Backup default haproxy.cfg] ***************************changed: [haproxy]TASK [Create new haproxy.cfg] *******************************changed: [haproxy]TASK [Update /etc/rsyslog.conf] *****************************changed: [haproxy] => (item={‘regexp’: ‘#\\$ModLoad imudp’, ‘line’: ‘$ModLoad imudp’})changed: [haproxy] => (item={‘regexp’: ‘#\\$UDPServerRun 514’, ‘line’: ‘$UDPServerRun 514’})TASK [Create /etc/rsyslog.d/haproxy.conf] *******************changed: [haproxy]TASK [Allow port 8080] **************************************changed: [haproxy]TASK [Restart rsyslog] **************************************changed: [haproxy]TASK [Start HAProxy server] *********************************changed: [haproxy]TASK [Wait for server to start] *****************************ok: [haproxy]PLAY RECAP **************************************************haproxy : ok=12 changed=9 unreachable=0 failed=0nginx1 : ok=1 changed=0 unreachable=0 failed=0nginx2 : ok=1 changed=0 unreachable=0 failed=0 |
Testing
You can open the HAProxy Web page using http://192.168.122.113:8080/stats and you will be prompted to log in as shown in Figure 2.
You can use the credentials (admin:password) as specified in /etc/haproxy/haproxy.cfg to log in, and you will see the HAProxy stats page as shown in Figure 3.
You can make multiple requests to the HAProxy front-end server in the browser or using curl http://192.168.122.113:8080 from the command line. You will observe that the requests are being sent to both the Nginx Web servers in a round-robin fashion, which can be seen in the app-main section of the HAProxy stats page.
