IT automation is in its larger sense—the processes and software that help with the management of the IT infrastructure (servers, networking, and storage). In the current shift, we are assisting to a huge implementation of such processes and software.

Ansible is an agent-less IT automation tool developed in 2012 by Michael DeHaan, a former Red Hat associate. The Ansible design goals are for it to be: minimal, consistent, secure, highly reliable, and easy to learn. The Ansible company has recently been bought out by Red Hat and now operates as part of Red Hat, Inc.

Ansible primarily runs in push mode using SSH, but you can also run Ansible using ansible-pull, where you can install Ansible on each agent, download the playbooks locally, and run them on individual machines. If there is a large number of machines (large is a relative term; in our view, greater than 500 and requiring parallel updates), and you plan to deploy updates to the machines in parallel, this might be the right way to go about it.

Secure Shell (also known as SSH) is a network service that allows you to login and access a shell remotely in a fully encrypted connection. The SSH daemon is today, the standard for UNIX system administration, after having replaced the unencrypted telnet. The most frequently used implementation of the SSH protocol is OpenSSH.

In the last few months, Microsoft has shown an implementation (at the time of writing) of OpenSSH for Windows.

Since Ansible performs SSH connections and commands in the same way any other SSH client would do, no specific configuration has been applied to the OpenSSH server.

To speed up default SSH connections, you can always enable ControlPersist and the pipeline mode, which makes Ansible faster and secure.

We will try and compare Ansible with Puppet and Chef during the course of this book since many people have good experience with those tools. We will also point out specifically how Ansible would solve a problem compared to Chef or Puppet.

Ansible, as well as Puppet and Chef, are declarative in nature and are expected to move a machine to the desired state specified in the configuration. For example, in each of these tools, in order to start a service at a point in time and start it automatically on restart, you would need to write a declarative block or module; every time the tool runs on the machine, it will aspire to obtain the state defined in your playbook (Ansible), cookbook (Chef), or manifest (Puppet).

The difference in the toolset is minimal at a simple level but as more situations arise and the complexity increases, you will start finding differences between the different toolsets. In Puppet, you need to take care of the order, and the Puppet server will create the sequence of instructions to execute every time you run it on a different box. To exploit the power of Chef, you will need a good Ruby team. Your team needs to be good at the Ruby language to customize both Puppet and Chef, and there will be a bigger learning curve with both of the tools.

With Ansible, the case is different. It uses the simplicity of Chef when it comes to the order of execution, the top-to-bottom approach, and allows you to define the end state in YAML format, which makes the code extremely readable and easy for everyone, from development teams to operations teams, to pick up and make changes. In many cases, even without Ansible, operations teams are given playbook manuals to execute instructions from, whenever they face issues. Ansible mimics that behavior. Do not be surprised if you end up having your project manager change the code in Ansible and check it into Git because of its simplicity!

Installing Ansible is rather quick and simple. You can use the source code directly, by cloning it from the GitHub project (https://github.com/ansible/ansible), install it using your system's package manager, or use Python's package management tool (pip). You can use Ansible on any Windows, Mac, or UNIX-like system. Ansible doesn't require any databases and doesn't need any daemons running. This makes it easier to maintain Ansible versions and upgrade without any breaks.

We'd like to call the machine where we will install Ansible our Ansible workstation. Some people also refer to it as the command center.

$ sudo dnf install ansible

$ sudo yum install ansible

$ sudo apt-get install ansible

$ brew update
$ brew install ansible

$ sudo easy_install pip

$ sudo pip install ansible

$ ansible --version

ansible 2.0.2

$ git clone git://github.com/ansible/ansible.git
Cloning into 'ansible'...
remote: Counting objects: 116403, done.
remote: Compressing objects: 100% (18/18), done.
remote: Total 116403 (delta 3), reused 0 (delta 0), pack-reused 116384
Receiving objects: 100% (116403/116403), 40.80 MiB | 844.00 KiB/s, done.
Resolving deltas: 100% (69450/69450), done.
Checking connectivity... done.
$ cd ansible/
$ source ./hacking/env-setup
Setting up Ansible to run out of checkout...
PATH=/home/vagrant/ansible/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/vagrant/bin
PYTHONPATH=/home/vagrant/ansible/lib:
MANPATH=/home/vagrant/ansible/docs/man:
Remember, you may wish to specify your host file with -i
Done!

$ sudo easy_install pip
<A long output follows>

$ sudo pip install paramiko PyYAML jinja2 httplib2
Requirement already satisfied (use --upgrade to upgrade): paramiko in /usr/lib/python2.6/site-packages
Requirement already satisfied (use --upgrade to upgrade): PyYAML in /usr/lib64/python2.6/site-packages
Requirement already satisfied (use --upgrade to upgrade): jinja2 in /usr/lib/python2.6/site-packages
Requirement already satisfied (use --upgrade to upgrade): httplib2 in /usr/lib/python2.6/site-packages
Downloading/unpacking markupsafe (from jinja2)
  Downloading MarkupSafe-0.23.tar.gz
  Running setup.py (path:/tmp/pip_build_root/markupsafe/setup.py) egg_info for package markupsafe
Installing collected packages: markupsafe
  Running setup.py install for markupsafe
    building 'markupsafe._speedups' extension
    gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.6 -c markupsafe/_speedups.c -o build/temp.linux-x86_64-2.6/markupsafe/_speedups.o
    gcc -pthread -shared build/temp.linux-x86_64-2.6/markupsafe/_speedups.o -L/usr/lib64 -lpython2.6 -o build/lib.linux-x86_64-2.6/markupsafe/_speedups.so
Successfully installed markupsafe
Cleaning up...

[node ansible]$ git checkout v2.0.2
Note: checking out 'v2.0.2'.
[node ansible]$ ansible --version
ansible 2.0.2 (v2.0.2 268e72318f) last updated 2014/09/28 21:27:25 (GMT +000)

To be able to learn Ansible, we will need to make quite a few playbooks and run them.

It's possible to create a test environment with cloud providers in a few seconds, but often it is more useful to have those machines locally. To do so, we will use Kernel-based Virtual Machine (KVM) with Quick Emulator (QEMU).

The first thing will be installing qemu-kvm and virt-install. On Fedora it will be enough to run:

$ sudo dnf install -y @virtualization

On Red Hat/CentOS/Scientific Linux/Unbreakable Linux it will be enough to run:

$ sudo yum install -y qemu-kvm virt-install virt-manager

If you use Ubuntu, you can install it using:

$ sudo apt install virt-manager

On Debian, you'll need to execute:

$ sudo apt install qemu-kvm libvirt-bin

For our examples, I'll be using CentOS 7. This is for multiple reasons; the main ones are:

At the time of writing this book, the most recent CentOS cloud image is http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1603.qcow2, So let's download this image with the help of the following command:

$ wget http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1603.qcow2

Since we will probably need to create many machines, it's better if we create a copy of it so the original one will not be modified:

$ cp CentOS-7-x86_64-GenericCloud-1603.qcow2 centos_1.qcow2

Since the qcow2 images will run cloud-init to set up the networking, users, and so on, we will need to provide a couple of files. Let's start by creating a metadata file for networking:

instance-id: centos_1
local-hostname: centos_1.local
network-interfaces: |
  iface eth0 inet static
  address (An IP in your virtual bridge class)
  network (The first IP of the virtual bridge class)
  netmask (Your virtual bridge class netmask)
  broadcast (Your virtual bridge class broadcast)
  gateway (Your virtual bridge class gateway)

To find your virtual bridge data, you have to look for a device that has the name virbrX or something similar, in my case it is virtbr0, so I can find all of its information using the following command:

$ ip addr show virbr0

The previous command will give this as an output:

5: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:38:1a:e6 brd ff:ff:ff:ff:ff:ff
    inet 192.168.124.1/24 brd 192.168.124.255 scope global virbr0
       valid_lft forever preferred_lft forever

So, for me the meta-data file looks like the following:

instance-id: centos_1
local-hostname: centos_1.local
network-interfaces: |
  iface eth0 inet static
  address 192.168.124.10
  network 192.168.124.1
  netmask 255.255.255.0
  broadcast 192.168.124.255
  gateway 192.168.124.1

This file will set up the eth0 interface of the virtual machine at boot time. We also need another file (user-data) to set up the users properly:

users:
- name: (yourname)
  shell: /bin/bash
  sudo: ['ALL=(ALL) NOPASSWD:ALL']
  ssh-authorized-keys:
  - (insert ssh public key here)

For me, the file looks like the following:

users:
- name: fale
  shell: /bin/bash
  sudo: ['ALL=(ALL) NOPASSWD:ALL']
  ssh-authorized-keys:
  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRoZzfNif+wXFqzsmvHg4jJt8+ZO/dQxm5k7pXYAwdWVbiFrZYGhMQl5FPfzC7rkDaC31fod3Y85QkQVgNKCVYUy5QR5LfxUjSQDv+y2Nfao4be/BKla0ffc7JVSzFFAELGGDLn1lMN0e0D9syqQbKgSRdOdvweq/0Et3KNIF9e7XgEdSuAHls17NDtMkWUfyi5yvEtdtMcp9gO4OlG6Vh0iCXOdx+f0QA2hh1JnvePvzJ4a8CeckN5JwL7Q027nlsHPBYq9K1jvv+diUs48FflPJI4fgMq3Zo7zyCpf8qE7Dlx+u7OvR5kxNdrpnOsDgHeAGNkrzfcmxU7kbU29NX4VFgWd0sdlzu1nOWFEH7Cnd547tx5VFxBzJwEAUCh7QSiU2Ne/hCnjFkZuDZ5pN4pNw+yu+Feoz79gV/utoLHuCodYyAvSQlQ7VSfC+djLD/9wHC2yGksvc9ICnSUv3JyQEEEG4K26z6szF9+a3vU0qIq7YYa8QHgWIHtzSxztYRIWJOzTZlwyuNmhbRNYDaMC5BMzvQ8JREv0obMLmrlvolJPWT4gn1N9sDNNXIC6RDRE5yGsIEf0CliYW1X/8XG40U+g9LG+lrYOGWD4OymZ2P/VDIzZbVT6NG/rdSSGnf4D1AwlOGR7eNTv30AK9o0LVjqGaJWKWYUF9zY6I3+Q==

To provide those files at boot time, we will need to create an ISO file containing them:

$ genisoimage -output centos_1.iso -volid cidata -joliet -rock user-data meta-data

After the ISO file is ready, we can instruct virt-install to actually create the virtual machine:

virt-install --name CentOS_1 \
--ram 2048 \
--disk centos_1.qcow2 \
--vcpus 2 \
--os-variant fedora21 \
--connect qemu:///system \
--network bridge:br0,model=virtio \
--cdrom centos_1.iso \
--boot hd
virt-install --name CentOS_1 \ --ram 2048 \ --disk centos_1.qcow2 \ --vcpus 2 \ --os-variant fedora21 \ --connect qemu:///system \ --network bridge:br0,model=virtio \ --cdrom centos_1.iso \ --boot hd

Since our network configuration is in the ISO file, we will need it at every boot. Sadly, by default this does not happen, so we will need to do a few more steps. Firstly, run virsh:

$ virsh

At this point, a virsh shell should appear with an output like the following:

Welcome to virsh, the virtualization interactive terminal.
Type:  'help' for help with commands
       'quit' to quit
virsh #

This means that we switched from bash (or your shell, if you are not using bash) to the virtualization shell. Issue the following command:

virsh # edit CentOS_1

By doing this we will be able to tweak the configuration of the CentOS_1 machine. In the disk section, you'll need to find the cdrom device that should look like this:

    <disk type='block' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0'
      unit='0'/>
    </disk>

You'll need to change it to the following as highlighted in bold:

    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
        <source file='(Put here your ISO path)/centos_1.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0'
      unit='0'/>
    </disk>

At this point, our virtual machine will always start with the ISO file mounted as a cdrom and therefore cloud-init will be able to correctly initiate the networking.

In this chapter, we have already encountered the expression infrastructure code to describe the Ansible code that will create and maintain your infrastructure. We use the expression infrastructure code to distinguish it from the application code, which is the code that composes your applications, websites, and so on. This distinction is needed for clarity, but in the end, both types are a bunch of text files that the software will be able to read and interpret.

For this reason, a version control system will help you a lot. Its main advantages are:

Those advantages are provided to you by the majority of version control systems out there.

Version control systems can be divided into three major groups based on the three different models that they can implement:

The first category, the local data model, is the oldest (circa 1972) approach and is used for very specific use cases. This model requires all users to share the same filesystem. Famous examples of it are the Revision Control System (RCS) and Source Code Control System (SCCS).

The second category, the client-server model, arrived later (circa 1990) and tried to solve the limitations of the local data model, creating a server that respected the local data model and a set of clients that dealt with the server instead of with the repository itself. This additional layer allowed multiple developers to use local files and synchronize them with a centralized server. Famous examples of this approach are Apache Subversion (SVN), and Concurrent Versions System (CVS).

The third category, the distributed model, arrived at the beginning of the twenty-first century and tried to solve the limitations of the client-server model. In fact, in the client-server mode, you could work on the code offline, but you needed to be online to commit the changes. The distributed model allows you to handle everything on your local repository (like the local data model), and to merge different repositories on different machines in an easy way. In this new model, it's possible to perform all actions as in the client-server model, with the added benefits of being able to work completely offline as well as the ability to merge changes between peers without passing by the centralized server. Examples of this model are BitKeeper (proprietary software), Git, GNU Bazaar, and Mercurial.

There are some additional advantages that will be provided by only the distributed model, such as:

When it comes to infrastructure code, we have to consider that, frequently, the infrastructure that retains and manages your infrastructure code is kept in the infrastructure code itself. This is a recursive situation that can create problems. In fact, until you have your code server in place you cannot deploy your Ansible, and until you have your Ansible in place, you cannot deploy your code server. A distributed version control system will prevent this problem.

As for the simplicity of managing multiple branches, even if this is not a hard rule, often distributed version control systems have much better merge handling than the other kinds of version control systems.

For the reasons that we have just seen and because of its huge popularity, I suggest always using Git for your Ansible repositories.

There are a few suggestions that I always provide to the people I talk to, so Ansible gets the best out of Git:

In this chapter, we have seen what IT automation is, it's advantages, disadvantages, what kind of tools you can find, and how Ansible fits into this big picture. We have also seen how to install Ansible and how to create a KVM-based virtual machine. In the end, we analyzed the version control systems and spoke about the advantages Git brings to Ansible if used properly.

In the next chapter, we will start looking at the infrastructure code that we mentioned in this chapter without explaining exactly what it is and how to write it. Also in the next chapter, we'll see how to automate simple operations that you probably perform every single day, such as managing users, managing files, and file content.

YAML, like many other data serialization languages (such as JSON), has very few, basic concepts:

A declaration is very similar to a variable in any other language, that is:

name: 'This is the name'

To create a list, we will have to use '-':

- 'item1'
- 'item2'
- 'item3'

YAML uses indentation to logically divide parents from children. So if we want to create associative arrays (also known as objects), we would just need to add an indentation:

item:
  name: TheName
  location: TheLocation

Obviously, we can mix those together, that is:

people:
  - name: Albert
    number: +1000000000
    country: USA
  - name: David
    number: +44000000000
    country: UK

Those are the basics of YAML. YAML can do much more, but for now this will be enough.

As we have seen in the previous chapter, it is possible to use Ansible to automate simple tasks that you probably already perform daily.

Let's start by checking if a remote machine is reachable; in other words, let's start by pinging a machine. The simplest way to do this, is to run the following:

$ ansible all -i HOST, -m ping

Here, HOST is an IP address, the Fully Qualified Domain Name (FQDN), or an alias of a machine where you have SSH access (you can use a Kernel-based Virtual Machine (KVM), as we have seen in the previous chapter).

In this case, we have performed it against a virtual machine on our system:

$ ansible all -i test01.fale.io, -m ping

You should receive something like this as a result:

test01.fale.io | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

Now, let's see what we did and why. Let's start from the Ansible help. To query it, we can use the following command:

$ ansible --help

To make it easier to be read, we have removed all the output related to options that we have not used:

Usage: ansible <host-pattern> [options]
Options:
  -i INVENTORY, --inventory-file=INVENTORY
                        specify inventory host path
                        (default=/etc/ansible/hosts) or comma
                        separated host list.
  -m MODULE_NAME, --module-name=MODULE_NAME
                        module name to execute (default=command)

So, what we did was:

Now that we can ping the server, let's echo hello ansible!

$ ansible all -i test01.fale.io, -m shell -a '/bin/echo hello ansible!'

You should receive something like this as a result:

test01.fale.io | SUCCESS | rc=0 >>
hello ansible!

In this example, we used an additional option. Let's check the help to see what it does:

Usage: ansible <host-pattern> [options]
Options:
  -a MODULE_ARGS, --args=MODULE_ARGS
                        module arguments

As you may have guessed from the context and the name, the args options allow you to pass additional arguments to the module. Some modules (like ping) do not support any arguments, while others (such as shell) will require arguments.

Playbooks are one of the core features of Ansible and tell Ansible what to execute. They are like a to-do list for Ansible that contains a list of tasks; each task internally links to a piece of code called a module. Playbooks are simple, human-readable YAML files, whereas modules are a piece of code that can be written in any language with the condition that its output be in the JSON format. You can have multiple tasks listed in a playbook and these tasks would be executed serially by Ansible. You can think of playbooks as an equivalent of manifests in Puppet, states in Salt, or cookbooks in Chef; they allow you to enter a list of tasks or commands you want to execute on your remote system.

---
- hosts: all
  remote_user: fale
  tasks:
  - name: Ensure the HTTPd package is installed
    yum:
      name: httpd
      state: present
      become: True
  - name: Ensure the HTTPd service is enabled and running
    service:
      name: httpd
      state: started
      enabled: True
    become: True

$ ansible-playbook -i HOST, setup_apache.yaml

$ ansible-playbook -i test01.fale.io, setup_apache.yaml

PLAY [all] *******************************************************
TASK [setup] *****************************************************
ok: [test01.fale.io]


TASK [Ensure the HTTPd package is installed] *********************
changed: [test01.fale.io]


TASK [Ensure the HTTPd service is enabled and running] ***********
changed: [test01.fale.io]


PLAY RECAP *******************************************************
test01.fale.io    : ok=3    changed=2    unreachable=0    failed=0   

$ rpm -qa | grep httpd

httpd-tools-2.4.6-40.el7.centos.x86_64
httpd-2.4.6-40.el7.centos.x86_64

$ systemctl status httpd

httpd.service - The Apache HTTP Server
  Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2016-05-07 13:22:14 EDT; 7min ago
     Docs: man:httpd(8)
           man:apachectl(8)
 Main PID: 2214 (httpd)
   Status: "Total requests: 0; Current requests/sec: 0; Current traffic:   0 B/sec"
   CGroup: /system.slice/httpd.service
           -2214 /usr/sbin/httpd -DFOREGROUND
           -2215 /usr/sbin/httpd -DFOREGROUND
           -2216 /usr/sbin/httpd -DFOREGROUND
           -2217 /usr/sbin/httpd -DFOREGROUND
           -2218 /usr/sbin/httpd -DFOREGROUND
           -2219 /usr/sbin/httpd -DFOREGROUND

PLAY [all] *******************************************************

TASK [setup] *****************************************************
ok: [test01.fale.io]

TASK [Ensure the HTTPd package is installed] *********************
changed: [test01.fale.io]
TASK [Ensure the HTTPd service is enabled and running] ***********
changed: [test01.fale.io]

PLAY RECAP *******************************************************
test01.fale.io       : ok=3    changed=2    unreachable=0    failed=0

PLAY [all] *******************************************************
TASK [setup] *****************************************************
ok: [test01.fale.io]
TASK [Ensure the HTTPd package is installed] *********************
ok: [test01.fale.io]
TASK [Ensure the HTTPd service is enabled and running] ***********
ok: [test01.fale.io]
PLAY RECAP *******************************************************
test01.fale.io    : ok=3    changed=0    unreachable=0    failed=0

PLAY RECAP ******************************************************
test01.fale.io    : ok=3    changed=2    unreachable=0    failed=0   

PLAY RECAP *******************************************************
test01.fale.io    : ok=3    changed=0    unreachable=0    failed=0

---
- hosts: all
  remote_user: fale
  tasks:
  - yum:
      name: httpd
      state: present
    become: True
  - service:
      name: httpd
      state: started
      enabled: True
    become: True

$ ansible-playbook -i test01.fale.io, setup_apache_no_com.yaml

PLAY [all] *******************************************************
TASK [setup] *****************************************************
ok: [test01.fale.io]


TASK [yum] *******************************************************
ok: [test01.fale.io]


TASK [service] ***************************************************
ok: [test01.fale.io]


PLAY RECAP *******************************************************
test01.fale.io    : ok=3    changed=0    unreachable=0    failed=0   

One of the first options anyone picks up is the debug option. To understand what is happening when you run the playbook, you can run it with the verbose (-v) option. Every extra v will provide the end user with more debug output.

Let's see an example of using the playbook debug for a single task using the following debug options:

    Using /etc/ansible/ansible.cfg as config file

    PLAYBOOK: setup_apache.yaml *******************************
    1 plays in setup_apache.yaml

    PLAY [all] ************************************************

    TASK [setup] **********************************************
    ok: [test01.fale.io]

    TASK [Ensure the HTTPd package is installed] **************
    task path: /home/fale/setup_apache.yaml:5
    ok: [test01.fale.io] => {"changed": false, "msg": "", "rc": 0, "results": ["httpd-2.4.6-40.el7.centos.x86_64 providing httpd is already installed"]}

    TASK [Ensure the HTTPd service is enabled and running] ****
    task path: /home/fale/setup_apache.yaml:10
    ok: [test01.fale.io] => {"changed": false, "enabled": true, "name": "httpd", "state": "started"}

    PLAY RECAP ************************************************
    test01.fale.io   : ok=3  changed=0   unreachable=0  failed=0  

TASK [Ensure the HTTPd package is installed] **************
task path: /home/fale/setup_apache.yaml:5
<test01.fale.io> ESTABLISH SSH CONNECTION FOR USER: fale
<test01.fale.io> SSH: EXEC ssh -C -q -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=fale -o ConnectTimeout=10 -o ControlPath=/home/fale/.ansible/cp/ansible-ssh-%C test01.fale.io '/bin/sh -c '"'"'( umask 22 && mkdir -p "` echo $HOME/.ansible/tmp/ansible-tmp-1462644055.19-51001413558638 `" && echo "` echo $HOME/.ansible/tmp/ansible-tmp-1462644055.19-51001413558638 `" )'"'"''
<test01.fale.io> PUT /tmp/tmp9JSYiP TO /home/fale/.ansible/tmp/ansible-tmp-1462644055.19-51001413558638/yum
<test01.fale.io> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=fale -o ConnectTimeout=10 -o ControlPath=/home/fale/.ansible/cp/ansible-ssh-%C '[test01.fale.io]'
<test01.fale.io> ESTABLISH SSH CONNECTION FOR USER: fale
<test01.fale.io> SSH: EXEC ssh -C -q -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=fale -o ConnectTimeout=10 -o ControlPath=/home/fale/.ansible/cp/ansible-ssh-%C -tt test01.fale.io '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-axnwopicemeccmdhnlmhawtwlysgfgjc; LANG=en_US.utf8 LC_ALL=en_US.utf8 LC_MESSAGES=en_US.utf8 /usr/bin/python -tt /home/fale/.ansible/tmp/ansible-tmp-1462644055.19-51001413558638/yum; rm -rf "/home/fale/.ansible/tmp/ansible-tmp-1462644055.19-51001413558638/" > /dev/null 2>&1'"'"'"'"'"'"'"'"''"'"''
ok: [test01.fale.io] => {"changed": false, "invocation": {"module_args": {"conf_file": null, "disable_gpg_check": false, "disablerepo": null, "enablerepo": null, "exclude": null, "install_repoquery": true, "list": null, "name": ["httpd"], "state": "present", "update_cache": false}, "module_name": "yum"}, "msg": "", "rc": 0, "results": ["httpd-2.4.6-40.el7.centos.x86_64 providing httpd is already installed"]}

Sometimes it is important to set and get variables in a playbook.

Very often, you'll need to automate multiple similar operations. In those cases, you'll want to create a single playbook that can be called with different variables to ensure code reusability.

Another case where variables are very important is when you have more than one datacenter and some values will be datacenter-specific. A common example are the DNS servers. Let's analyze the following simple code that will introduce us to the Ansible way to set and get variables:

---
- hosts: all
  remote_user: fale
  tasks:
  - name: Set variable 'name'
    set_fact:
      name: Test machine
  - name: Print variable 'name'
    debug:
      msg: '{{ name }}'

Let's run it in the usual way:

$ ansible-playbook -i test01.fale.io, variables.yaml

You should see the following result:

PLAY [all] **********************************************************
TASK [setup] ********************************************************
ok: [test01.fale.io]


TASK [Set variable 'name'] ******************************************
ok: [test01.fale.io]


TASK [Print variable 'name'] ****************************************
ok: [test01.fale.io] => {
    "msg": "Test machine"
}


PLAY RECAP **********************************************************
test01.fale.io       : ok=3    changed=0    unreachable=0    failed=0

If we analyze the code we have just executed, it should be pretty clear what's going on. We set a variable (that in Ansible are called facts) and then we print it with the debug function.

Ansible allows you to set your variables in many different ways, that is, either by passing a variable file, declaring it in a playbook, passing it to the ansible-playbook command using -e / --extra-vars, or by declaring it in an inventory file (we will be discussing more in-depth about this in the next chapter).

It's now time to start using some metadata that Ansible obtained during the setup phase. Let's start by looking at the data that is gathered by Ansible. To do this, we will execute:

$ ansible all -i HOST, -m setup

In our specific case, this means executing the following:

$ ansible all -i test01.fale.io, -m setup

We can obviously do the same with a playbook, but this way is faster. Also, for the "setup" case, you will need to see the output only during the development to be sure to use the right variable name for your goal.

The output will be something like this:

test01.fale.io | SUCCESS => {
    "ansible_facts": {
        "ansible_all_ipv4_addresses": [
            "178.62.36.208",
            "10.16.0.7"
        ],
        "ansible_all_ipv6_addresses": [
            "fe80::601:e2ff:fef1:1301"
        ],
        "ansible_architecture": "x86_64",
        "ansible_bios_date": "04/25/2016",
        "ansible_bios_version": "20160425",
        "ansible_cmdline": {
            "ro": true,
            "root": "LABEL=DOROOT"
        },
        "ansible_date_time": {
            "date": "2016-05-14",
            "day": "14",
            "epoch": "1463244633",
            "hour": "12",
            "iso8601": "2016-05-14T16:50:33Z",
            "iso8601_basic": "20160514T125033231663",
            "iso8601_basic_short": "20160514T125033",
            "iso8601_micro": "2016-05-14T16:50:33.231770Z",
            "minute": "50",
            "month": "05",
            "second": "33",
            "time": "12:50:33",
            "tz": "EDT",
            "tz_offset": "-0400",
            "weekday": "Saturday",
            "weekday_number": "6",
            "weeknumber": "19",
            "year": "2016"
        },
        "ansible_default_ipv4": {
            "address": "178.62.36.208",
            "alias": "eth0",
            "broadcast": "178.62.63.255",
            "gateway": "178.62.0.1",
            "interface": "eth0",
            "macaddress": "04:01:e2:f1:13:01",
            "mtu": 1500,
            "netmask": "255.255.192.0",
            "network": "178.62.0.0",
            "type": "ether"
        },
        "ansible_default_ipv6": {},
        "ansible_devices": {
            "vda": {
                "holders": [],
                "host": "",
                "model": null,
                "partitions": {
                    "vda1": {
                        "sectors": "41943040",
                        "sectorsize": 512,
                        "size": "20.00 GB",
                        "start": "2048"
                    }
                },
                "removable": "0",
                "rotational": "1",
                "scheduler_mode": "",
                "sectors": "41947136",
                "sectorsize": "512",
                "size": "20.00 GB",
                "support_discard": "0",
                "vendor": "0x1af4"
            }
        },
        "ansible_distribution": "CentOS",
        "ansible_distribution_major_version": "7",
        "ansible_distribution_release": "Core",
        "ansible_distribution_version": "7.2.1511",
        "ansible_dns": {
            "nameservers": [
                "8.8.8.8",
                "8.8.4.4"
            ]
        },
        "ansible_domain": "",
        "ansible_env": {
            "HOME": "/home/fale",
            "LANG": "en_US.utf8",
            "LC_ALL": "en_US.utf8",
            "LC_MESSAGES": "en_US.utf8",
            "LESSOPEN": "||/usr/bin/lesspipe.sh %s",
            "LOGNAME": "fale",
            "MAIL": "/var/mail/fale",
            "PATH": "/usr/local/bin:/usr/bin",
            "PWD": "/home/fale",
            "SHELL": "/bin/bash",
            "SHLVL": "2",
            "SSH_CLIENT": "86.187.141.39 37764 22",
            "SSH_CONNECTION": "86.187.141.39 37764 178.62.36.208 22",
            "SSH_TTY": "/dev/pts/0",
            "TERM": "rxvt-unicode-256color",
            "USER": "fale",
            "XDG_RUNTIME_DIR": "/run/user/1000",
            "XDG_SESSION_ID": "180",
            "_": "/usr/bin/python"
        },
        "ansible_eth0": {
            "active": true,
            "device": "eth0",
            "ipv4": {
                "address": "178.62.36.208",
                "broadcast": "178.62.63.255",
                "netmask": "255.255.192.0",
                "network": "178.62.0.0"
            },
            "ipv4_secondaries": [
                {
                    "address": "10.16.0.7",
                    "broadcast": "10.16.255.255",
                    "netmask": "255.255.0.0",
                    "network": "10.16.0.0"
                }
            ],
            "ipv6": [
                {
                    "address": "fe80::601:e2ff:fef1:1301",
                    "prefix": "64",
                    "scope": "link"
                }
            ],
            "macaddress": "04:01:e2:f1:13:01",
            "module": "virtio_net",
            "mtu": 1500,
            "pciid": "virtio0",
            "promisc": false,
            "type": "ether"
        },
        "ansible_eth1": {
            "active": false,
            "device": "eth1",
            "macaddress": "04:01:e2:f1:13:02",
            "module": "virtio_net",
            "mtu": 1500,
            "pciid": "virtio1",
            "promisc": false,
            "type": "ether"
        },
        "ansible_fips": false,
        "ansible_form_factor": "Other",
        "ansible_fqdn": "test",
        "ansible_hostname": "test",
        "ansible_interfaces": [
            "lo",
            "eth1",
            "eth0"
        ],
        "ansible_kernel": "3.10.0-327.10.1.el7.x86_64",
        "ansible_lo": {
            "active": true,
            "device": "lo",
            "ipv4": {
                "address": "127.0.0.1",
                "broadcast": "host",
                "netmask": "255.0.0.0",
                "network": "127.0.0.0"
            },
            "ipv6": [
                {
                    "address": "::1",
                    "prefix": "128",
                    "scope": "host"
                }
            ],
            "mtu": 65536,
            "promisc": false,
            "type": "loopback"
        },
        "ansible_machine": "x86_64",
        "ansible_machine_id": "fd8cf26e06e411e4a9d004010897bd01",
        "ansible_memfree_mb": 6,
        "ansible_memory_mb": {
            "nocache": {
                "free": 381,
                "used": 108
            },
            "real": {
                "free": 6,
                "total": 489,
                "used": 483
            },
            "swap": {
                "cached": 0,
                "free": 0,
                "total": 0,
                "used": 0
            }
        },
        "ansible_memtotal_mb": 489,
        "ansible_mounts": [
            {
                "device": "/dev/vda1",
                "fstype": "ext4",
                "mount": "/",
                "options": "rw,relatime,data=ordered",
                "size_available": 18368385024,
                "size_total": 21004894208,
                "uuid": "c5845b43-fe98-499a-bf31-4eccae14261b"
            }
        ],
        "ansible_nodename": "test",
        "ansible_os_family": "RedHat",
        "ansible_pkg_mgr": "yum",
        "ansible_processor": [
            "GenuineIntel",
            "Intel(R) Xeon(R) CPU E5-2630L v2 @ 2.40GHz"
        ],
        "ansible_processor_cores": 1,
        "ansible_processor_count": 1,
        "ansible_processor_threads_per_core": 1,
        "ansible_processor_vcpus": 1,
        "ansible_product_name": "Droplet",
        "ansible_product_serial": "NA",
        "ansible_product_uuid": "NA",
        "ansible_product_version": "20160415",
        "ansible_python_version": "2.7.5",
        "ansible_selinux": {
            "status": "disabled"
        },
        "ansible_service_mgr": "systemd",
        "ansible_ssh_host_key_dsa_public": "AAAAB3NzaC1kc3MAAACBAPEf4dzeET6ukHemTASsamoRLxo2R8iHg5J1bYQUyuggtRKlbRrHMtpQ8qN5CQNtp8J+2Hq6/JKiDF+cdxgOehf9b7F4araVvJxqx967RvLNBrMWXv7/4hi+efgXG9eejGoGQNAD66up/fkLMd0L8fwSwmTJoZXwOxFwcbnxCZsFAAAAFQDgK7fka+1AKjYZNFIfCB2b0ZitGQAAAIADeofiC5q+SLgEvkBCUCTyJ+EVb6WHeHbVdrpE2GdnUr03R6MmmYhYZMijruS/rcpzBLmi8juDkqAWy6Xqxd+DwixykntXPeUFS3F7LK5vNwFalaRltPwr4Azh+EeSUQ2Zz2AdKx6zSqtLOD8ZMPkRDvz4WGHGmeR+i7UFsFDZdgAAAIEAy26Tx0jAlY3mEaTW9lQ9DoGXgPBxsSX/XqeLh5wBaBO6AJaIrs0dQJdNeHcMhFy0seVkOMN1SpeoBTJSoTOx15HAGsKsAcmnA5mcJeUZqptVR6JxROztHw3zQePQ3/V3KQzAN31tIm3PbKztlEZbXRUM7RV5WsdRHTb8rutENhY=",
        "ansible_ssh_host_key_ecdsa_public": "AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPDXQ9rjgDmUKsEWH4U2vg4iqtK+75urlj9nwW+rNNTFHTE5oG82sOlO6o0tUY8LXgB/tJnIcJ1hINdrWrZNpn4=",
        "ansible_ssh_host_key_rsa_public": "AAAAB3NzaC1yc2EAAAADAQABAAABAQCwQx5EElH7FeD/agB/gCJfBUEVhk44tldzdEzwc2IEbI59relTGNOU7soCCMcSH7nwlEbOOvmLa2R/YaXdHv/cb1aXBC/wj/m4ZHylBeF5qzECUkeaB3+CT+hp8qHHApclFr2lm2CwZ+YXjEyjJ3en4K3gLlIQyQjgE2F57kmD1FVVDSJFvNTn+NQvb3DPppND+HKEeHwrJ0GgznoP62yobEgriAIBSGf//0WHCO/9shEvauoRpPM+U9pU7lv637s7qyubIqyrs5fz3u34qBj8oCATOefRN1wsfJDeMG0D5ryI6BI6t/eAi8BPr7VHJSQBk+buM9Jr1yoMQTEasq2J",
        "ansible_swapfree_mb": 0,
        "ansible_swaptotal_mb": 0,
        "ansible_system": "Linux",
        "ansible_system_vendor": "DigitalOcean",
        "ansible_uptime_seconds": 603067,
        "ansible_user_dir": "/home/fale",
        "ansible_user_gecos": "",
        "ansible_user_gid": 1000,
        "ansible_user_id": "fale",
        "ansible_user_shell": "/bin/bash",
        "ansible_user_uid": 1000,
        "ansible_userspace_architecture": "x86_64",
        "ansible_userspace_bits": "64",
        "ansible_virtualization_role": "host",
        "ansible_virtualization_type": "kvm",
        "module_setup": true
    },
    "changed": false
}

As you can see, from this huge list of options, you can gain a huge quantity of information, and you can use them as any other variable. Let's print the OS name and the version. To do so, we can create a new playbook called setup_variables.yaml with the following content:

---
- hosts: all
  remote_user: fale
  tasks:
  - name: Print OS and version
    debug:
      msg: '{{ ansible_distribution }} {{ ansible_distribution_version }}'

Run it with the following:

$ ansible-playbook -itest01.fale.io, setup_variables.yaml

This will give us the following output:

PLAY [all] *******************************************************
TASK [setup] *****************************************************
ok: [test01.fale.io]


TASK [Print OS and version] **************************************
ok: [test01.fale.io] => {
    "msg": "CentOS 7.2.1511"

}


PLAY RECAP *******************************************************
test01.fale.io     : ok=2   changed=0    unreachable=0    failed=0

As you can see, it printed the OS name and version, as expected. In addition to the methods seen previously, it's also possible to pass a variable using a command-line argument. In fact, if we look in the Ansible help, we will notice the following:

-e EXTRA_VARS, --extra-vars=EXTRA_VARS
set additional variables as key=value or YAML/JSON

The same lines are present in the ansible-playbook command as well. Let's make a small playbook called cli_variables.yaml with the following content:

---
- hosts: all
  remote_user: fale
  tasks:
  - name: Print variable 'name'
    debug:
      msg: '{{ name }}'

Execute it with the following:

$ ansible-playbook -i test01.fale.io, cli_variables.yaml -e 'name=test01'

We will receive the following:

PLAY [all] *******************************************************
TASK [setup] *****************************************************
ok: [test01.fale.io]
TASK [Print variable 'name'] *************************************
ok: [test01.fale.io] => {
    "msg": "test01"
}
PLAY RECAP *******************************************************
test01.fale.io     : ok=2   changed=0    unreachable=0    failed=0

In case we forgot to add the additional parameter to specify the variable, we would have executed it as:

$ ansible-playbook -i test01.fale.io, cli_variables.yaml

We would have received the following output:

PLAY [all] *******************************************************
TASK [setup] *****************************************************
ok: [test01.fale.io]


TASK [Print variable 'name'] *************************************
fatal: [test01.fale.io]: FAILED! => {"failed": true, "msg": "'name' is undefined"}


NO MORE HOSTS LEFT ***********************************************
to retry, use: --limit @cli_variables.retry


PLAY RECAP *******************************************************
test01.fale.io     : ok=1   changed=0    unreachable=0    failed=1

Now that we have learned the basics of playbooks, let's create a web server from scratch using them. To do so, let's start from the beginning, creating an Ansible user and then moving forward from there.

When you create a machine (or rent one from any hosting company) it arrives only with the root user. Let's start creating a playbook that ensures that an Ansible user is created, it's accessible with an SSH key, and is able to perform actions on behalf of other users (sudo) with no password asked. I often call this playbook, firstrun.yaml since I execute it as soon as a new machine is created, but after that, I don't use it since it uses the root user that I disable for security reasons. Our script will look something like the following:

---
- hosts: all
  user: root
  tasks:
  - name: Ensure ansible user exists
    user:
      name: ansible
      state: present
      comment: Ansible
  - name: Ensure ansible user accepts the SSH key
    authorized_key:
      user: ansible
      key: https://github.com/fale.keys
   state: present
  - name: Ensure the ansible user is sudoer with no password required
    lineinfile:
      dest: /etc/sudoers
      state: present
      regexp: '^ansible ALL\='
      line: 'ansible ALL=(ALL) NOPASSWD:ALL'
      validate: 'visudo -cf %s'

Before running it, let's look at it a little bit. We have used three different modules (user, authorized_key, and lineinfile) that we have never seen. The user module, as the name suggests, allows us to make sure a user is present (or absent).

The authorized_key module allows us to ensure that a certain SSH key can be used to login as a specific user on that machine. This module will not substitute all the SSH keys that are already enabled for that user, but will simply add (or remove) the specified key. If you want to alter this behavior, you can use the exclusive option, that allows you to delete all the SSH keys that are not specified in this step.

The lineinfile module allows us to alter the content of a file. It works in a very similar way to sed (a stream editor), where you specify the regular expression that will be used to match the line, and then specify the new line that will be used to substitute the matched line. If no line is matched, the line is added at the end of the file. Now let's run it with:

$ ansible-playbook -i test01.fale.io, firstrun.yaml

This will give us the following result:

PLAY [all] *******************************************************
TASK [setup] *****************************************************
ok: [test01.fale.io]


TASK [Ensure ansible user exists] ********************************
changed: [test01.fale.io]


TASK [Ensure ansible user accepts the SSH key] *******************
changed: [test01.fale.io]


TASK [Ensure the anisble user is sudoer with no password required] *
changed: [test01.fale.io]


PLAY RECAP *******************************************************
test01.fale.io     : ok=4   changed=3    unreachable=0    failed=0

After we have created the user for Ansible with the necessary privileges, we can go on to make some other small changes to the OS. To make it more clear, we will see how each action is performed and then we'll look at the whole playbook.

- name: Ensure EPEL is enabled
  yum:
    name: epel-release
    state: present
  become: True

- name: Ensure libselinux-python is present
  yum:
    name: libselinux-python
    state: present
  become: True
- name: Ensure libsemanage-python is present
  yum:
    name: libsemanage-python
    state: present
  become: True

- name: Ensure we have last version of every package
  yum:
    name: "*"
    state: latest
  become: True

- name: Ensure NTP is installed
  yum:
    name: ntp
    state: present
  become: True

- name: Ensure the timezone is set to UTC
  file:
    src: /usr/share/zoneinfo/GMT
    dest: /etc/localtime
    state: link
  become: True

- name: Ensure the NTP service is running and enabled
  service:
    name: ntpd
    state: started
    enabled: True
  become: True

- name: Ensure FirewallD is installed
  yum:
    name: firewalld
    state: present
  become: True

- name: Ensure SSH can pass the firewall
  firewalld:
    service: ssh
    state: enabled
    permanent: True
    immediate: True
  become: True

- name: Ensure FirewallD is running
  service:
    name: firewalld
    state: started
    enabled: True
  become: True

                This system is managed by Ansible
  Any change done on this system could be overwritten by Ansible

OS: {{ ansible_distribution }} {{ ansible_distribution_version }}
Hostname: {{ inventory_hostname }}
eth0 address: {{ ansible_eth0.ipv4.address }}
All connections are monitored and recorded
    Disconnect IMMEDIATELY if you are not an authorized user

- name: Ensure the MOTD file is present and updated
  template:
    src: motd
    dest: /etc/motd
    owner: root
    group: root
    mode: 0644
  become: True

- name: Ensure the hostname is the same of the inventory
  hostname:
    name: "{{ inventory_hostname }}"
  become: True

---
- hosts: all
  remote_user: ansible
  tasks:
  - name: Ensure EPEL is enabled
    yum:
      name: epel-release
      state: present
    become: True
  - name: Ensure libselinux-python is present
    yum:
      name: libselinux-python
      state: present
    become: True
  - name: Ensure libsemanage-python is present
    yum:
      name: libsemanage-python
      state: present
    become: True
  - name: Ensure we have last version of every package
    yum:
      name: "*"
      state: latest
    become: True
  - name: Ensure NTP is installed
    yum:
      name: ntp
      state: present
    become: True
  - name: Ensure the timezone is set to UTC
    file:
      src: /usr/share/zoneinfo/GMT
      dest: /etc/localtime
      state: link
    become: True
  - name: Ensure the NTP service is running and enabled
    service:
      name: ntpd
      state: started
      enabled: True
    become: True
  - name: Ensure FirewallD is installed
    yum:
      name: firewalld
      state: present
    become: True
  - name: Ensure FirewallD is running
    service:
      name: firewalld
      state: started
      enabled: True
    become: True
  - name: Ensure SSH can pass the firewall
    firewalld:
      service: ssh
      state: enabled
      permanent: True
      immediate: True
    become: True
  - name: Ensure the MOTD file is present and updated
    template:
      src: motd
      dest: /etc/motd
      owner: root
      group: root
      mode: 0644
    become: True
  - name: Ensure the hostname is the same of the inventory
    hostname:
      name: "{{ inventory_hostname }}"
    become: True

$ ansible-playbook common_tasks.yaml --list-tasks

playbook: common_tasks.yaml
  play #1 (all): all TAGS: []
    tasks:
      Ensure EPEL is enabled TAGS: []
      Ensure libselinux-python is present TAGS: []
      Ensure libsemanage-python is present TAGS: []
      Ensure we have last version of every package TAGS: []
      Ensure NTP is installed TAGS: []
      Ensure the timezone is set to UTC TAGS: []
      Ensure the NTP service is running and enabled TAGS: []
      Ensure FirewallD is installed TAGS: []
      Ensure FirewallD is running TAGS: []
      Ensure SSH can pass the firewall TAGS: []
      Ensure the MOTD file is present and updated TAGS: []
      Ensure the hostname is the same of the inventory TAGS: []

$ ansible-playbook -itest01.fale.io, common_tasks.yaml

PLAY [all] *******************************************************

TASK [setup] *****************************************************
ok: [test01.fale.io]

TASK [Ensure EPEL is enabled] ************************************
changed: [test01.fale.io]

TASK [Ensure libselinux-python is present] ***********************
ok: [test01.fale.io]

TASK [Esure libsemanage-python is present] ***********************
ok: [test01.fale.io]

TASK [Ensure we have last version of every package] **************
changed: [test01.fale.io]

TASK [Ensure NTP is installed] ***********************************
ok: [test01.fale.io]

TASK [Ensure the timezone is set to UTC] *************************
changed: [test01.fale.io]

TASK [Ensure the NTP service is running and enabled] *************
changed: [test01.fale.io]

TASK [Ensure FirewallD is installed] *****************************
ok: [test01.fale.io]

TASK [Ensure FirewallD is running] *******************************
changed: [test01.fale.io]

TASK [Ensure SSH can pass the firewall] **************************
ok: [test01.fale.io]

TASK [Ensure the MOTD file is present and updated] ***************
changed: [test01.fale.io]

TASK [Ensure the hostname is the same of the inventory] **********
changed: [test01.fale.io]

PLAY RECAP *******************************************************
test01.fale.io     : ok=9   changed=7    unreachable=0    failed=0

Now that we have made some generic changes to the operating system, let's move on to actually creating a web server. We are splitting those two phases so we can share the first phase between every machine and apply the second only to web servers.

For this second phase, we will create a new playbook called webserver.yaml with the following content:

---
- hosts: all
  remote_user: ansible
  tasks:
  - name: Ensure the HTTPd package is installed
    yum:
      name: httpd
      state: present
    become: True
  - name: Ensure the HTTPd service is enabled and running
    service:
      name: httpd
      state: started
      enabled: True
    become: True
  - name: Ensure HTTP can pass the firewall
    firewalld:
      service: http
      state: enabled
      permanent: True
      immediate: True
    become: True
  - name: Ensure HTTPS can pass the firewall
    firewalld:
      service: https
      state: enabled
      permanent: True
      immediate: True
    become: True

As you can see, the first two tasks are the same as the ones in the example at the beginning of this chapter, and the last two tasks are used to instruct FirewallD to let HTTP and HTTPS traffic pass.

Let's run this script with the following:

$ ansible-playbook -i test01.fale.io, webserver.yaml

This results in the following:

PLAY [all] *******************************************************
TASK [setup] *****************************************************
ok: [test01.fale.io]


TASK [Ensure the HTTPd package is installed] *********************
changed: [test01.fale.io]


TASK [Ensure the HTTPd service is enabled and running] ***********
changed: [test01.fale.io]


TASK [Ensure HTTP can pass the firewall] *************************
changed: [test01.fale.io]


TASK [Ensure HTTPS can pass the firewall] ************************
changed: [test01.fale.io]


PLAY RECAP *******************************************************
test01.fale.io    : ok=5    changed=4    unreachable=0    failed=0

Now that we have a web server, let's publish a small single-page static website.

Since our website will be a simple, single page website, we can easily create it and publish it using a single Ansible task. To make this page a little bit more interesting, we will create it from a template that will be populated by Ansible with a little data about the machine. The script to publish it will be called deploy_website.yaml and will have the following content:

---
- hosts: all
  remote_user: ansible
  tasks:
  - name: Ensure the website is present and updated
    template:
      src: index.html.j2
      dest: /var/www/html/index.html
      owner: root
      group: root
      mode: 0644
    become: True

Let's start with a simple template that we will call index.html.j2:

<html>
    <body>
        <h1>Hello World!</h1>
    </body>
</html>

Now we can test our website deployment by running the following:

$ ansible-playbook -i test01.fale.io, deploy_website.yaml

We should receive the following output:

PLAY [all] *******************************************************
TASK [setup] *****************************************************
ok: [test01.fale.io]


TASK [Ensure the website is present and updated] *****************
changed: [test01.fale.io]


PLAY RECAP *******************************************************
test01.fale.io    : ok=2    changed=1    unreachable=0    failed=0

If you now go to your test machine IP/FQDN in your browser, you'll find the "Hello World!" page.

Jinja2 is a widely-used and fully-featured template engine for Python. Let's look at some syntax that will help us with Ansible. This paragraph does not want to be a replacement for the official documentation, but its goal is to teach you some components that you'll find very useful when using them with Ansible.

<html>
    <body>
        <h1>Hello World!</h1>
        <p>This page was created on {{ ansible_date_time.date }}.</p>
    </body>
</html>

<html>
    <body>
        <h1>Hello World!</h1>
        <p>This page was created on {{ ansible_date_time.date }}.</p>
{% if ansible_eth0.active == True %}
        <p>eth0 address {{ ansible_eth0.ipv4.address }}.</p>
{% endif %}
    </body>
</html>

<html>
    <body>
        <h1>Hello World!</h1>
        <p>This page was created on {{ ansible_date_time.date }}.</p>
{% if ansible_eth0.active is equalto True %}
        <p>eth0 address {{ ansible_eth0.ipv4.address }}.</p>
{% endif %}
    </body>
</html>

<html>
    <body>
        <h1>Hello World!</h1>
        <p>This page was created on {{ ansible_date_time.date }}.</p>
        <p>This machine can be reached on the following IP addresses</p>
        <ul>
{% for address in ansible_all_ipv4_addresses %}
            <li>{{ address }}</li>
{% endfor %}
        </ul>
    </body>
</html>

In this chapter, we started looking at YAML and saw what a playbook is, how it works, and how to use it to create a web server (and a deployment for your static website). We have also seen multiple Ansible modules such as the user, yum, service, FirewalID, lineinfile, and template modules. At the end of the chapter, we focused on templates.

In the next chapter, we will talk about inventories so that we can easily manage multiple machines.

An inventory file is the source of truth for Ansible (there is also an advanced concept called dynamic inventory, which we will cover later). It follows the Initialization (INI) format and tells Ansible whether the remote host or hosts provided by the user are genuine.

Ansible can run its tasks against multiple hosts in parallel. To do this, you can directly pass the list of hosts to Ansible using an inventory file. For such parallel execution, Ansible allows you to group your hosts in the inventory file; the file passes the group name to Ansible. Ansible will search for that group in the inventory file and run its tasks against all the hosts listed in that group.

You can pass the inventory file to Ansible using the -i or --inventory-file option followed by the path to the file. If you do not explicitly specify any inventory file to Ansible, it will take the default path from the host_file parameter of ansible.cfg, which defaults to /etc/ansible/hosts.

test01.fale.io

$ ansible-playbook -i test01.fale.io, webserver.yaml

$ ansible-playbook -i hosts webserver.yaml

[webserver]
ws01.fale.io
ws02.fale.io

[database]
db01.fale.io

- hosts: all

- hosts: webserver

$ ansible-playbook -i hosts firstrun.yaml

PLAY [all] *******************************************************
TASK [setup] *****************************************************
ok: [ws02.fale.io]
ok: [db01.fale.io]
ok: [ws01.fale.io]


TASK [Ensure ansible user exists] ********************************
changed: [ws01.fale.io]
changed: [db01.fale.io]
changed: [ws02.fale.io]


TASK [Ensure ansible user accepts the SSH key] *******************
changed: [ws02.fale.io]
changed: [ws01.fale.io]
changed: [db01.fale.io]


TASK [Ensure the ansible user is sudoer with no password required]
changed: [ws01.fale.io]
changed: [db01.fale.io]
changed: [ws02.fale.io]


PLAY RECAP *******************************************************
db01.fale.io      : ok=4    changed=3    unreachable=0    failed=0
ws01.fale.io      : ok=4    changed=3    unreachable=0    failed=0
ws02.fale.io      : ok=4    changed=3    unreachable=0    failed=0

[webserver]
ws[01:02].fale.io

[database]
db01.fale.io

Ansible allows you to define variables in many ways, from a variable file within a playbook, by passing it from the Ansible command using the -e / --extra-vars option, or by passing it to an inventory file. You can define variables in an inventory file either on a per-host basis, for an entire group, or by creating a variable file in the directory where your inventory file exists.

[webserver]
ws01.fale.io domainname=example1.fale.io
ws02.fale.io domainname=example2.fale.io

[database]
db01.fale.io

[webserver]
ws01.fale.io
ws02.fale.io

[webserver:vars]
https_enabled=True

[database]
db01.fale.io

domainname=example1.fale.io

domainname=example2.fale.io

https_enabled=True

There are environments where you have a system that creates and destroys machines automatically. We will see how to do this with Ansible in Chapter 5, Going Cloud. In such environments, the list of machines changes very quickly and keeping the hosts file becomes complicated. In this case, we can use dynamic inventories to solve the problem.

The idea behind dynamic inventories is that Ansible will not read the hosts file, but instead execute a script that will return the list of hosts to Ansible in JSON format. This allows you, for instance, to query your cloud provider and ask it directly, what machines in your entire infrastructure are running at any given moment.

Many scripts for the most common cloud providers are already present in Ansible at: https://github.com/ansible/ansible/tree/devel/contrib/inventory but you can create a custom script if you have different needs. The Ansible inventory scripts can be written in any language but, for consistency reasons, dynamic inventory scripts should be written in Python. Remember that these scripts need to be executable directly, so please remember to set them with the executable flag (chmod + x inventory.py).

In this chapter, we will take a look at Amazon Web Services and DigitalOcean scripts that can be downloaded from the official Ansible repository.

ansible -i ec2.py all -m ping

52.28.138.231 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

ansible -i digital_ocean.py all -m ping

188.166.150.79 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
46.101.77.55 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

You may have noticed that up to now we have never used cycles, so every time we had to do multiple, similar operations, we wrote the code multiple times. An example of this is the webserver.yaml code.

In fact, this was the content of the webserver.yaml file:

---
- hosts: webserver
  remote_user: ansible
  tasks:
  - name: Ensure the HTTPd package is installed
    yum:
      name: httpd
      state: present
    become: True
  - name: Ensure the HTTPd service is enabled and running
    service:
      name: httpd
      state: started
      enabled: True
    become: True
  - name: Ensure HTTP can pass the firewall
    firewalld:
      service: http
      state: enabled
      permanent: True
      immediate: True
    become: True
  - name: Ensure HTTPS can pass the firewall
    firewalld:
      service: https
      state: enabled
      permanent: True
      immediate: True
    become: True

As you can see, the last two blocks do the same operation (ensure that a certain port of the firewall is open).

---
- hosts: webserver
  remote_user: ansible
  tasks:
  - name: Ensure the HTTPd package is installed
    yum:
      name: httpd
      state: present
    become: True
  - name: Ensure the HTTPd service is enabled and running
    service:
      name: httpd
      state: started
      enabled: True
    become: True
  - name: Ensure HTTP and HTTPS can pass the firewall
    firewalld:
      service: '{{ item }}'
      state: enabled
      permanent: True
      immediate: True
    become: True
    with_items:
    - http
    - https

ansible-playbook -i hosts webserver.yaml

PLAY [webserver] *************************************************
TASK [setup] *****************************************************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [Ensure the HTTPd package is installed] *********************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [Ensure the HTTPd service is enabled and running] ***********
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [Ensure HTTP can pass the firewall] *************************
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [Ensure HTTP and HTTPS can pass the firewall] ***************
ok: [ws02.fale.io] => (item=http)
ok: [ws01.fale.io] => (item=http)
ok: [ws02.fale.io] => (item=https)
ok: [ws01.fale.io] => (item=https)


PLAY RECAP *******************************************************
ws01.fale.io      : ok=5    changed=0    unreachable=0    failed=0   
ws02.fale.io      : ok=5    changed=0    unreachable=0    failed=0  

---
- hosts: all
  remote_user: ansible
  vars:
    users:
    - alice
    - bob
    folders:
    - mail
    - public_html
  tasks:
  - name: Ensure the users exist
    user:
      name: '{{ item }}'
    become: True
    with_items:
    - '{{ users }}'
  - name: Ensure the folders exist
    file:
      path: '/home/{{ item.0 }}/{{ item.1 }}'
      state: directory
    become: True
    with_nested:
    - '{{ users }}'
    - '{{ folders }}'

ansible-playbook -i hosts with_nested.yaml

PLAY [all] *******************************************************
TASK [setup] *****************************************************
ok: [ws01.fale.io]
ok: [db01.fale.io]
ok: [ws02.fale.io]


TASK [Ensure the users exist] ************************************
changed: [db01.fale.io] => (item=alice)
changed: [ws01.fale.io] => (item=alice)
changed: [ws02.fale.io] => (item=alice)
changed: [db01.fale.io] => (item=bob)
changed: [ws01.fale.io] => (item=bob)
changed: [ws02.fale.io] => (item=bob)


TASK [Ensure the folders exist] **********************************
changed: [ws02.fale.io] => (item=[u'alice', u'mail'])
changed: [ws01.fale.io] => (item=[u'alice', u'mail'])
changed: [db01.fale.io] => (item=[u'alice', u'mail'])
changed: [ws01.fale.io] => (item=[u'alice', u'public_html'])
changed: [ws02.fale.io] => (item=[u'alice', u'public_html'])
changed: [db01.fale.io] => (item=[u'alice', u'public_html'])
changed: [ws02.fale.io] => (item=[u'bob', u'mail'])
changed: [ws01.fale.io] => (item=[u'bob', u'mail'])
changed: [db01.fale.io] => (item=[u'bob', u'mail'])
changed: [ws02.fale.io] => (item=[u'bob', u'public_html'])
changed: [ws01.fale.io] => (item=[u'bob', u'public_html'])
changed: [db01.fale.io] => (item=[u'bob', u'public_html'])


PLAY RECAP *******************************************************
db01.fale.io      : ok=3    changed=2    unreachable=0    failed=0   
ws01.fale.io      : ok=3    changed=2    unreachable=0    failed=0   
ws02.fale.io      : ok=3    changed=2    unreachable=0    failed=0   

---
- hosts: all
  remote_user: ansible
  tasks:
  - name: Ensure the folder /tmp/iproute2 is present
    file:
      dest: '/tmp/iproute2'
      state: directory
    become: True
  - name: Copy files that start with rt to the tmp folder
    copy:
      src: '{{ item }}'
      dest: '/tmp/iproute2'
      remote_src: True
    become: True
    with_fileglob:
    - '/etc/iproute2/rt_*'

ansible-playbook -i hosts with_fileglobs.yaml

PLAY [all] *******************************************************
TASK [setup] *****************************************************
ok: [ws02.fale.io]
ok: [db01.fale.io]
ok: [ws01.fale.io]


TASK [Ensure the folder /tmp/iproute2 is present] ****************
changed: [db01.fale.io]
changed: [ws01.fale.io]
changed: [ws02.fale.io]


TASK [Copy files that start with rt to the tmp folder] ***********
changed: [db01.fale.io] => (item=/etc/iproute2/rt_dsfield)
changed: [ws02.fale.io] => (item=/etc/iproute2/rt_dsfield)
changed: [ws01.fale.io] => (item=/etc/iproute2/rt_dsfield)
changed: [db01.fale.io] => (item=/etc/iproute2/rt_protos)
changed: [ws01.fale.io] => (item=/etc/iproute2/rt_protos)
changed: [ws02.fale.io] => (item=/etc/iproute2/rt_protos)
changed: [db01.fale.io] => (item=/etc/iproute2/rt_tables)
changed: [ws01.fale.io] => (item=/etc/iproute2/rt_tables)
changed: [ws02.fale.io] => (item=/etc/iproute2/rt_tables)
changed: [db01.fale.io] => (item=/etc/iproute2/rt_scopes)
changed: [ws01.fale.io] => (item=/etc/iproute2/rt_scopes)
changed: [ws02.fale.io] => (item=/etc/iproute2/rt_scopes)
changed: [db01.fale.io] => (item=/etc/iproute2/rt_realms)
changed: [ws01.fale.io] => (item=/etc/iproute2/rt_realms)
changed: [ws02.fale.io] => (item=/etc/iproute2/rt_realms)


PLAY RECAP *******************************************************
db01.fale.io      : ok=3    changed=2    unreachable=0    failed=0   
ws01.fale.io      : ok=3    changed=2    unreachable=0    failed=0   
ws02.fale.io      : ok=3    changed=2    unreachable=0    failed=0   

---
- hosts: all
  remote_user: ansible
  tasks:
  - name: Create the folders /tmp/dirXY with XY from 1 to 10
    file:
      dest: '/tmp/dir{{ item }}'
      state: directory
    with_sequence: start=1 end=10
    become: True

ansible-playbook -i hosts with_sequence.yaml

PLAY [all] *******************************************************
TASK [setup] *****************************************************
ok: [db01.fale.io]
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [Create the folders /tmp/dirXY with XY from 1 to 10] ********
changed: [ws02.fale.io] => (item=1)
changed: [ws01.fale.io] => (item=1)
changed: [db01.fale.io] => (item=1)
changed: [db01.fale.io] => (item=2)
changed: [ws02.fale.io] => (item=2)
changed: [ws01.fale.io] => (item=2)
changed: [db01.fale.io] => (item=3)
changed: [ws01.fale.io] => (item=3)
changed: [ws02.fale.io] => (item=3)
changed: [db01.fale.io] => (item=4)
changed: [ws01.fale.io] => (item=4)
changed: [ws02.fale.io] => (item=4)
changed: [db01.fale.io] => (item=5)
changed: [ws01.fale.io] => (item=5)
changed: [ws02.fale.io] => (item=5)
changed: [db01.fale.io] => (item=6)
changed: [ws01.fale.io] => (item=6)
changed: [ws02.fale.io] => (item=6)
changed: [db01.fale.io] => (item=7)
changed: [ws01.fale.io] => (item=7)
changed: [ws02.fale.io] => (item=7)
changed: [db01.fale.io] => (item=8)
changed: [ws01.fale.io] => (item=8)
changed: [ws02.fale.io] => (item=8)
changed: [db01.fale.io] => (item=9)
changed: [ws01.fale.io] => (item=9)
changed: [ws02.fale.io] => (item=9)
changed: [db01.fale.io] => (item=10)
changed: [ws01.fale.io] => (item=10)
changed: [ws02.fale.io] => (item=10)


PLAY RECAP *******************************************************
db01.fale.io      : ok=2    changed=1    unreachable=0    failed=0   
ws01.fale.io      : ok=2    changed=1    unreachable=0    failed=0   
ws02.fale.io      : ok=2    changed=1    unreachable=0    failed=0   

In this chapter, we have seen a large number of concepts that will help scale your infrastructure beyond the single node. We started with inventories files used to instruct Ansible about our machines, then how to have host-specific and group-specific variables while running the same command on multiple heterogeneous hosts. We then moved on to dynamics inventories that are populated directly by some other system (usually a cloud provider). In the end, we analyzed multiple kinds of iteration in the Ansible playbooks.

In the next chapter, we will structure our Ansible files in a saner way to ensure maximum readability. To do this, we introduce roles which simplify the management of complex environments even more .

The local_action feature of Ansible is a powerful one, especially when we think of Orchestration. This feature allows you to run certain tasks locally on the machine that runs Ansible.

Consider the following situations:

These are tasks that can be run on the same machine that runs the ansible-playbook command rather than logging in to a remote box and running these commands.

Let's look at an example. Suppose you want to run a shell module on your local system where you are running your Ansible playbook. The local_action option comes into the picture in such situations. If you pass the module name and the module argument to local_action, it will run that module locally. Let's see how this option works with the shell module. Consider the following code that shows the output of the local_action option:

    --- 
    - hosts: database 
      remote_user: ansible 
      tasks: 
      - name: Count processes running on the remote system 
        shell: ps | wc -l 
        register: remote_processes_number 
      - name: Print remote running processes 
        debug: 
          msg: '{{ remote_processes_number.stdout }}' 
      - name: Count processes running on the local system 
        local_action: shell ps | wc -l 
        register: local_processes_number 
      - name: Print local running processes 
        debug: 
          msg: '{{ local_processes_number.stdout }}' 

We can now save it as local_action.yaml and run it with the following:

ansible-playbook -i hosts local_action.yaml

We receive the following result:

PLAY [database] **************************************************
TASK [setup] *****************************************************
ok: [db01.fale.io]


TASK [Count processes running on the remote system] **************
changed: [db01.fale.io]


TASK [Print remote running processes] ****************************
ok: [db01.fale.io] => {
    "msg": "7"
}


TASK [Count processes running on the local system] ***************
changed: [db01.fale.io -> localhost]


TASK [Print local running processes] *****************************
ok: [db01.fale.io] => {
    "msg": "11"
}


PLAY RECAP *******************************************************
db01.fale.io      : ok=5    changed=2    unreachable=0    failed=0   

As you can see, the two commands provided us different numbers since they have been executed on different hosts. You can run any module with local_action, and Ansible will make sure that the module is run locally on the box where the ansible-playbook command is run. Another simple example you can (and should!) try is running two tasks:

This will crystallize the idea of local_action further.

Ansible provides another method to delegate certain actions to a specific (or different) machine: the delegate_to system.

Sometimes you want to execute an action on a different system. This could be, for instance, a database node while you are deploying something on an application server node or to the local host. To do so, you can just add the 'delegate_to: HOST' property to your task and it will be run on the proper node. Let's rework the previous example to achieve this:

    --- 
    - hosts: database 
      remote_user: ansible 
      tasks: 
      - name: Count processes running on the remote system 
        shell: ps | wc -l 
        register: remote_processes_number 
      - name: Print remote running processes 
        debug: 
          msg: '{{ remote_processes_number.stdout }}' 
      - name: Count processes running on the local system 
        shell: ps | wc -l 
        delegate_to: localhost 
        register: local_processes_number 
      - name: Print local running processes 
        debug: 
          msg: '{{ local_processes_number.stdout }}' 

Saving it as delegate_to.yaml, we can run it with the following:

ansible-playbook -i hosts delegate_to.yaml

We will receive the same output as the previous example:

PLAY [database] **************************************************
TASK [setup] *****************************************************
ok: [db01.fale.io]


TASK [Count processes running on the remote system] **************
changed: [db01.fale.io]


TASK [Print remote running processes] ****************************
ok: [db01.fale.io] => {
    "msg": "7"
}


TASK [Count processes running on the local system] ***************
changed: [db01.fale.io -> localhost]


TASK [Print local running processes] *****************************
ok: [db01.fale.io] => {
    "msg": "11"
}


PLAY RECAP *******************************************************
db01.fale.io      : ok=5    changed=2    unreachable=0    failed=0   

Until now, we have only seen how playbooks work and how tasks are executed. We also saw that Ansible executes all these tasks sequentially. However, this would not help you while writing an advanced playbook that contains tens of tasks and have to execute only a subset of these tasks. For example, let's say you have a playbook that will install Apache HTTPd server on the remote host. Now, the Apache HTTPd server has a different package name for a Debian-based operating system, and it's called apache2; for a Red-Hat-based operating system, it's called httpd.

Having two tasks, one for the httpd package (for Red-Hat-based systems) and the other for the apache2 package (for Debian-based systems) in a playbook, will make Ansible install both packages, and this execution will fail, as apache2 will not be available if you're installing on a Red-Hat-based operating system. To overcome such problems, Ansible provides conditional statements that help run a task only when a specified condition is met. In this case, we do something similar to the following pseudocode:

    If os = "redhat" 
      Install httpd 
    Else if os = "debian" 
      Install apache2 
    End 

While installing httpd on a Red-Hat-based operating system, we first check whether the remote system is running a Red-Hat-based operating system, and if it is, we then install the httpd package; otherwise, we skip the task. Without wasting your time, let's dive into an example playbook called conditional_httpd.yaml with the following content:

    --- 
    - hosts: webserver 
      remote_user: ansible 
      tasks: 
      - name: Print the ansible_os_family value 
        debug: 
          msg: '{{ ansible_os_family }}' 
      - name: Ensure the httpd package is updated 
        yum: 
          name: httpd 
          state: latest 
        become: True 
        when: ansible_os_family == 'RedHat' 
      - name: Ensure the apache2 package is updated 
        apt: 
          name: apache2 
          state: latest 
        become: True 
        when: ansible_os_family == 'Debian' 

Run it with the following:

ansible-playbook -i hosts conditional_httpd.yaml

This is the result:

PLAY [webserver] *************************************************
TASK [setup] *****************************************************
ok: [ws03.fale.io]
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [Print the ansible_os_family value] *************************
ok: [ws01.fale.io] => {
    "msg": "RedHat"
}
ok: [ws02.fale.io] => {
    "msg": "RedHat"
}
ok: [ws03.fale.io] => {
    "msg": "Debian"
}


TASK [Ensure the httpd package is updated] ***********************
skipping: [ws03.fale.io]
changed: [ws01.fale.io]
changed: [ws02.fale.io]


TASK [Ensure the apache2 package is updated] *********************
skipping: [ws02.fale.io]
skipping: [ws01.fale.io]
changed: [ws03.fale.io]


PLAY RECAP *******************************************************
ws01.fale.io      : ok=3    changed=1    unreachable=0    failed=0   
ws02.fale.io      : ok=3    changed=1    unreachable=0    failed=0   
ws03.fale.io      : ok=3    changed=1    unreachable=0    failed=0   

As you can see, I've created a new server (ws03) for this example that is Debian-based. As expected, the installation of the httpd package was performed on the two CentOS nodes, while the installation of the apache2 package was performed on the Debian node.

Likewise, you can match for different conditions as well. Ansible supports equal to (==), different than (!=), bigger than (>), smaller than (<), bigger than or equal to (>=), and smaller than or equal to (<=).

The operators we have seen so far will match the entire content of the variable, but what if you just want to check whether a particular character or a string is present in a variable? To perform these kinds of checks, Ansible provides the in and not operators. You can also match multiple conditions using the AND and OR operators. The AND operator will make sure that all conditions are matched before executing this task, whereas the OR operator will make sure that at least one of the conditions there is a match for at least one of the conditions, for example, you can use foo >= 0 and foo <= 5.

    --- 
    - hosts: all 
      remote_user: ansible 
      vars: 
        backup: True 
      tasks: 
      - name: Copy the crontab in tmp if the backup variable is true 
        copy: 
          src: /etc/crontab 
          dest: /tmp/crontab 
          remote_src: True 
        when: backup 

ansible-playbook -i hosts crontab_backup.yaml

PLAY [all] *******************************************************
TASK [setup] *****************************************************
ok: [db01.fale.io]
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [Copy the crontab in tmp if the backup variable is true] ****
changed: [ws02.fale.io]
changed: [db01.fale.io]
changed: [ws01.fale.io]


PLAY RECAP *******************************************************
db01.fale.io      : ok=2    changed=1    unreachable=0    failed=0   
ws01.fale.io      : ok=2    changed=1    unreachable=0    failed=0   
ws02.fale.io      : ok=2    changed=1    unreachable=0    failed=0   

ansible-playbook -i hosts crontab_backup.yaml --extra-vars="backup=False"

PLAY [all] *******************************************************
TASK [setup] *****************************************************
ok: [db01.fale.io]
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [Copy the crontab in tmp if the backup variable is true] ****
skipping: [ws01.fale.io]
skipping: [ws02.fale.io]
skipping: [db01.fale.io]


PLAY RECAP *******************************************************
db01.fale.io      : ok=1    changed=0    unreachable=0    failed=0   
ws01.fale.io      : ok=1    changed=0    unreachable=0    failed=0   
ws02.fale.io      : ok=1    changed=0    unreachable=0    failed=0   

    --- 
    - hosts: all 
      remote_user: ansible 
      vars: 
        backup: True 
      tasks: 
      - name: Check if the backup_folder is set 
        fail: 
          msg: 'The backup_folder needs to be set' 
        when: backup_folder is not defined 
      - name: Copy the crontab in tmp if the backup variable is true 
        copy: 
          src: /etc/crontab 
          dest: '{{ backup_folder }}/crontab' 
          remote_src: True 
        when: backup 

The include feature helps you to reduce duplicity while writing tasks. This also allows us to have smaller playbooks by including reusable code in separate tasks using the Don't Repeat Yourself (DRY) principle.

To trigger the inclusion of another file, you need to put the following under the tasks object:

- include: FILENAME.yaml

You can also pass some variables to the included file. To do so, we can specify them in the following way:

- include: FILENAME.yaml variable1="value1" variable2="value2"

In addition of passing variables, you can also use conditionals to include a file only when certain conditions are matched, for instance to include the redhat.yaml  file only if the machine is running an OS in the Red Hat family using the following code:

    - name: Include the file only for Red Hat OSes
    include: redhat.yaml
    when: ansible_os_family == "RedHat"

In many situations, you will have a task or a group of tasks that change certain resources on the remote machines, which need to trigger an event to become effective. For example, when you change a service configuration, you will need to restart or reload the service itself. In Ansible you can trigger this event using the notify action.

Every handler task will run at the end of the playbook if notified. For example, you changed your HTTPd server configuration multiple times and you want to restart the HTTPd service so that the changes are applied. Now, restarting HTTPd every single time you make a configuration change is not a good practice; it is not a good practice to restart the server even if no changes has been made to its configurations. To deal with such a situation, you can notify Ansible to restart the HTTPd service on every configuration change, but Ansible will make sure that no matter how many times you notify it for the HTTPd restart, it will call that task just once after all other tasks complete. Let's change the webserver.yaml file we created in the previous chapters a little bit, in the following way:

--- 
- hosts: webserver 
  remote_user: ansible 
  tasks: 
  - name: Ensure the HTTPd package is installed 
    yum: 
      name: httpd 
      state: present 
    become: True 
  - name: Ensure the HTTPd service is enabled and running 
    service: 
      name: httpd 
      state: started 
      enabled: True 
    become: True 
  - name: Ensure HTTP can pass the firewall 
    firewalld: 
      service: http 
      state: enabled 
      permanent: True 
      immediate: True 
    become: True 
  - name: Ensure HTTPd configuration is updated 
    copy: 
      src: website.conf 
      dest: /etc/httpd/conf.d 
    become: True 
    notify: Restart HTTPd 
  handlers: 
  - name: Restart HTTPd 
    service: 
      name: httpd 
      state: restarted 
    become: True 

Run this script with:

ansible-playbook -i hosts webserver.yaml

We will have the following output:

PLAY [webserver] *************************************************
TASK [setup] *****************************************************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [Ensure the HTTPd package is installed] *********************
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [Ensure the HTTPd service is enabled and running] ***********
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [Ensure HTTP can pass the firewall] *************************
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [Ensure HTTPd configuration is updated] *********************
changed: [ws02.fale.io]
changed: [ws01.fale.io]


RUNNING HANDLER [Restart HTTPd] **********************************
changed: [ws02.fale.io]
changed: [ws01.fale.io]


PLAY RECAP *******************************************************
ws01.fale.io      : ok=6    changed=2    unreachable=0    failed=0   
ws02.fale.io      : ok=6    changed=2    unreachable=0    failed=0   

In this case, the handler has been triggered from the configuration file change. But if we run it a second time, the configuration will not change and therefore we will have the following result:

PLAY [webserver] *************************************************
TASK [setup] *****************************************************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [Ensure the HTTPd package is installed] *********************
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [Ensure the HTTPd service is enabled and running] ***********
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [Ensure HTTP can pass the firewall] *************************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [Ensure HTTPd configuration is updated] *********************
ok: [ws02.fale.io]
ok: [ws01.fale.io]


PLAY RECAP *******************************************************
ws01.fale.io      : ok=5    changed=0    unreachable=0    failed=0   
ws02.fale.io      : ok=5    changed=0    unreachable=0    failed=0

We have seen how we can automate simple tasks, but what we have seen up till now will not solve all your problems. This is because playbooks are very good at executing operations, but are not very good for configuring huge amounts of machines, because they will soon become messy. To solve this, Ansible has roles.

My definition of a role is a set of playbooks, templates, files, or variables to achieve a specific goal. For instance, we could have a database role and a web server role so that those configurations stay cleanly separated.

Before starting to look inside a role, let's talk about a project organization.

    ansible.cfg
    hosts
    master.yaml
    playbooks
        firstrun.yaml
        groups
            database.yaml
            webserver.yaml
    roles
         common
         database
         webserver

    --- 
    - name: Ensure EPEL is enabled 
      yum: 
        name: epel-release 
        state: present 
      become: True 
    - name: Ensure libselinux-python is present 
      yum: 
        name: libselinux-python 
        state: present 
      become: True 
    - name: Ensure libsemanage-python is present 
      yum: 
        name: libsemanage-python 
        state: present 
      become: True 
    - name: Ensure we have last version of every package 
      yum: 
        name: "*" 
        state: latest 
      become: True 
    - name: Ensure NTP is installed 
      yum: 
        name: ntp 
        state: present 
      become: True 
    - name: Ensure the timezone is set to UTC 
      file: 
        src: /usr/share/zoneinfo/GMT 
        dest: /etc/localtime 
        state: link 
      become: True 
    - name: Ensure the NTP service is running and enabled 
      service: 
        name: ntpd 
        state: started 
        enabled: True 
      become: True 
    - name: Ensure FirewallD is installed 
      yum: 
        name: firewalld 
        state: present 
      become: True 
    - name: Ensure FirewallD is running 
      service: 
        name: firewalld 
        state: started 
        enabled: True 
      become: True 
    - name: Ensure SSH can pass the firewall 
      firewalld: 
        service: ssh 
        state: enabled 
        permanent: True 
        immediate: True 
      become: True 
    - name: Ensure the MOTD file is present and updated 
      template: 
        src: motd 
        dest: /etc/motd 
        owner: root 
        group: root 
        mode: 0644 
      become: True 
    - name: Ensure the hostname is the same of the inventory 
      hostname: 
        name: "{{ inventory_hostname }}" 
      become: True 

    common/ 
        tasks 
            main.yaml 
        templates 
            motd 

    --- 
    - hosts: database 
      user: ansible 
      roles: 
      - common 

    --- 
    - hosts: webserver 
      user: ansible 
      roles: 
      - common 

    [defaults] 
    hostfile = hosts 
    host_key_checking = False 
    roles_path = roles 

    --- 
    - include: playbooks/groups/database.yaml 
    - include: playbooks/groups/webserver.yaml 

ansible-playbook master.yaml

PLAY [database] **************************************************
TASK [setup] *****************************************************
ok: [db01.fale.io]


TASK [common : Ensure EPEL is enabled] ***************************
ok: [db01.fale.io]


TASK [common : Ensure libselinux-python is present] **************
ok: [db01.fale.io]


TASK [common : Ensure libsemanage-python is present] *************
ok: [db01.fale.io]


TASK [common : Ensure we have last version of every package] *****
changed: [db01.fale.io]


TASK [common : Ensure NTP is installed] **************************
ok: [db01.fale.io]


TASK [common : Ensure the timezone is set to UTC] ****************
ok: [db01.fale.io]


TASK [common : Ensure the NTP service is running and enabled] ****
ok: [db01.fale.io]


TASK [common : Ensure FirewallD is installed] ********************
ok: [db01.fale.io]


TASK [common : Ensure FirewallD is running] **********************
ok: [db01.fale.io]


TASK [common : Ensure SSH can pass the firewall] *****************
ok: [db01.fale.io]


TASK [common : Ensure the MOTD file is present and updated] ******
ok: [db01.fale.io]


TASK [common : Ensure the hostname is the same of the inventory] *
ok: [db01.fale.io]


PLAY [webserver] *************************************************
TASK [setup] *****************************************************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure EPEL is enabled] ***************************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure libselinux-python is present] **************
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [common : Ensure libsemanage-python is present] *************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure we have last version of every package] *****
changed: [ws01.fale.io]
changed: [ws02.fale.io]


TASK [common : Ensure NTP is installed] **************************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure the timezone is set to UTC] ****************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure the NTP service is running and enabled] ****
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [common : Ensure FirewallD is installed] ********************
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [common : Ensure FirewallD is running] **********************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure SSH can pass the firewall] *****************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure the MOTD file is present and updated] ******
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure the hostname is the same of the inventory] *
ok: [ws02.fale.io]
ok: [ws01.fale.io]


PLAY RECAP *******************************************************
db01.fale.io      : ok=13   changed=1    unreachable=0    failed=0   
ws01.fale.io      : ok=13   changed=1    unreachable=0    failed=0   
ws02.fale.io      : ok=13   changed=1    unreachable=0    failed=0

    --- 
    - name: Ensure the HTTPd package is installed 
      yum: 
        name: httpd 
        state: present 
      become: True 
    - name: Ensure the HTTPd service is enabled and running 
      service: 
        name: httpd 
        state: started 
        enabled: True 
      become: True 
    - name: Ensure HTTP can pass the firewall 
      firewalld: 
        service: http 
        state: enabled 
        permanent: True 
        immediate: True 
      become: True 
    - name: Ensure HTTPd configuration is updated 
      copy: 
        src: website.conf 
        dest: /etc/httpd/conf.d 
      become: True 
      notify: Restart HTTPd 
    - name: Ensure the website is present and updated 
      template: 
        src: index.html.j2 
        dest: /var/www/html/index.html 
        owner: root 
        group: root 
        mode: 0644 
      become: True 

    <html> 
        <body> 
            <h1>Hello World!</h1> 
            <p>This page was created on {{ ansible_date_time.date }}.</p> 
            <p>This machine can be reached on the following IP addresses</p> 
            <ul> 
    {% for address in ansible_all_ipv4_addresses %} 
                <li>{{ address }}</li> 
    {% endfor %} 
            </ul> 
        </body> 
    </html> 

    --- 
    - name: Restart HTTPd 
      service: 
        name: httpd 
        state: restarted 
      become: True 

    --- 
    - hosts: webserver 
      user: ansible 
      roles: 
      - common 
      - webserver 

    ansible-playbook playbooks/groups/webserver.yaml

PLAY [webserver] *************************************************
TASK [setup] *****************************************************
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [common : Ensure EPEL is enabled] ***************************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure libselinux-python is present] **************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure libsemanage-python is present] *************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure we have last version of every package] *****
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure NTP is installed] **************************
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [common : Ensure the timezone is set to UTC] ****************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure the NTP service is running and enabled] ****
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure FirewallD is installed] ********************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure FirewallD is running] **********************
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [common : Ensure SSH can pass the firewall] *****************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure the MOTD file is present and updated] ******
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure the hostname is the same of the inventory] *
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [webserver : Ensure the HTTPd package is installed] *********
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [webserver : Ensure the HTTPd service is enabled and running]
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [webserver : Ensure HTTP can pass the firewall] *************
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [webserver : Ensure HTTPd configuration is updated] *********
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [webserver : Ensure the website is present and updated] *****
changed: [ws01.fale.io]
changed: [ws02.fale.io]


PLAY RECAP *******************************************************
ws01.fale.io      : ok=18   changed=1    unreachable=0    failed=0   
ws02.fale.io      : ok=18   changed=1    unreachable=0    failed=0

Before Ansible 2, every task needed to be executed (and completed) on each machine before Ansible issued a new task to all machines. This meant that if you are performing tasks on a hundred machines and one of them is under-performing, all machines will go at the under-performing machine's speed.

With Ansible 2, the execution strategies have been made modular and therefore you can now choose which execution strategy you prefer for your playbooks. You can also write custom execution strategies, but this is beyond the scope of this book. At the moment (in Ansible 2.1) there are only three execution strategies: linear, serial, and free:

In Ansible 2.0 blocks have been made available. Blocks allow you to group tasks in a logical way and they can also help for a better error handling. The majority of properties you can add to a standard task, you can also add it to the blocks. You may need to perform a yum task to install NTPd and enable of the service only if the machine is CentOS. To do so, the following code can be used:

    tasks:
    - block:
       - name: Ensure NTPd is present
       yum:
         name: ntpd
         state: present
       - name: Ensure NTPd is running
       service:
         name: ntpd
         state: started
       enabled: True
     when: ansible_distribution == 'CentOS'

As you can notice, the when clause has been applied to the block so all tasks within the block will be performed only if the when clause will be true.

We have seen in the second chapter that templates allow you to dynamically complete your playbook and place files on servers based on dynamic data such as host and group variables. In this section, we will move forward and see how Jinja2 filters work with Ansible.

Jinja2 filters are simple Python functions that take some arguments, process them, and return the result. For example, consider the following command:

{{ myvar | filter }}

In the preceding example, myvar is a variable; Ansible will pass myvar to the Jinja2 filter as an argument. The Jinja2 filter will then process it and return the resulting data. Jinja2 filters even accept additional arguments as follows:

{{ myvar | filter(2) }}

In this example, Ansible will now pass two arguments, that is, myvar and 2. Likewise, you can pass multiple arguments to filters separated by commas.

Ansible supports a wide variety of Jinja2 filters, we will see some of the important Jinja2 filters that you might need to use while writing your playbook.

{{ users | to_nice_json }}

{{ users | to_nice_yaml }}

    --- 
    - hosts: webserver 
      remote_user: ansible 
      tasks: 
      - name: Checking HTTPd service status 
        service: 
          name: httpd 
          state: running 
        register: httpd_result 
        ignore_errors: true 
      - debug: 
          msg: Previous task failed 
        when: httpd_result|failed 

ansible-playbook playbooks/http_status.yaml

PLAY [webserver] *************************************************
TASK [setup] *****************************************************
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [Checking HTTPd service status] *****************************
ok: [ws02.fale.io]
fatal: [ws01.fale.io]: FAILED! => {"changed": false, "failed": true, "msg": "Failed to start httpd.service: Interactive authentication required.\n"}
...ignoring


TASK [debug] *****************************************************
skipping: [ws02.fale.io]
ok: [ws01.fale.io] => {
    "msg": "Previous task failed"
}


PLAY RECAP *******************************************************
ws01.fale.io      : ok=3    changed=0    unreachable=0    failed=0   
ws02.fale.io      : ok=2    changed=0    unreachable=0    failed=0  

{{ backup_disk | default("/dev/sdf") }}

{{['a', 'b', 'c', 'd'] | random}}

{{100 | random}}

{{50 | random(10)}}

{{50 | random(20, 10)}}

{{["This", "is", "a", "string"] | join(" ")}}

      {{variable | b64encode}}

      {{"aGFoYWhhaGE=" | b64decode}}

The last section in this chapter is about security management. If you tell your sysadmin that you want to introduce a new feature or a tool, one of the first questions they would ask you would be; "what security feature(s) are present with your tool?". We'll try to answer these questions from an Ansible perspective in this section. Let's look at them in greater detail.

echo 'ansible' > .password

ansible-vault create secret.yaml

cat secret.yaml

$ANSIBLE_VAULT;1.1;AES256
66346431333933663461383331393763666538373163336536353335646532323135383630646366
3432353561393533623764323961666639326132323331370a636363613032616664333039356565
64643735626162646166313861366532323161646137333634333336393062303461343638333737
6534326135326430390a643739336461616334313833313363343030666662653864353138666233
38386266383866353836373036303339383962363362333364346432613062363830316330653866
6431343764386132663066303761346532643632633432643861

ansible-vault --vault-password-file=.password edit secret.yaml

ansible-vault --vault-password-file=.password view secret.yaml

ansible-vault --vault-password-file=.password decrypt secret.yaml

This is a password protected file

ansible-vault --vault-password-file=.password encrypt secret.yaml

ansible-vault --vault-password-file=.password --new-vault-password-file=.newpassword rekey secret.yaml

$ANSIBLE_VAULT;1.1;AES256
63313864643434663939333132333537336362313133616430376463613833353366326662303832
6431316131613033343266373137356166383564326234300a386236633635333939333234643435
64353932383930613934343730386635333030373663313631646462613566313362313363393135
3935613661373263330a316634333536653461356535383662376464656466623536363537386462
31636637346538636161616632313866366365666361633138666134303433316665376237326162
3638653738383830323430313161336465323264613634323434

$ ansible-playbook site.yml --vault-password-file .password

$ ansible-playbook site.yml --vault-password-file ~/.get_password.py

    vars_prompt:
    - name: ssh_password 
      prompt: Enter ssh_password 
      private: True 
      encryption: md5_crypt 
      confirm: True 
      salt_size: 7 

name: ssh_password

prompt: Enter ssh password

private: True

encrypt: md5_crypt
salt_size: 7

confirm: True

- name: Running a script
  shell: script.sh
    password: my_password

REMOTE_MODULE command script.sh password=VALUE_HIDDEN #USE_SHELL

    - name: Running a script
      shell: script.sh
        password: my_password
      no_log: True

In this chapter, we have seen a very large number of Ansible features. We started with local_actions for performing operations on a machine, then we moved to the delegation for performing the task on a third machine. We then moved to conditionals and include for making playbooks more flexible. We learned about roles and how they can help you keep your system aligned and learned how to organize an Ansible repository properly, making the most of Ansible and Git. Later, we covered execution strategies and Jinja filters for more flexible executions.

We ended this chapter with Ansible vault and many other tips to make your Ansible execution safer.

In the next chapter, we will be looking at how to use Ansible to create infrastructures and more specifically, how to do it using the cloud providers, AWS and DigitalOcean.

With that, let's jump to the first topic. Teams managing infrastructures have a lot of choices today for running their builds, tests, and deployments. Providers such as Amazon, Rackspace, and DigitalOcean primarily provide Infrastructure as a Service (IaaS). When we speak about IaaS, it's better to speak about resources not virtual machines for different reasons:

Those companies usually provide API, CLI, GUI, and SDK utilities to create and manage cloud resources throughout their whole lifecycle. We're more interested in using their SDK as it will play an important part in our automation effort. Setting up new servers and provisioning them is interesting at first but at some stage it can become boring as it's quite repetitive in nature. Each provisioning step will involve several similar steps to get them up-and-running.

Imagine one fine morning you receive an e-mail asking for three new customer setups, where each customer setup has three to four instances and a bunch of services and dependencies. This might be an easy task for you, but would require running the same set of repetitive commands multiple times, followed by monitoring the servers once they come up to confirm that everything went well. In addition, anything you do manually has a chance of introducing problems. What if two of the customer setups come up correctly but, due to fatigue, you miss out a step for the third customer and hence introduce a problem? To deal with such situations, there exists automation.

Cloud provisioning automation makes it easy for an engineer to build up a new server as quickly as possible, allowing her to concentrate on other priorities. Using Ansible, you can easily perform these actions and automate cloud provisioning with minimal effort. Ansible provides you with the power to automate various different cloud platforms, such as Amazon, Azure, DigitalOcean, Google Cloud, Rackspace, and many more, with modules for different services available in the Ansible core or extended module packages.

In the next sections we will provision the environment that we have used in the previous chapters (two web servers and one database server) in the following environments:

Amazon Web Service is the most used public cloud by a fair amount and it's often chosen due to their huge amount of available services as well as the huge amount of documentation, answered questions, and articles that can be expected from such a popular product.

Since AWS' goal is to be a complete virtual data center provider (and much more) we will need to create and manage our network as we would do if we had to set up a real data center. Obviously, we will not need to cable stuff since it's a virtual data center. Due to this, a few lines of an Ansible playbook will be enough.

    [default] 
    aws_access_key_id = YOUR_ACCESS_KEY 
    aws_secret_access_key = YOUR_SECRET_KEY 

    - hosts: localhost 
      tasks:

    To create the VPC subnetwork: 
      - name: Ensure the VPC subnetwork is present 
        ec2_vpc_subnet: 
          state: present 
          az: AWS_AZ 
          vpc_id: '{{ aws_simple_net.vpc_id }}' 
          cidr: 10.0.1.0/24 
        register: aws_subnet 

      - name: Ensure websg Security Group is present 
        ec2_group: 
          name: web 
          description: Web Security Group 
          region: AWS_AZ 
          vpc_id: VPC_ID 
          rules: 
          - proto: tcp 
            from_port: 80 
            to_port: 80 
            cidr_ip: 0.0.0.0/0 
          - proto: tcp 
            from_port: 443 
            to_port: 443 
            cidr_ip: 0.0.0.0/0 
          rules_egress: 
          - proto: all 
            cidr_ip: 0.0.0.0/0 
        register: aws_simple_websg 

      - name: Ensure wssg Security Group is present 
        ec2_group: 
          name: wssg 
          description: Web Security Group 
          region: eu-west-1 
          vpc_id: '{{ aws_simple_net.vpcs.0.id }}' 
          rules: 
          - proto: tcp 
            from_port: 22 
            to_port: 22 
            cidr_ip: 0.0.0.0/0 
          - proto: tcp 
            from_port: 80 
            to_port: 80 
            cidr_ip: 0.0.0.0/0 
          - proto: tcp 
            from_port: 443 
            to_port: 443 
            cidr_ip: 0.0.0.0/0 
          rules_egress: 
          - proto: all 
            cidr_ip: 0.0.0.0/0 
        register: aws_simple_wssg 

      - name: Ensure dbsg Security Group is present 
        ec2_group: 
          name: dbsg 
          description: DB Security Group 
          region: eu-west-1 
          vpc_id: '{{ aws_simple_net.vpcs.0.id }}' 
          rules: 
          - proto: tcp 
            from_port: 3036 
            to_port: 3036 
            group_id: '{{ aws_simple_wssg.group_id }}' 
          rules_egress: 
          - proto: all 
            cidr_ip: 0.0.0.0/0 
        register: aws_simple_dbsg 

      - name: Setup instances 
        ec2: 
          assign_public_ip: '{{ item.assign_public_ip }}' 
          image: ami-7abd0209 
          region: eu-west-1 
          exact_count: 1 
          key_name: fale 
          count_tag: 
            Name: '{{ item.name }}' 
          instance_tags: 
            Name: '{{ item.name }}' 
          instance_type: t2.micro 
          group_id: '{{ item.group_id }}' 
          vpc_subnet_id: '{{ aws_simple_subnet.subnets.0.id }}' 
          volumes: 
            - device_name: /dev/sda1 
              volume_type: gp2 
              volume_size: 10 
              delete_on_termination: True 
        register: aws_simple_instances 
        with_items: 
        - name: ws01.simple.aws.fale.io 
          group_id: '{{ aws_simple_wssg.group_id }}' 
          assign_public_ip: True 
        - name: ws02.simple.aws.fale.io 
          group_id: '{{ aws_simple_wssg.group_id }}' 
          assign_public_ip: True 
        - name: db01.simple.aws.fale.io 
          group_id: '{{ aws_simple_dbsg.group_id }}' 
          assign_public_ip: False 

      - name: Add route53 entry for server SERVER_NAME 
        route53: 
          command: create 
          zone: ZONE_NAME 
          record: RECORD_TO_ADD 
          type: RECORD_TYPE 
          ttl: TIME_TO_LIVE 
          value: IP_VALUES 
          wait: True 

      - name: Add route53 rules for instances 
        route53: 
          command: create 
          zone: aws.fale.io 
          record: '{{ item.tagged_instances.0.tags.Name }}' 
          type: A 
          ttl: 1 
          value: '{{ item.tagged_instances.0.public_ip }}' 
          wait: True 
        with_items: '{{ aws_simple_instances.results }}' 
        when: item.tagged_instances.0.public_ip 
      - name: Add internal route53 rules for instances 
        route53: 
          command: create 
          zone: aws.fale.io 
          private_zone: True 
          record: '{{ item.tagged_instances.0.tags.Name }}' 
          type: A 
          ttl: 1 
          value: '{{ item.tagged_instances.0.private_ip }}' 
          wait: True 
        with_items: '{{ aws_simple_instances.results }}' 

    - hosts: localhost 
      tasks: 
      - name: Gather information of the EC2 VPC net in eu-west-1 
        ec2_vpc_net_facts: 
          region: eu-west-1 
        register: aws_simple_net 
      - name: Gather information of the EC2 VPC subnet in eu-west-1 
        ec2_vpc_subnet_facts: 
          region: eu-west-1 
          filters: 
            vpc-id: '{{ aws_simple_net.vpcs.0.id }}' 
        register: aws_simple_subnet 
      - name: Ensure wssg Security Group is present 
        ec2_group: 
          name: wssg 
          description: Web Security Group 
          region: eu-west-1 
          vpc_id: '{{ aws_simple_net.vpcs.0.id }}' 
          rules: 
          - proto: tcp 
            from_port: 22 
            to_port: 22 
            cidr_ip: 0.0.0.0/0 
          - proto: tcp 
            from_port: 80 
            to_port: 80 
            cidr_ip: 0.0.0.0/0 
          - proto: tcp 
            from_port: 443 
            to_port: 443 
            cidr_ip: 0.0.0.0/0 
          rules_egress: 
          - proto: all 
            cidr_ip: 0.0.0.0/0 
        register: aws_simple_wssg 
      - name: Ensure dbsg Security Group is present 
        ec2_group: 
          name: dbsg 
          description: DB Security Group 
          region: eu-west-1 
          vpc_id: '{{ aws_simple_net.vpcs.0.id }}' 
          rules: 
          - proto: tcp 
            from_port: 3036 
            to_port: 3036 
            group_id: '{{ aws_simple_wssg.group_id }}' 
          rules_egress: 
          - proto: all 
            cidr_ip: 0.0.0.0/0 
        register: aws_simple_dbsg 
      - name: Setup instances 
        ec2: 
          assign_public_ip: '{{ item.assign_public_ip }}' 
          image: ami-7abd0209 
          region: eu-west-1 
          exact_count: 1 
          key_name: fale 
          count_tag: 
            Name: '{{ item.name }}' 
          instance_tags: 
            Name: '{{ item.name }}' 
          instance_type: t2.micro 
          group_id: '{{ item.group_id }}' 
          vpc_subnet_id: '{{ aws_simple_subnet.subnets.0.id }}' 
          volumes: 
            - device_name: /dev/sda1 
              volume_type: gp2 
              volume_size: 10 
              delete_on_termination: True 
        register: aws_simple_instances 
        with_items: 
        - name: ws01.simple.aws.fale.io 
          group_id: '{{ aws_simple_wssg.group_id }}' 
          assign_public_ip: True 
        - name: ws02.simple.aws.fale.io 
          group_id: '{{ aws_simple_wssg.group_id }}' 
          assign_public_ip: True 
        - name: db01.simple.aws.fale.io 
          group_id: '{{ aws_simple_dbsg.group_id }}' 
          assign_public_ip: False 
      - name: Add route53 rules for instances 
        route53: 
          command: create 
          zone: aws.fale.io 
          record: '{{ item.tagged_instances.0.tags.Name }}' 
          type: A 
          ttl: 1 
          value: '{{ item.tagged_instances.0.public_ip }}' 
          wait: True 
        with_items: '{{ aws_simple_instances.results }}' 
        when: item.tagged_instances.0.public_ip 
      - name: Add internal route53 rules for instances 
        route53: 
          command: create 
          zone: aws.fale.io 
          private_zone: True 
          record: '{{ item.tagged_instances.0.tags.Name }}' 
          type: A 
          ttl: 1 
          value: '{{ item.tagged_instances.0.private_ip }}' 
          wait: True 
        with_items: '{{ aws_simple_instances.results }}' 

PLAY [localhost] ***************************************************
TASK [setup] *******************************************************
ok: [localhost]


TASK [Gather information of the EC2 VPC net in eu-west-1] **********
ok: [localhost]


TASK [Gather information of the EC2 VPC subnet in eu-west-1] *******
ok: [localhost]


TASK [Ensure wssg Security Group is present] ***********************
changed: [localhost]


TASK [Ensure dbsg Security Group is present] ***********************
changed: [localhost]


TASK [Setup instances] *********************************************
changed: [localhost] => (item={u'group_id': u'sg-950c2cf2', u'name': u'ws01.simple.aws.fale.io', u'assign_public_ip': True})
changed: [localhost] => (item={u'group_id': u'sg-950c2cf2', u'name': u'ws02.simple.aws.fale.io', u'assign_public_ip': True})
changed: [localhost] => (item={u'group_id': u'sg-940c2cf3', u'name': u'db01.simple.aws.fale.io', u'assign_public_ip': False})


TASK [Add route53 rules for instances] *****************************
changed: [localhost] =>
    ....

changed: [localhost] =>
    ....

skipping: [localhost] =>
    ....

TASK [Add internal route53 rules for instances] ******************
changed: [localhost] =>
    ....

changed: [localhost] =>
    ....

changed: [localhost] =>
    ....

PLAY RECAP ****************************************************
localhost                  : ok=7    changed=4    unreachable=0    failed=0

      - name: Setup instances 
        ec2: 
          assign_public_ip: '{{ item.assign_public_ip }}' 
          image: ami-7abd0209 
          region: eu-west-1 
          exact_count: 1 
          key_name: fale 
          count_tag: 
            Name: '{{ item.name }}' 
          instance_tags: 
            Name: '{{ item.name }}' 
          instance_type: t2.micro 
          group_id: '{{ item.group_id }}' 
          vpc_subnet_id: '{{ item.vpc_subnet_id }}' 
          volumes: 
            - device_name: /dev/sda1 
              volume_type: gp2 
              volume_size: 10 
              delete_on_termination: True 
        register: aws_simple_instances 
        with_items: 
        - name: ws01.simple.aws.fale.io 
          group_id: '{{ aws_simple_wssg.group_id }}' 
          assign_public_ip: True 
          vpc_subnet_id: '{{ aws_simple_subnet.subnets.0.id }}' 
        - name: ws02.simple.aws.fale.io 
          group_id: '{{ aws_simple_wssg.group_id }}' 
          assign_public_ip: True 
          vpc_subnet_id: '{{ aws_simple_subnet.subnets.1.id }}' 
        - name: db01.simple.aws.fale.io 
          group_id: '{{ aws_simple_dbsg.group_id }}' 
          assign_public_ip: False 
          vpc_subnet_id: '{{ aws_simple_subnet.subnets.0.id }}' 

Compared to Amazon Web Services, DigitalOcean seems to be very incomplete. DigitalOcean, until a few months ago only provided droplets, SSH key management, and DNS management. At the time of writing this, DigitalOcean has very recently launched an additional block storage service. The advantages of DigitalOcean compared to many competitors are:

    - hosts: localhost 
      tasks: 
      - name: Add the SSH Key to Digital Ocean 
        digital_ocean: 
          state: present 
          command: ssh 
          name: SSH_KEY_NAME 
          ssh_pub_key: 'ssh-rsa AAAA...' 
          api_token: XXX 
        register: ssh_key 

    - hosts: localhost 
      tasks: 
      - name: Add the SSH Key to Digital Ocean 
        digital_ocean: 
          state: present 
          command: ssh 
          name: faleKey 
          ssh_pub_key: 'ssh-rsa AAAA...==' 
          api_token: 259...b3b 
        register: ssh_key 

    ansible-playbook -i localhost, playbooks/do_provision.yaml

PLAY [localhost] **************************************************
TASK [setup] ******************************************************
ok: [localhost]


TASK [Add the SSH Key to Digital Ocean] ***************************
changed: [localhost]


PLAY RECAP ********************************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0

      - name: Ensure the ws and db servers are present 
        digital_ocean: 
          state: present 
          ssh_key_ids: KEY_ID 
          name: '{{ item }}' 
          api_token: DIGITAL_OCEAN_KEY 
          size_id: 512mb 
          region_id: lon1 
          image_id: centos-7-0-x64 
          unique_name: True 
        with_items: 
        - WEBSERVER 1 
        - WEBSERVER 2 
        - DBSERVER 1 

      - name: Ensure the ws and db servers are present 
        digital_ocean: 
          state: present 
          ssh_key_ids: '{{ ssh_key.ssh_key.id }}' 
          name: '{{ item }}' 
          api_token: 259...b3b 
          size_id: 512mb 
          region_id: lon1 
          image_id: centos-7-0-x64 
          unique_name: True 
        with_items: 
        - ws01.do.fale.io 
        - ws02.do.fale.io 
        - db01.do.fale.io 
        register: droplets 

      - name: Ensure domain resolve properly
        digital_ocean_domain:
          api_token: 259...b3b
          state: present
          name: '{{ item.droplet.name }}'
          ip: '{{ item.droplet.ip_address }}'
        with_items: '{{ droplets.results }}'

    - hosts: localhost 
      tasks: 
      - name: Add the SSH Key to Digital Ocean 
        digital_ocean: 
          state: present 
          command: ssh 
          name: faleKey 
          ssh_pub_key: 'ssh-rsa AAAA...==' 
          api_token: 7e7...f6f 
        register: ssh_key 
      - name: Ensure the ws and db servers are present 
        digital_ocean: 
          state: present 
          ssh_key_ids: '{{ ssh_key.ssh_key.id }}' 
          name: '{{ item }}' 
          api_token: 259...b3b 
          size_id: 512mb 
          region_id: lon1 
          image_id: centos-7-0-x64 
          unique_name: True 
        with_items: 
        - ws01.do.fale.io 
        - ws02.do.fale.io 
        - db01.do.fale.io 
        register: droplets 
      - name: Ensure domain resolve properly 
        digital_ocean_domain: 
          api_token: 259...b3b 
          state: present 
          name: '{{ item.droplet.name }}' 
          ip: '{{ item.droplet.ip_address }}' 
        with_items: '{{ droplets.results }}' 

ansible-playbook -i localhost, playbooks/do_provision.yaml

PLAY [localhost] **************************************************
TASK [setup] ******************************************************
ok: [localhost]


TASK [Add the SSH Key to Digital Ocean] ***************************
changed: [localhost]


TASK [Ensure the ws and db servers are present] *******************
changed: [localhost] => (item=ws01.do.fale.io)
changed: [localhost] => (item=ws02.do.fale.io)
changed: [localhost] => (item=db01.do.fale.io)


TASK [Ensure domain resolve properly] *****************************
changed: [localhost] =>
    ....


changed: [localhost] =>
    ....

changed: [localhost] =>
    ....

PLAY RECAP ************************************************************
localhost                  : ok=4    changed=3    unreachable=0    failed=0

In this chapter, we have seen how we can provision our machines in both the AWS cloud and the DigitalOcean one. In the case of the AWS cloud, we have seen two different examples, one very simple and one slightly more complex.

In the next chapter, we will talk about getting notified by Ansible if something went wrong.

The easiest and most common way of alerting people is to send e-mails. Ansible allows you to send e-mails from your playbook using a mail module. You can use this module in between any of your tasks and notify your user whenever required. Also, in some cases, you cannot automate each and every thing because either you lack the authority or it requires some manual checking and confirmation. If this is the case, you can notify the responsible user that Ansible has done its job and it's time for him/her to perform his/her duty. Let's see how you can use the mail module to notify your users with a very simple playbook called uptime_and_email.yaml:

    - hosts: localhost
      tasks:
      - name: Read the machine uptime
        command: uptime -p
        register: uptime
      - name: Send the uptime via e-mail
        mail:
          host: mail.fale.io
          username: ansible@fale.io
          password: PASSWORD
          to: me@fale.io
          subject: Ansible-report
          body: 'Local system uptime is {{ uptime.stdout }}.'

In the preceding playbook, we will first read the current machine uptime and then send it via e-mail to someone. This example is very easy and will allow us to keep the examples short, but obviously you can generate the e-mails in a similar way in very long and complex playbooks. If we focus on the mail task a little bit, we can see that we are using it with the following data:

Other interesting parameters that the mail module supports are:

An interesting thing about this module is that the only mandatory field is subject, and not the body, as many people would expect.

We can now proceed to execute the script to validate its functionality with the following:

ansible-playbook -i localhost, uptime_and_email.yaml

We will have a result similar to the following:

PLAY [localhost] *************************************************
TASK [setup] *****************************************************
ok: [localhost]


TASK [Read the machine uptime] ***********************************
changed: [localhost]


TASK [Send the uptime via e-mail] ********************************
changed: [localhost]


PLAY RECAP *******************************************************
localhost         : ok=3    changed=2    unreachable=0    failed=0

Also, as expected, Ansible has sent me an e-mail with the following content:

    Local system uptime is up 38 min.

This module can be used in many different ways. An example of a real world case that I've seen is a playbook that was created to automate a piece of a very long procedure done by multiple people. The procedure, historically, changed owners using the e-mails and every person involved in the procedure was supposed to do their part after an e-mail was received from the owner of the previous piece. They then sent an e-mail at the end of their piece to the next owner. When we started to automate that procedure, we did it for one specific piece and no one noticed that that part was automated. This is not the best way to handle procedures, but it's widely used in organizations and often you cannot change it.

E-mails are slow, unreliable, and often people do not react to them immediately. There are cases where you want to send a real-time message to one of your users. Many organizations rely on XMPP/Jabber for their internal chat system and the great thing is that Ansible is able to directly send messages to XMPP/Jabber users and conference rooms.

Let's tweak the previous example to send uptime information to a user in the file uptime_and_xmpp_user.yaml:

    - hosts: localhost
      tasks:
      - name: Read the machine uptime
        command: 'uptime -p'
        register: uptime
      - name: Send the uptime to user
        jabber:
          user: ansible@fale.io
          password: PASSWORD
          to: me@fale.io
          msg: 'Local system uptime is {{ uptime.stdout }}.'

As you can see, the jabber module is very similar to the mail module and requires similar parameters. In the XMPP case, we don't need to specify the server host and port, since that information is automatically gathered by XMPP from the DNS. In cases where we would need to use a different server host or port, we can use respectively, the host and port parameters.

We can now proceed to execute the script to validate its functionality with the following:

ansible-playbook -i localhost, uptime_and_xmpp_user.yaml

We will have a result similar to the following:

PLAY [localhost] *************************************************
TASK [setup] *****************************************************
ok: [localhost]


TASK [Read the machine uptime] ***********************************
changed: [localhost]


TASK [Send the uptime to user] ***********************************
changed: [localhost]


PLAY RECAP *******************************************************
localhost         : ok=3    changed=2    unreachable=0    failed=0

In cases where we want to send a message to a conference room instead of a single user, it is enough to just change the to parameter, adding the appropriate one, that is:

    to=sysop@conference.fale.io (mailto:sysop@conference.fale.io)/ansiblebot

In the last few years, many new chat and collaboration platforms have appeared. One of the most used ones is Slack. Slack is a cloud-based team collaboration tool, and this allows even easier integration with Ansible.

Let's put the following lines in the file uptime_and_slack.yaml:

    - hosts: localhost
      tasks:
      - name: Read the machine uptime
        command: 'uptime -p'
        register: uptime
      - name: Send the uptime to slack channel
        slack:
          token: TOKEN
          channel: '#ansible'
          msg: 'Local system uptime is {{ uptime.stdout }}.'

As we discussed, this module has an even simpler syntax than the XMPP one, in fact it only needs to know the token (which you can generate on the Slack website), the channel to send the message to, and the message itself.

Run the playbook with the following:

ansible-playbook -i localhost, uptime_and_slack.yaml

This results in the following output:

PLAY [localhost] *************************************************
TASK [setup] *****************************************************
ok: [localhost]


TASK [Read the machine uptime] ***********************************
changed: [localhost]


TASK [Send the uptime to slack channel] **************************
changed: [localhost]


PLAY RECAP *******************************************************
localhost         : ok=3    changed=2    unreachable=0    failed=0

Since Slack's goal is to make communications more efficient, it allows us to tweak multiple aspects of the message. The most interesting points from my point of view are the following:

Many companies like the functionality of Slack, but have problems to tradeoff the privacy that an on-premises service gives you for the Slack functionality. Rocket Chat is open source software that implements most of the features of Slack, as well as the majority of its interface. Being open source, every company can install it on-premises and manage it in a way that is compliant with their IT rules.

As Rocket Chat's goal is to be a drop-in replacement for Slack, from our point of view, very few changes need to be done, in fact, we can create the file uptime_and_rocket.yaml with the following content:

    - hosts: localhost
      tasks:
      - name: Read the machine uptime
        command: 'uptime -p'
        register: uptime
      - name: Send the uptime to rocketchat channel
        rocketchat:
          token: TOKEN
          domain: chat.example.com
          channel: '#ansible'
          msg: 'Local system uptime is {{ uptime.stdout }}.'

As you can see, the only lines that changed are the 6th and 7th, where the word slack has been replaced by rocketchat. Also, we need to add the domain field specifying where our installation of Rocket Chat is located.

Run the code with the following:

ansible-playbook -i localhost, uptime_and_rocketchat.yaml

This results in the following output:

PLAY [localhost] *************************************************
TASK [setup] *****************************************************
ok: [localhost]


TASK [Read the machine uptime] ***********************************
changed: [localhost]


TASK [Send the uptime to rocketchat channel] *********************
changed: [localhost]


PLAY RECAP *******************************************************
localhost         : ok=3    changed=2    unreachable=0    failed=0

IRC is probably the most well-known and widely-used chat protocol of the 1990s and it's still used today, mainly due to its use in open source communities and its simplicity. From an Ansible perspective, IRC is a pretty straightforward module and we can use it as in the following example (to be put in the uptime_and_irc.yaml file):

    - hosts: localhost
      tasks:
      - name: Read the machine uptime
        command: 'uptime -p'
        register: uptime
      - name: Send the uptime to IRC channel
        irc:
          port: 6669
          server: irc.example.net
          channel: #desired_channel
          msg: 'Local system uptime is {{ uptime.stdout }}.'
          color: green

In the IRC module, the following fields are required:

Other configurations you will usually specify are:

Run the code with the following:

ansible-playbook uptime_and_irc.yaml

This results in the following output:

PLAY [localhost] *************************************************
TASK [setup] *****************************************************
ok: [localhost]


TASK [Read the machine uptime] ***********************************
changed: [localhost]


TASK [Send the uptime to IRC channel] ****************************
changed: [localhost]


PLAY RECAP *******************************************************
localhost         : ok=3    changed=2    unreachable=0    failed=0

Sometimes, you want your playbooks to be agnostic in the way you receive the alert. This has several advantages, mainly in terms of flexibility. In fact, in this model, Ansible will deliver the messages to a notification service and the notification service will then take care of delivering them. Amazon Simple Notification Service (SNS) is not the only notification service available, but it's probably the most used. SNS has the following components:

So in our case, we will have, specifically:

As we said, one of the big advantages of SNS is that you can decouple between the way Ansible sends messages (SNS API) and the way your users will receive the messages. In fact, you will be able to choose different delivery systems per user and per topic rules, and eventually you can change them dynamically to ensure that the messages are sent in the best way possible for any situation. The five ways SNS can send messages, at the moment, are:

Let's see how we can send SNS messages with Ansible. To do so, we can create a file called uptime_and_sns.yaml with the following content:

    - hosts: localhost
      tasks:
      - name: Read the machine uptime
        command: 'uptime -p'
        register: uptime
      - name: Send the uptime to SNS
        sns:
          msg: 'Local system uptime is {{ uptime.stdout }}.'
          subject: "System uptime"
          topic: "uptime"

In this example, we are using the msg key to set the message that will be sent, the topic to choose the most appropriate topic, and subject that will be used as the subject for e-mail deliveries. There are many other options you can set. Mainly, they are useful for sending different messages using different delivery methods. For instance, it would make sense to send a short message via SMS (in the end, the first S in SMS means short) and longer and more detailed messages via e-mails. To do so, the SNS module provides us with the following delivery-specific options:

This module allows us also to set three AWS-specific parameters that I've not specified because I have a configuration file for AWS credentials and options:

Run the code with the following command:

ansible-playbook uptime_and_sns.yaml

This will result in the following output:

    PLAY [localhost] *************************************************

    TASK [setup] *****************************************************
    ok: [localhost]


    TASK [Read the machine uptime] ***********************************
    changed: [localhost]


    TASK [Send the uptime to SNS] ************************************
    changed: [localhost]


    PLAY RECAP *******************************************************
    localhost         : ok=3    changed=2    unreachable=0    failed=0

Nagios is one of the most used tools for controlling the status of services and servers. Nagios is capable of regularly auditing the state of servers and services, and notifying users in case of problems. If you have Nagios in your environment, you need to be very careful when you administer your machines, because in cases where Nagios finds servers or services in an unhealthy state, it will start sending e-mails, SMS messages, and calls to your whole team. When you run Ansible scripts against nodes that are controlled by Nagios you have to be even more careful, because you risk e-mails, SMS messages, and calls being triggered during the night or other inappropriate times. To avoid this, Ansible is able to notify Nagios beforehand, so that Nagios does not send notifications in that time window even if some services are down (for instance, because they are rebooted) or other checks fail.

In this example, we are going to stop a service, wait for 5 minutes, then start it again since this would actually create a Nagios failure in the majority of configurations. In fact, usually, Nagios is configured to accept up to two consecutive failures of a test (with usually one execution every minute) putting the service in a warning state before raising a critical state. We are going to create the file, long_restart_service.yaml which will trigger the Nagios critical state:

    - hosts: ws01.fale.io
      tasks:
      - name: Stop the HTTPd service
        service:
          name: httpd
          state: stopped
      - name: Wait for 5 minutes
        pause:
          minutes: 5
      - name: Start the HTTPd service
        service:
          name: httpd
          state: stopped

Run the code with the following:

ansible-playbook long_restart_service.yaml

This should trigger a Nagios alert and result in the following output:

    PLAY [ws01.fale.io] **********************************************

    TASK [setup] *****************************************************
    ok: [ws01.fale.io]


    TASK [Stop the HTTpd service] ************************************
    changed: [ws01.fale.io]


    TASK [Wait for 5 minutes] ****************************************
    changed: [ws01.fale.io]


    TASK [Start the HTTpd service] ***********************************
    changed: [ws01.fale.io]


    PLAY RECAP *******************************************************
    ws01.fale.io      : ok=4    changed=3    unreachable=0    failed=0

We can now create a very similar playbook that will ensure that Nagios will not send any alerts. We are going to create a file called long_restart_service_no_alert.yaml with the following content:

    - hosts: ws01.fale.io
      tasks:
      - name: Silence Nagios
    nagios:
      action: disable_alerts
      service: httpd
      host: '{{ inventory_hostname }}'
    delegate_to: nagios.fale.io
  - name: Stop the HTTPd service
    service:
      name: httpd
      state: stopped
  - name: Wait for 5 minutes
    pause:
      minutes: 5
  - name: Start the HTTPd service
    service:
      name: httpd
      state: stopped
  - name: Desilence Nagios
    nagios:
      action: enable_alerts
      service: httpd
      host: '{{ inventory_hostname }}'
    delegate_to: nagios.fale.io

As you can see, we have added two tasks. The first to inform Nagios not to send alerts for the HTTPd service on the given host, and the second to inform Nagios to start sending alerts for the service again. Even if you do not specify the service and therefore all alerts on that host are silenced, my advice is to disable only the alert you are going to break so that Nagios is still able to work normally on the majority of your infrastructure.

This module's goal is to toggle the Nagios alerts as well as schedule downtime, and from Ansible 2.2 this module can also unscheduled downtimes.

Run the code with the following command:

ansible-playbook long_restart_service_no_alert.yaml

This should trigger a Nagios alert and result in the following output:

PLAY [ws01.fale.io] **********************************************
TASK [setup] *****************************************************
ok: [ws01.fale.io]


TASK [Silence Nagios] ********************************************
changed: [nagios.fale.io]


TASK [Stop the HTTpd service] ************************************
changed: [ws01.fale.io]


TASK [Wait for 5 minutes] ****************************************
changed: [ws01.fale.io]


TASK [Start the HTTpd service] ***********************************
changed: [ws01.fale.io]


TASK [Desilence Nagios] ******************************************
changed: [nagios.fale.io]


PLAY RECAP *******************************************************
ws01.fale.io      : ok=4    changed=3    unreachable=0    failed=0
nagios.fale.io    : ok=2    changed=2    unreachable=0    failed=0

Sometimes, what you want to achieve with a Nagios integration is exactly the opposite, in fact, you are not interested to silentiate it, but you want Nagios to handle your test results. A common case is if you want to leverage your Nagios configuration to notify your administrators of the output of a task. To do so, we can use the Nagios nsca utility, integrating it into our playbooks. Ansible does not yet have a specific module for managing it, but you can always run it using the command module, leveraging the send_nsca CLI program.

In this chapter, we have seen how we can teach Ansible how to send notifications to other systems and/or people.

In the next chapter, we will learn how to create a module so that you can extend Ansible to perform any kind of task.

Ansible intends to allow users to write modules in any language. Writing the module in Python, however, has its own advantages. You can take advantage of Ansible's libraries to shorten your code, an advantage not available for modules in other languages. Parsing user arguments, handling errors, and returning the required values becomes easier with the help of the Ansible libraries. We will see two examples for a custom Python module, one with and one without using the Ansible library, to give you a glimpse of how custom modules work. Make sure you organize your directory structure as mentioned in the previous section before creating the module. The first example creates a module named check_user. To do so, we will need to create the check_user file in the library folder within the Ansible top-level directory, with the following content:

    #!/usr/bin/env python 
 
    import pwd 
    import sys 
    import shlex 
    import json 
 
    def main(): 
        # Parsing argument file 
        args = {} 
        args_file = sys.argv[1] 
        args_data = file(args_file).read() 
        arguments = shlex.split(args_data) 
        for arg in arguments: 
            if '=' in arg: 
                (key, value) = arg.split('=') 
                args[key] = value 
        user = args['user'] 
     
        # Check if user exists 
        try: 
            pwd.getpwnam(user) 
            success = True 
            ret_msg = 'User %s exists' % user 
        except KeyError: 
            success = False 
            ret_msg = 'User %s does not exists' % user 
     
        # Error handling and JSON return 
        if success: 
            print json.dumps({ 
                'msg': ret_msg 
            }) 
            sys.exit(0) 
        else: 
            print json.dumps({ 
                'failed': True, 
                'msg': ret_msg 
            }) 
            sys.exit(1) 
    main() 

The preceding custom module, check_user, will check whether a user exists on a host. The module expects a user argument from Ansible. Let's break down the preceding module and see what it does. We first declare the Interpreter (Python) and import the libraries required to parse the arguments:

    #!/usr/bin/env python 
 
    import pwd 
    import sys 
    import shlex 
    import json 

Using the sys library, we then parse the arguments, which are passed in a file by Ansible. The arguments are in the format param1=value1 param2=value2 where param1 and param2 are parameters and value1 and value2 are values of the parameters. There are multiple ways to split arguments and create a dictionary and we've chosen an easy way to perform the operation. We first create a list of arguments by splitting the arguments with a whitespace character, and then separate the key and value by splitting the arguments with an = character and assigning it to a Python dictionary. For example, if you have a string such as user=foo gid=1000, then you will first create a list, which will look like ["user=foo", "gid=1000"] and then loop over this list to create a dictionary. This dictionary will look like {"user": "foo", "gid": 1000}. This is performed by the following lines:

    def main(): 
        # Parsing argument file 
        args = {} 
        args_file = sys.argv[1] 
        args_data = file(args_file).read() 
        arguments = shlex.split(args_data) 
        for arg in arguments: 
            if '=' in arg: 
                (key, value) = arg.split('=') 
                args[key] = value 
        user = args['user'] 

Once we have the user argument, we then check whether that user exists on the host as follows:

    # Check if user exists 
    try: 
        pwd.getpwnam(user) 
        success = True 
        ret_msg = 'User %s exists' % user 
    except KeyError: 
        success = False 
        ret_msg = 'User %s does not exists' % user 

We use the pwd library to check the passwd file for the user. For the sake of simplicity, we use two variables: one to store the success or failure message and the other to store the message for the user. Finally, we use the variables created in the try-catch block to check if the module succeeded or failed, as you can see in this snippet:

    # Error handling and JSON return 
    if success: 
        print json.dumps({ 
            'msg': ret_msg 
        }) 
        sys.exit(0) 
    else: 
        print json.dumps({ 
            'failed': True, 
            'msg': ret_msg 
        }) 
        sys.exit(1) 

If the module succeeds, then we will exit the execution with an exit code 0 [exit(0)]; else, we will exit with a non-zero code. Ansible will look for the failed variable and if it is set to True, it will exit unless you have explicitly asked Ansible to ignore errors using the ignore_errors parameter. You can use customized modules like any other core module of Ansible. To test the custom module, we will need a playbook, so let's create the file playbooks/check_user.yaml with the following content:

    - hosts: localhost 
      vars: 
        user_ok: root 
        user_ko: this_user_does_not_exists 
      tasks: 
      - name: 'Check if user {{ user_ok }} exists' 
        check_user: 
          user: '{{ user_ok }}' 
      - name: 'Check if user {{ user_ko }} exists' 
        check_user: 
          user: '{{ user_ko }}' 

As you can see, we used the check_user module like any other core module. Ansible will execute this module on the remote host by copying the module to the remote host with arguments in a separate file. Let's see how this playbook runs with the following:

ansible-playbook playbooks/check_user.yaml

We should receive the following output:

PLAY [localhost] *************************************************
TASK [setup] *****************************************************
ok: [localhost]


TASK [Check if user root exists] *********************************
ok: [localhost]


TASK [Check if user this_user_does_not_exists exists] ************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "User this_user_does_not_exists does not exists"}


NO MORE HOSTS LEFT ***********************************************
    to retry, use: --limit @playbooks/check_user.retry


PLAY RECAP *******************************************************
localhost         : ok=2    changed=0    unreachable=0    failed=1

As expected, since we have the root user, but not the this_user_does_not_exists, it passed the first check, but failed at the second.

Ansible also provides a Python library to parse user arguments and handle errors and returns. It's time to see how the Ansible Python library is useful to make your code shorter, faster, and less error prone. To do so, let's create a file called library/check_user_py2.py with the following content:

    #!/usr/bin/env python 
 
    import pwd 
    from ansible.module_utils.basic import AnsibleModule 
 
    def main(): 
        # Parsing argument file 
        module = AnsibleModule( 
            argument_spec = dict( 
                user = dict(required=True) 
            ) 
        ) 
        user = module.params.get('user') 
 
        # Check if user exists 
        try: 
            pwd.getpwnam(user) 
            success = True 
            ret_msg = 'User %s exists' % user 
        except KeyError: 
            success = False 
            ret_msg = 'User %s does not exists' % user 
 
        # Error handling and JSON return 
        if success: 
            module.exit_json(msg=ret_msg) 
        else: 
            module.fail_json(msg=ret_msg) 
 
    if __name__ == "__main__": 
        main() 

Let's break down the preceding module and see how it works, as follows:

    #!/usr/bin/env python 
 
    import pwd 
    from ansible.module_utils.basic import AnsibleModule 

As you can see, we do not import sys, shlex and json; we are not using them anymore, since all the operations that required them are now done by Ansible module_utils.

    # Parsing argument file 
    module = AnsibleModule( 
        argument_spec = dict( 
            user = dict(required=True) 
        ) 
    ) 
    user = module.params.get('user') 

Previously, we performed a lot of processing on the argument file to get the final user arguments. Ansible makes it easy by providing an AnsibleModule class, which does all the processing on its own and provides us with the final arguments. The required=True parameter means that the argument is mandatory and the execution will fail if the argument is not passed. The default value for required is False, which will allow users to skip the argument. You can then access the value of the arguments through the module.params dictionary by calling the get method on module.params. The logic to check users on the remote host will remain the same, but the error handling and return aspect will change as follows:

    # Error handling and JSON return 
    if success: 
        module.exit_json(msg=ret_msg) 
    else: 
        module.fail_json(msg=ret_msg) 

One of the advantages of using the AnsibleModule object, is that you have very nice facility to handle returning values to the playbook. We will go into more depth in the next section.

To verify that everything works as expected, we can create a new playbook in playbooks/check_user_py2.yaml with the following content:

    - hosts: localhost 
      vars: 
        user_ok: root 
        user_ko: this_user_does_not_exists 
      tasks: 
      - name: 'Check if user {{ user_ok }} exists' 
        check_user_py2: 
          user: '{{ user_ok }}' 
      - name: 'Check if user {{ user_ko }} exists' 
        check_user_py2: 
          user: '{{ user_ko }}' 

Run it with the following:

ansible-playbook playbooks/check_user.yaml

We should receive the following output:

PLAY [localhost] *************************************************
TASK [setup] *****************************************************
ok: [localhost]


TASK [Check if user root exists] *********************************
ok: [localhost]


TASK [Check if user this_user_does_not_exists exists] ************
fatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "User this_user_does_not_exists does not exists"}


NO MORE HOSTS LEFT ***********************************************
    to retry, use: --limit @playbooks/check_user_py2.retry


PLAY RECAP *******************************************************
localhost         : ok=2    changed=0    unreachable=0    failed=1

Which is consistent with our expectations.

    #!/usr/bin/env python 
 
    import pwd 
    from ansible.module_utils.basic import AnsibleModule 
     
    class CheckUser: 
        def __init__(self, user): 
            self.user = user 
     
        # Check if user exists 
        def check_user(self): 
            uid = '' 
            gid = '' 
            try: 
                user = pwd.getpwnam(self.user) 
                success = True 
                ret_msg = 'User %s exists' % self.user 
                uid = user.pw_uid 
                gid = user.pw_gid 
            except KeyError: 
                success = False 
                ret_msg = 'User %s does not exists' % self.user 
            return success, ret_msg, uid, gid 
 
    def main(): 
        # Parsing argument file 
        module = AnsibleModule( 
            argument_spec = dict( 
                user = dict(required=True) 
            ) 
        ) 
        user = module.params.get('user') 
     
        chkusr = CheckUser(user) 
        success, ret_msg, uid, gid = chkusr.check_user() 
 
        # Error handling and JSON return 
        if success: 
            module.exit_json(msg=ret_msg, uid=uid, gid=gid) 
        else: 
            module.fail_json(msg=ret_msg) 
 
    if __name__ == "__main__": 
        main() 

    - hosts: localhost 
      vars: 
        user: root 
      tasks: 
      - name: 'Retrive {{ user }} data if it exists' 
        check_user_id: 
          user: '{{ user }}' 
        register: user_data 
      - name: 'Print user {{ user }} data' 
        debug: 
          msg: '{{ user_data }}' 

ansible-playbook playbooks/check_user.yaml

PLAY [localhost] *************************************************
TASK [setup] *****************************************************
ok: [localhost]


TASK [Retrieve fale data if it exists] ****************************
ok: [localhost]


TASK [Print user fale data] **************************************
ok: [localhost] => {
    "msg": {
        "changed": false,
        "gid": 1000,
        "msg": "User root exists",
        "uid": 1000
    }
}


PLAY RECAP *******************************************************
localhost         : ok=3    changed=0    unreachable=0    failed=0

git clone git://github.com/ansible/ansible.git --recursive

source ansible/hacking/env-setup

ansible/hacking/test-module -m library/check_user_id.py -a "user=root"

    * including generated source, if any, saving to: /home/fale/.ansible_module_generated 
    * ansiballz module detected; extracted module source to: /home/fale/debug_dir 
    *********************************** 
    RAW OUTPUT 
 
    {"msg": "User root exists", "invocation": {"module_args": {"user": "root"}}, "gid":     0, "uid": 0, "changed": false} 
 
 
    *********************************** 
    PARSED OUTPUT 
    { 
        "changed": false, 
        "gid": 0, 
        "invocation": { 
            "module_args": { 
                "user": "root" 
            } 
        }, 
        "msg": "User root exists", 
        "uid": 0 
    }

Bash modules in Ansible are no different than any other bash scripts, except the way it prints the data on stdout. Bash modules could be as simple as checking if a process is running on the remote host to running some complex commands.

Let's create the file library/kill_java.sh with the following content:

    #!/bin/bash 
    source $1 
 
    SERVICE=$service_name 
 
    JAVA_PIDS=$(/usr/java/default/bin/jps | grep ${SERVICE} | awk '{print $1}') 
 
    if [ ${JAVA_PIDS} ]; then 
        for JAVA_PID in ${JAVA_PIDS}; do 
            /usr/bin/kill -9 ${JAVA_PID} 
        done 
        echo "failed=False msg="Killed all the orphaned processes for ${SERVICE}"" 
        exit 0 
    else 
        echo "failed=False msg="No orphaned processes to kill for ${SERVICE}"" 
        exit 0 
    fi 

The preceding bash module will take the service_name argument and forcefully kill all of the Java processes that belong to that service. As you know, Ansible passes the argument file to the module. We then source the arguments file using source $1. This will actually set the environment variable with the name, service_name. We then access this variable using $service_name as follows:

    source $1 
 
    SERVICE=$service_name 

We then check to see if we obtained any PIDs for the service and run a loop over it to forcefully kill all of the Java processes that match service_name. Once they're killed, we exit the module with failed=False and a message with an exit code of 0, as you can see here:

    if [ ${JAVA_PIDS} ]; then 
        for JAVA_PID in ${JAVA_PIDS}; do 
            /usr/bin/kill -9 ${JAVA_PID} 
        done 
        echo "failed=False msg="Killed all the orphaned processes for ${SERVICE}"" 
        exit 0 

If we do not find any running process for the service, we will still exit the module with an exit code of 0 because terminating the Ansible run might not make sense; this is in the following part:

    else 
        echo "failed=False msg="No orphaned processes to kill for ${SERVICE}"" 
        exit 0 
    fi 

Ansible allows you to return a key-value output if the language itself doesn't support JSON. This makes Ansible more developer/sysadmin-friendly and allows custom modules to be written in any language of one's choice. Let's test the bash module by passing the arguments file to the module. We can now create an arguments file in /tmp/arguments that has the service_name parameter set to Jenkins, as follows:

    service_name=jenkins 

Now, you can run the module like any other bash script. Let's see what happens when we run it with:

bash library/kill_java.sh /tmp/arguments

We should receive the following output:

failed=False msg="No orphaned processes to kill for jenkins"

As expected, the module did not fail even though there was no Jenkins process running on the localhost.

Writing modules in Ruby is as easy as writing a module in Python or bash. You just need to take care of the arguments, errors, return statements, and of course, know basic Ruby! Let's create the library/rsync.rb file with the following code:

    #!/usr/bin/env ruby 
 
    require 'rsync' 
    require 'json' 
 
    src = '' 
    dest = '' 
    ret_msg = '' 
    SUCCESS = '' 
 
    def print_message(state, mdg, key='Failed') 
        message = { 
            key => state, 
            "msg" => msg 
        } 
        print message.to_json 
        exit 1 if state == false 
        exit 0 
    end 
 
    args_file = ARGV[0] 
    data = File.read(args_file) 
    arguments = data.split(" ") 
    arguments.each do |argument| 
        print_message(false, "Argument should be name-value pairs. Example name=foo") if not argument.include("=") 
        field.value = argument.split("=") 
        if field == "src" 
            src = value 
        elseif field == "dest" 
            dest = value 
        else print_message(false, "Invalid argument provided. Valid arguments are src and dest.") 
        end 
    end 
 
    result - Rsync.run("#{src}", "#{dest}") 
    if result.success? 
        success = true 
        ret_msg = "Copied file successfully" 
    else 
        success = false 
        ret_msg = result.error 
    end 
 
    if success 
        print_message(false, "#{ret_msg}") 
    else 
        print_message(true, "#{ret_msg}") 
    end 

In the preceding module, we first process the user arguments, then copy the file using the rsync library, and finally, return the output. Let's break down the preceding code and see how it works.

We first wrote a method, print_message, which will print the output in a JSON format. By doing this, we can reuse the same code in multiple places. Remember, the output of your module should contain failed=true if you want the Ansible run to fail; otherwise, Ansible will think that the module succeeded and will continue with the next task. The output obtained is as follows:

    #!/usr/bin/env ruby 
 
    require 'rsync' 
    require 'json' 
     
    src = '' 
    dest = '' 
    ret_msg = '' 
    SUCCESS = '' 
 
    def print_message(state, mdg, key='Failed') 
        message = { 
            key => state, 
            "msg" => msg 
        } 
        print message.to_json 
        exit 1 if state == false 
        exit 0 
    end 

We then process the arguments file, which contains a key-value pair separated by a whitespace character. This is similar to what we did with the Python module earlier, where we took care of parsing out the arguments. We also perform some checks to make sure that the user has not missed any required argument. In this case, we check if the src and dest parameters have been specified and print a message if the arguments are not provided. Further checks could include the format and type of arguments. You can add these checks and any other checks you deem important. For example, if one of your parameters is a date, then you'd like to verify that the input is actually the right date. Consider the following piece of code, which shows the discussed parameters:

    args_file = ARGV[0] 
    data = File.read(args_file) 
    arguments = data.split(" ") 
    arguments.each do |argument| 
        print_message(false, "Argument should be name-value pairs. Example name=foo") if not argument.include("=") 
        field.value = argument.split("=") 
        if field == "src" 
            src = value 
        elseif field == "dest" 
            dest = value 
        else print_message(false, "Invalid argument provided. Valid arguments are src and dest.") 
        end 
    end 

Once we have the required arguments, we will go ahead and copy the file using the rsync library as follows:

    result - Rsync.run("#{src}", "#{dest}") 
    if result.success? 
        success = true 
        ret_msg = "Copied file successfully" 
    else 
        success = false 
        ret_msg = result.error 
    end 

Finally, we check if the rsync task passed or failed and call the print_message function to print the output on stdout as follows:

    if success 
        print_message(false, "#{ret_msg}") 
    else 
        print_message(true, "#{ret_msg}") 
    end 

You can test your Ruby module by simply passing the arguments file to the module. To do so, we can create the file /tmp/arguments with the following content:

    src=/var/log/ansible.log dest=/tmp/ansible_backup.log 

Let's now run the module, as shown:

ruby library/rsync.rb /tmp/arguments

We will receive the following output:

    {"failed":false,"msg":"Copied file successfully"} 

We will leave the serverspec testing for you to complete.

Testing is often undervalued due to lack of understanding of its purpose and the benefits it can bring to the business. Testing modules is as important as testing any other part of the Ansible playbook because a small change in a module can break your entire playbook. We will take an example of the Python module that we wrote in the first section of this chapter and write an integration test using Python's nose test framework. Unit tests are also encouraged, but for our scenario where we check if a user exists remotely, an integration test makes more sense.

To test the module, we convert our previous module into a Python class so that we can directly import the class in our test, and run only the main logic of the module. The following code shows the library/check_user_py3.py restructured module, which will check whether a user exists on a remote host:

    #!/usr/bin/env python 
 
    import pwd 
    from ansible.module_utils.basic import AnsibleModule 
 
    class User: 
        def __init__(self, user): 
            self.user = user 
 
        # Check if user exists 
        def check_if_user_exists(self): 
            try: 
                user = pwd.getpwnam(self.user) 
                success = True 
                ret_msg = 'User %s exists' % self.user 
            except KeyError: 
                success = False 
                ret_msg = 'User %s does not exists' % self.user 
            return success, ret_msg 
 
    def main(): 
        # Parsing argument file 
        module = AnsibleModule( 
            argument_spec = dict( 
                user = dict(required=True) 
            ) 
        ) 
        user = module.params.get('user') 
 
        chkusr = User(user) 
        success, ret_msg = chkusr.check_if_user_exists() 
 
        # Error handling and JSON return 
        if success: 
            module.exit_json(msg=ret_msg, uid=uid, gid=gid) 
        else: 
            module.fail_json(msg=ret_msg) 
 
    if __name__ == "__main__": 
        main() 

As you can see in the preceding code, we created a class named User. We instantiated the class, and called the check_if_user_exists method to check if the user actually exists on the remote machine. It's time to write an integration test now. We assume that you have the nose package installed on your system. If not, don't worry! You can still install the package by using the following command:

pip install nose

Let's now write the integration test file in library/test_check_user_py3.py as follows:

    from nose.tools import assert_equals, assert_false, assert_true 
    import imp 
    imp.load_source("check_user","check_user_py3.py") 
    from check_user import User 
 
    def test_check_user_positive(): 
        chkusr = User("root") 
        success, ret_msg = chkusr.check_if_user_exists() 
        assert_true(success) 
        assert_equals('User root exists', ret_msg) 
 
    def test_check_user_negative(): 
        chkusr = User("this_user_does_not_exists") 
        success, ret_msg = chkusr.check_if_user_exists() 
        assert_false(success) 
        assert_equals('User this_user_does_not_exists does not exists', ret_msg) 

In the preceding integration test, we import the nose package and our module, check_user. We call the User class by passing the user we want to check. We then check whether the user exists on the remote host by calling the check_if_user_exists() method. The nose methods, assert_true, assert_false, and assert_equals can be used to compare the expected value against the actual. Only if the assert methods pass, will the test pass. You can have multiple tests inside the same file by having multiple methods whose names start with test_, for example, the test_check_user_positive() and test_check_user_negative() methods. Nose tests will take all the methods that start with test_ and execute them.

We can now test if it works running nose in the following way:

cd library
nosetests -v test_check_users_py3.py

You should receive output similar to this:

test_check_user_py3.test_check_user_positive ... ok
test_check_user_py3.test_check_user_negative ... ok
---------------------------------------------------
Ran 2 tests in 0.001s
OK

As you can see, the test passed because the root user existed on the host while the this_user_does_not_exists user does not exist.

For more complicated modules, we recommend that you write unit tests and integration tests. You might wonder why we didn't use serverspec to test the module.

We still recommend running serverspec tests for functional testing as part of playbooks, but for unit and integration tests, it's recommended to use well-known frameworks. Similarly, if you write Ruby modules, we recommend you write tests for them with a framework such as rspec. If your custom Ansible module has multiple parameters with multiple combinations, then you will write more tests to test each scenario. Finally, we recommend that your run all these tests as part of your CI system, be it Jenkins, Travis, or any other system.

Questions

A couple of questions to think about are given in the this section:

With this, we come to the end of this rather small but important chapter, which focused on how you can extend Ansible by writing your own custom modules. You learned how to use Python, Bash, and Ruby in order to write your modules. We've also seen how to write integration tests for modules so that they can be integrated into your CI system. In future, hopefully, extending your Ansible functionality using modules should be way easier!

Next, we will step into the world of provisioning, deployment, and orchestration and look at how Ansible solves our infrastructure problems when we provision new instances or want to deploy software updates to various instances in our environments. We promise that the journey is going to be fun!

The check mode (also known as the dry run or no-op mode) will run your playbook in a no-operation mode, that is, it will not apply any changes to the remote host; instead, it will just show the changes that will be introduced when a task is run. Whether the check mode is actually enabled or not depends on each task. There are few commands that you may find interesting. All those modules will have to be run in /usr/lib/python2.7/site-packages/ansible/modules or where your Ansible module folder is (different paths could be possible based on the operating system you are using as well as the way you installed Ansible).

To count the number of available modules on your installation, you can perform this command:

find . -type f | grep '.py$' | grep -v '__init__' | wc -l

With Ansible 2.1.1, the result of this command is 569, since Ansible has that many modules.

If you want to see how many of these support the check mode, you can run:

grep -r 'supports_check_mode=True' | awk -F: '{print $1}' | sort | uniq | wc -l

With Ansible 2.1.1 the result of this command is 242.

You might also find the following command useful for listing all modules that support the check mode:

grep -r 'supports_check_mode=True' | awk -F: '{print $1}' | sort | uniq

This helps you test how your playbook will behave and check if there may be any failures before running it on your production server. You run a playbook in the check mode by simply passing the --check option to your ansible-playbook command. Let's see how the check mode works with the setup_apache.yml playbook, as follows:

    PLAY [localhost] *************************************************

    TASK [setup] *****************************************************
    ok: [localhost]

    TASK [Install Apache] ********************************************
    ok: [localhost]

    TASK [Enable Apache] *********************************************
    changed: [localhost]

    PLAY RECAP *******************************************************
    localhost         : ok=3    changed=1    unreachable=0    failed=0

In the preceding run, instead of making the changes on the target host, Ansible highlighted all the changes that would have occurred during the actual run. From the preceding run, you can find that httpd service was already installed on the target host, because of which, Ansible's exit message for that task was ok.

    TASK [Install Apache] ********************************************
    ok: [localhost]

Whereas, with the second task, it found that httpd service was not running on the target host:

    TASK [Enable Apache] *********************************************
    changed: [localhost]

When you run the preceding playbook again without the check mode enabled, Ansible will make sure that the service state is running.

In the check mode, you can use the --diff option to show the changes that would be applied to a file. To be able to see the --diff option in use, we need to change our playbooks/setup_apache.yaml playbook to match the following:

    - hosts: localhost
      tasks:
      - name: Ensure Apache is installed
        yum:
          name: httpd
          state: present
      - name: Ensure Apache in enabled
        service:
          name: httpd
          state: running
          enabled: True
      - name: Ensure Apache userdirs are properly configured
        template:
          src: '../templates/userdir.conf'
          dest: '/etc/httpd/conf.d/userdir.conf'

As you can see, we added a task, which will ensure a certain state of the /etc/httpd/conf.d/userdir.conf file.

We also need to create a template file placed in templates/userdir.conf with the following content:

    # UserDir: The name of the directory that is appended onto a user's home
    # directory if a ~user request is received.
    # The path to the end user account 'public_html' directory must be
    # accessible to the webserver userid.  This usually means that ~userid
    # must have permissions of 711, ~userid/public_html must have permissions
    # of 755, and documents contained therein must be world-readable.
    # Otherwise, the client will only receive a "403 Forbidden" message.
    #
    <IfModule mod_userdir.c>
        #
        # UserDir is disabled by default since it can confirm the presence
        # of a username on the system (depending on home directory
        # permissions).
        #
        UserDir enabled
    
        #
        # To enable requests to /~user/ to serve the user's public_html
        # directory, remove the "UserDir disabled" line above, and uncomment
        # the following line instead:
        #
        #UserDir public_html
    </IfModule>
    
    #
    # Control access to UserDir directories.  The following is an example
    # for a site where these directories are restricted to read-only.
    #
    <Directory "/home/*/public_html">
        AllowOverride FileInfo AuthConfig Limit Indexes
        Options MultiViews Indexes SymLinksIfOwnerMatch IncludesNoExec
        Require method GET POST OPTIONS
    </Directory>

In this template, we only changed the UserDir enabled line, which by default is UserDir disabled.

We can now test the result of this with the following command:

ansible-playbook playbooks/setup_apache.yaml --diff --check

As you can see, we are using the --check parameter that will ensure this will be a dry-run. We will receive the following output:

    PLAY [localhost] *************************************************

    TASK [setup] *****************************************************
    ok: [localhost]

    TASK [Ensure Apache is installed] ********************************
    ok: [localhost]

    TASK [Ensure Apache in enabled] **********************************
    changed: [localhost]

    TASK [Ensure Apache userdirs are properly configured] ************
    changed: [localhost]
    --- before: /etc/httpd/conf.d/userdir.conf
    +++ after: dynamically generated
    @@ -14,7 +14,7 @@
        # of a username on the system (depending on home directory
        # permissions).
        #
    -    UserDir disabled
    +    UserDir enabled
    
        #
        # To enable requests to /~user/ to serve the user's public_html
    @@ -33,4 +33,3 @@
        Options MultiViews Indexes SymLinksIfOwnerMatch IncludesNoExec
        Require method GET POST OPTIONS
    </Directory>
    -
    
    
    PLAY RECAP *******************************************************
    localhost         : ok=4    changed=2    unreachable=0    failed=0

As we can see, Ansible compares the current file of the remote host with the source file; a line starting with + indicates that a line was added to the file, whereas - indicates that a line was removed.

Using --diff and --check modes together is a test step that can potentially be used as part of your CI tests to assert how many steps have changed as part of the run. Another case where you can use those features together is the part of the deployment process that checks what exactly will change when you run Ansible on that machine.

There are also cases - that should not happen, but sometimes happen-where you have not run a playbook on a machine for a very long time and you are worried that running it again will break something. Using those options together should help you understand if it was just you worrying or if this is a real risk.

Wikipedia says functional testing is a Quality Assurance (QA) process and a type of black-box testing that bases its test cases on the specifications of the software component under the test. Functions are tested by feeding them input and examining the output; the internal program structure is rarely considered. Functional testing is as important as code when it comes to infrastructure.

From an infrastructure perspective, with respect to functional testing, we test output of our Ansible runs on the actual machines. Ansible provides multiple ways to perform the functional testing of your playbook; let's look at some of the most commonly used methods.

The check mode will only work when you want to check whether a task will change anything on the host or not. This will not help when you want to check whether the output of your module is what you expected. For example, let's say you wrote a module that will check if a port is up or not. In order to test this, you might need to check the output of your module and see whether it matches the desired output or not. To perform such tests, Ansible provides a way to directly compare the output of a module with the desired output.

Let's see how this works creating the file playbooks/assert_ls.yaml with the following content:

    - hosts: localhost
      tasks:
      - name: List files in /tmp
        command: ls /tmp
        register: list_files
      - name: Check if file testfile.txt exists
        assert:
          that:
          - "'testfile.txt' in list_files.stdout_lines"

In the preceding playbook, we're running the ls command on the target host and registering the output of that command in the list_files variable. Further, we ask Ansible to check whether the output of the ls command has the expected result. We do this using the assert module, which uses some conditional checks to verify if the stdout value of a task meets the expected output of the user. Let's run the preceding playbook to see what output Ansible returns with the command:

ansible-playbook playbooks/assert_ls.yaml

Since we don't have the file, we will receive the following output:

    PLAY [localhost] *************************************************

    TASK [setup] *****************************************************
    ok: [localhost]

    TASK [List files in /tmp] ****************************************
    changed: [localhost]

    TASK [Check if file testfile.txt exists] *************************
    fatal: [localhost]: FAILED! => {"assertion": "'testfile.txt' in list_files.stdout_lines", "changed":     false, "evaluated_to": false, "failed": true}
    
    NO MORE HOSTS LEFT ***********************************************
        to retry, use: --limit @playbooks/assert_ls.retry

    PLAY RECAP *******************************************************
    localhost         : ok=2    changed=1    unreachable=0    failed=1

If we re-run the playbook after we create the expected file, it will not fail and therefore this will be the result:

    PLAY [localhost] *************************************************

    TASK [setup] *****************************************************
    ok: [localhost]

    TASK [List files in /tmp] ****************************************
    changed: [localhost]

    TASK [Check if file testfile.txt exists] *************************
    ok: [localhost]

    PLAY RECAP *******************************************************
    localhost         : ok=3    changed=1    unreachable=0    failed=0

This time, the task passed with an ok message as testfile.txt was present in the list_files variable. Likewise, you can match multiple strings in a variable or multiple variables using the and and or operators. The assertion feature is quite powerful, and users who have written either unit or integration tests in their projects will be quite happy to see this feature!

Tags are a great way to test a bunch of tasks without running an entire playbook. We can use tags to run actual tests on the nodes to verify the state that the user intended to be in, the playbook. We can treat this as another way to run integration tests for Ansible on the actual box. The tag method to test can be run on the actual machines where you run Ansible, and also, it can be used primarily during deployments to test the state of your end systems. In this section, we'll first look at how to use tags in general, their features that can possibly help us, not just with testing but even otherwise, and finally for testing purposes.

To add tags in your playbook, use the tags parameter followed by one or more tag names separated by commas. Let's create a simple playbook in playbooks/tags_example.yaml to see how the tags work with the following content:

    - hosts: localhost
      tasks:
      - name: Ensure the file /tmp/ok exists
        file:
          name: /tmp/ok
          state: touch
        tags:
        - file_present
      - name: Ensure the file /tmp/ok does not exists
        file:
          name: /tmp/ok
          state: absent
        tags:
        - file_absent

If we now run the playbook, the file will be created and destroyed. We can see it running with the following command:

ansible-playbook playbooks/tags_example.yaml

It will give us this output:

    PLAY [localhost] *************************************************

    TASK [setup] *****************************************************
    ok: [localhost]
    
    TASK [Ensure the file /tmp/ok exists] ****************************
    changed: [localhost]
    
    TASK [Ensure the file /tmp/ok does not exists] *******************
    changed: [localhost]
    
    PLAY RECAP *******************************************************
    localhost         : ok=3    changed=2    unreachable=0    failed=0

Since this is not an idempotent playbook, if we run it over and over, we will always see the same result, as the playbook will create and delete the file every time.

You can now simply pass the file_present tag or the file_absent tag to only perform one of the actions, like in the following example:

ansible-playbook playbooks/tags_example.yaml -t file_present

Thanks to the -t file_present part, only the tasks with the file_present tag will be executed, in fact this will be the output:

    PLAY [localhost] *************************************************

    TASK [setup] *****************************************************
    ok: [localhost]

    TASK [Ensure the file /tmp/ok exists] ****************************
    changed: [localhost]

    PLAY RECAP *******************************************************
    localhost         : ok=2    changed=1    unreachable=0    failed=0

You can also use tags to perform a set of tasks on the remote host just like taking a server out of a load balancer and adding it back to the load balancer.

You can also use the --check option with tags. By doing this, you can test your tasks without actually running them on your hosts. This allows you to test a bunch of individual tasks directly, instead of copying your tasks to a temporary playbook and running it from there.

Ansible also provides a way to skip some tags in a playbook. If you have a long playbook with multiple tags, like 10, and you want to execute them all but one, then it would not be a good idea to pass nine tags to Ansible. The situation would be more difficult if you forgot to pass a tag and the ansible-playbook command fails. To overcome such situations, Ansible provides a way to skip a couple of tags, instead of passing multiple tags. It's functioning is pretty straightforward, and can be triggered in the following way:

ansible-playbook playbooks/tags_example.yaml --skip-tags file_present

The output will be something like:

    PLAY [localhost] *************************************************

    TASK [setup] *****************************************************
    ok: [localhost]

    TASK [Ensure the file /tmp/ok does not exists] *******************
    ok: [localhost]

    PLAY RECAP *******************************************************
    localhost         : ok=2    changed=1    unreachable=0    failed=0

As you can see, all tasks have been executed except the one with the file_present tag.

There are many cases, where for one reason or another, you want your playbook and roles to carry on in the case one or more tasks fail. A typical example of this could be that you want to check if software is installed or not. Let's see the following example to install Java. In the roles/java/tasks/main.ymal file, we are going to put the following code:

    - name: Verify if the current version of Java is installed
      command: rpm -q jdk1.8.0_91-1.8.0_91-fcs
      register: java
      ignore_errors: True
      changed_when: java|failed

    - name: Ensure that JavaSE is download
      uri:
        url: 'http://download.oracle.com/otn-pub/java/jdk/8u91-b14/jdk-8u91-linux-x64.rpm'
        method: GET
        HEADER_Cookie: 'gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie'
        dest: /tmp
        creates: /tmp/jdk-8u91-linux-x64.rpm
      when: java|failed

    - name: Ensure JavaSE is installed
      dnf:
        name: /tmp/jdk-8u91-linux-x64.rpm
        state: present
      when: java|failed

    - name: Set alternatives for java
      alternatives:
        path: /usr/java/jdk1.8.0_91/jre/bin/java
        name: java
        link: /usr/bin/java
      when: java|failed
    
    - name: Set alternatives for javac
      alternatives:
        path: /usr/java/jdk1.8.0_91/bin/javac
        name: javac
        link: /usr/bin/javac
      when: java|failed

    - name: Set alternatives for javaws
      alternatives:
        path: /usr/java/jdk1.8.0_91/bin/javaws
        name: javaws
        link: /usr/bin/javaws
      when: java|failed

Before going forward with the other parts that are needed to execute this role, I'd like to spend some words on the various parts of this role task list, since there are many new things:

    - name: Verify if the current version of Java is installed
      command: rpm -q jdk1.8.0_91-1.8.0_91-fcs
      register: java
      ignore_errors: True
      changed_when: java|failed

In this task, we execute an rpm command that could have two different outputs:

Since we only want to check if the package exists or not and then to go forward, we register the output (third line) and ignore eventual failures (fourth line):

    - name: Ensure that JavaSE is download
      uri:
        url: 'http://download.oracle.com/otn-pub/java/jdk/8u91-b14/jdk-8u91-linux-x64.rpm'
        method: GET
        HEADER_Cookie: 'gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie'
        dest: /tmp
        creates: /tmp/jdk-8u91-linux-x64.rpm
      when: java|failed

In this part, we use the uri module that allows us to hit a remote URI with an HTTP request. This module is very nice since it allows you to use all HTTP methods as well as to customize HTTP headers. This makes this module very flexible. Since in the last line we have when: java|failed, this will only be executed if Java is not installed:

    - name: Ensure JavaSE is installed
      dnf:
        name: /tmp/jdk-8u91-linux-x64.rpm
        state: present
      when: java|failed

Here we use dnf to install the Java package. Since in the last line we have when: java|failed, this will only be executed if Java is not installed:

    - name: Set alternatives for java
      alternatives:
        path: /usr/java/jdk1.8.0_91/jre/bin/java
        name: java
        link: /usr/bin/java
      when: java|failed

    - name: Set alternatives for javac
      alternatives:
        path: /usr/java/jdk1.8.0_91/bin/javac
        name: javac
        link: /usr/bin/javac
      when: java|failed

    - name: Set alternatives for javaws
      alternatives:
        path: /usr/java/jdk1.8.0_91/bin/javaws
        name: javaws
        link: /usr/bin/javaws
      when: java|failed

Here we are going to set new alternatives, in case we are installing Java. alternatives is an Ansible module that allows us to manage the configuration of the Linux alternatives program. This program is often used to manage which version of a program should be run by default in case you have multiple versions installed by default.

After we create the role, we will need the hosts file containing the host machine, in my case:

    j01.fale.io

And a playbook to apply the role, placed in playbooks/hosts/j01.fale.io.yaml and with the following content:

    - hosts: j01.fale.io
      user: root
      roles:
      - java

We can now execute it with the following:

ansible-playbook playbooks/hosts/j01.fale.io.yaml

We will get the following result:

    PLAY [j01.fale.io] ***********************************************

    TASK [setup] *****************************************************
    ok: [j01.fale.io]

    TASK [java : Verify if the current version of Java is installed] *
    fatal: [j01.fale.io]: FAILED! => {"changed": true, "cmd": ["rpm", "-q", "jdk1.8.0_91-1.8.0_91-fcs"],         "delta": "0:00:00.009788", "end": "2016-09-27 11:04:56.185618", "failed": true, "rc": 1, "start":         "2016-    09-27 11:04:56.175830", "stderr": ``, "stdout": "package jdk1.8.0_91-1.8.0_91-fcs is not         installed", "stdout_lines": ["package jdk1.8.0_91-1.8.0_91-fcs is not installed"], "warnings":             ["Consider using yum, dnf or zypper module rather than running rpm"]}
      ...ignoring

    TASK [java : Ensure that JavaSE is download] *********************
    changed: [j01.fale.io]

    TASK [java : Ensure JavaSE is installed] *************************
    changed: [j01.fale.io]

    TASK [java : Set alternatives for java] **************************
    ok: [j01.fale.io]

    TASK [java : Set alternatives for javac] *************************
    ok: [j01.fale.io]

    TASK [java : Set alternatives for javaws] ************************
    ok: [j01.fale.io]

    PLAY RECAP *******************************************************
    j01.fale.io       : ok=7    changed=2    unreachable=0    failed=0

As you can see, the installation check failed since Java was not installed on the machine, and for this reason all other tasks have been executed as expected.

There are cases when you want to trigger a failure directly. This can happen for multiple reasons, even if there are disadvantages doing so, since when you trigger the failure, the playbook will be brutally interrupted and this could leave your machine in an inconsistent state if you are not careful. One case where I have seen it work very well, is when you are running a non-idempotent playbook (for instance building of a newer version of an application) and you need a variable (for instance: the version/branch to deploy) set. In this case, you can check that the expected variable is correctly configured before starting to run the operations to ensure that everything will work as expected later on.

Let's put the following code in playbooks/maven_build.yaml:

    - hosts: j01.fale.io
      tasks:
      - name: Ensure the tag variable is properly set
        fail: 'The version needs to be defined. To do so, please add: --extra-vars                                 "version=$[TAG/BRANCH]"'
        when: version is not defined
      - name: Get last Project version
        git:
          repo: https://github.com/org/project.git
          dest: "/tmp"
          version: '{{ version }}'
      - name: Maven clean install
        shell: "cd /tmp/project && mvn clean install"

As you can see, we expect the user to add --extra-vars "version=$[TAG/BRANCH]" in the script to call the command. We could have put a branch to use by default but this is too risky because the user may lose focus and forget to add the right branch name themselves, which would lead to compiling (and deploying) the wrong version of the application. The fail module also allows us to specify a message that will be displayed to the user.

In this chapter, we have seen how to debug Ansible playbooks using multiple techniques. Then we moved to the management of failures and lastly we saw how to trigger failures intentionally.

In the next chapter, we will discuss multi-tier environments as well as deployment methodologies.

Let's assume you have four environments to take care of, which are as follows:

In the Git branch-based method, you will have one environment per branch. You will always make changes to Development first, and then promote those changes to Testing (merge or cherry-pick, and tag commits in Git), Stage, and Production. In this approach, you will hold one single inventory file, one set of variable files, and finally, a bunch of folders dedicated to roles and playbooks per branch.

In this approach, you will always maintain the dev and master branches. The initial code is committed to the dev branch, and once stable, you will promote it to the master branch. The same roles and playbooks that exist in the master branch will run across all environments. On the other hand, you will have separate folders for each of your environments. Let's look at an example. We'll show how you can have a separate configuration and an inventory for two environments: stage and production. You can extend it for your scenario to fit all the environments you use. Let's first look at the playbook in playbooks/variables.yaml that will run across these multiple environments and has the following content:

- hosts: web
  user: root
  tasks:
  - name: Print environment name
    debug:
      var: env
  - name: Print db server url
    debug:
      var: db_url
  - name: Print domain url
    debug:
      var: domain
- hosts: db
  user: root
  tasks:
  - name: Print environment name
    debug:
      var: env
  - name: Print database username
    debug:
      var: db_user
  - name: Print database password
    debug:
      var: db_pass

As you can see, there are two sets of tasks in this plays:

There is also an extra task to print the environment name that is common to all servers in a particular environment. We will also have two different inventory files.

The first one will be called inventory/production with the following content:

[web]
ws01.fale.io
ws02.fale.io

[db]
db01.fale.io

[production:children]
db
web

The second one will be called inventory/staging with the following content:

[web]
ws01.stage.fale.io
ws02.stage.fale.io

[db]
db01.stage.fale.io

[staging:children]
db
web

As you can see, we have two machines for the web section and one for the db in each environment. Further, we have a different set of machines for stage and production environments. The additional section, [ENVIRONMENT:children], allows you to create a group of groups. This would mean that any variables that are defined in the ENVIRONMENT section will apply to both the db and web groups, unless they're overridden in the individual sections, respectively. The next interesting part would be to look at variable values for each of the environments and see how they are separated out in each environment.

Let's start with the variables that will be the same for all our environments, located in inventory/group_vars/all:

db_user: mysqluser

The only variable that is the same for both our environments is the db_user.

We can now look at the production-specific variables, located in inventory/group_vars/production:

env: production
domain: fale.io
db_url: db.fale.io
db_pass: this_is_a_safe_password

If we now look at the stage-specific variables located in inventory/group_vars/staging, we will find the same variables we had in the production one, but with different values:

env: staging
domain: stage.fale.io
db_url: db.stage.fale.io
db_pass: this_is_an_unsafe_password

We can now validate that we receive the expected results. First we are going to run against the staging environment:

ansible-playbook -i staging playbooks/variables.yaml

And we should receive an output similar to the following:

PLAY [web] *******************************************************


TASK [setup] *****************************************************
ok: [ws02.stage.fale.io]
ok: [ws01.stage.fale.io]


TASK [Print environment name] ************************************
ok: [ws01.stage.fale.io] => {
    "env": "staging"
}
ok: [ws02.stage.fale.io] => {
    "env": "staging"
}


TASK [Print db server url] ***************************************
ok: [ws01.stage.fale.io] => {
    "db_url": "db.stage.fale.io"
}
ok: [ws02.stage.fale.io] => {
    "db_url": "db.stage.fale.io"
}


TASK [Print domain url] ******************************************
ok: [ws01.stage.fale.io] => {
    "domain": "stage.fale.io"
}
ok: [ws02.stage.fale.io] => {
    "domain": "stage.fale.io"
}


PLAY [db] ********************************************************


TASK [setup] *****************************************************
ok: [db01.stage.fale.io]


TASK [Print environment name] ************************************
ok: [db01.stage.fale.io] => {
    "env": "staging"
}


TASK [Print database username] ***********************************
ok: [db01.stage.fale.io] => {
    "db_user": "mysqluser"
}


TASK [Print database password] **********************************
ok: [db01.stage.fale.io] => {
    "db_pass": "this_is_an_unsafe_password"
}


PLAY RECAP *******************************************************
db01.stage.fale.io: ok=4    changed=0    unreachable=0    failed=0
ws01.stage.fale.io: ok=4    changed=0    unreachable=0    failed=0
ws02.stage.fale.io: ok=4    changed=0    unreachable=0    failed=0

We can now run against the production environment:

ansible-playbook -i production playbooks/variables.yaml

We will receive the following result:

PLAY [web] *******************************************************


TASK [setup] *****************************************************
ok: [ws02.fale.io]
ok: [ws01.fale.io]


TASK [Print environment name] ************************************
ok: [ws01.fale.io] => {
    "env": "production"
}
ok: [ws02.fale.io] => {
    "env": "production"
}


TASK [Print db server url] ***************************************
ok: [ws01.fale.io] => {
    "db_url": "db.fale.io"
}
ok: [ws02.fale.io] => {
    "db_url": "db.fale.io"
}


TASK [Print domain url] ******************************************
ok: [ws01.fale.io] => {
    "domain": "fale.io"
}
ok: [ws02.fale.io] => {
    "domain": "fale.io"
}


PLAY [db] ********************************************************


TASK [setup] *****************************************************
ok: [db01.fale.io]


TASK [Print environment name] ************************************
ok: [db01.fale.io] => {
    "env": "production"
}


TASK [Print database username] ***********************************
ok: [db01.fale.io] => {
    "db_user": "mysqluser"
}


TASK [Parint database password] **********************************
ok: [db01.fale.io] => {
    "db_pass": "this_is_a_safe_password"
}


PLAY RECAP *******************************************************
db01.fale.io      : ok=4    changed=0    unreachable=0    failed=0
ws01.fale.io      : ok=4    changed=0    unreachable=0    failed=0
ws02.fale.io      : ok=4    changed=0    unreachable=0    failed=0

You can see that the Ansible run picked up all the relevant variables defined for the staging environment.

If you're using this approach to gain a stable master branch for multiple environments, it's best to use an amalgamation of environment-specific directories, group_vars, and inventory groups to tackle the scenario.

Deploying applications is probably one of the most complex tasks in the Information and Communication Technology (ICT) field. This is mainly caused by the fact that it often requires changing the state of the majority of machines that are somehow part of that application. In fact, often you find yourself having to change the state of load balancers, distribution servers, application servers, and database servers all at the same time during a deployment. New technologies, like containers, are trying to make those operations simpler, but often is not easy or possible to just move a legacy application to a container.

What we are now going to see are the various software distribution strategies and how Ansible can help with each one.

To see how we can deploy the code in the various ways we talked about in the previous pages, we will need an environment, and obviously we are going to create it using Ansible. First of all, to ensure that our roles are properly loaded, we need the ansible.cfg file with the following content:

[defaults]
roles_path = roles

Then we need the playbooks/firstrun.yaml to ensure that we can configure our machines with a basic configuration, with the following content:

- hosts: all
  user: root
  tasks:
  - name: Ensure ansible user exists
    user:
      name: ansible
      state: present
      comment: Ansible
  - name: Ensure ansible user accepts the SSH key
    authorized_key:
      user: ansible
      key: https://github.com/fale.keys
      state: present
  - name: Ensure the ansible user is sudoer with no password required
    lineinfile:
      dest: /etc/sudoers
      state: present
      regexp: '^ansible ALL\='
      line: 'ansible ALL=(ALL) NOPASSWD:ALL'
      validate: 'visudo -cf %s'

The playbooks/groups/web.yaml will also need to be created to allow us to properly bootstrap our web servers:

- hosts: web
  user: ansible
  roles:
  - common
  - webserver

As you can imagine from the previous file content, we will need to create the roles: common and webserver which are very similar to the ones we created in Chapter 4, Handling Complex Deployment. We start with the roles/common/tasks/main.yaml file with the following content:

- name: Ensure EPEL is enabled
   yum:
     name: epel-release
     state: present
   become: True
- name: Ensure needed packages are present
   yum:
     name: '{{ item }}'
     state: present
   become: True
   with_items:
   - libsemanage-python
   - libselinux-python
   - ntp
   - firewalld
- name: Ensure we have last version of every package
   yum:
     name: "*"
     state: latest
   become: True
- name: Ensure the timezone is set to UTC
   file:
     src: /usr/share/zoneinfo/GMT
     dest: /etc/localtime
     state: link
   become: True
- name: Ensure the NTP service is running and enabled
   service:
   name: ntpd
   state: started
   enabled: True
 become: True
- name: Ensure FirewallD is running
   service:
     name: firewalld
     state: started
     enabled: True
   become: True
- name: Ensure SSH can pass the firewall
   firewalld:
     service: ssh
     state: enabled
     permanent: True
     immediate: True
   become: True
- name: Ensure the MOTD file is present and updated
   template:
     src: motd
     dest: /etc/motd
     owner: root
     group: root
     mode: 0644
   become: True
- name: Ensure the hostname is the same of the inventory
   hostname:
     name: "{{ inventory_hostname }}"
   become: True

It's motd template is in roles/common/templates/motd:

                This system is managed by Ansible
  Any change done on this system could be overwritten by Ansible

OS: {{ ansible_distribution }} {{ ansible_distribution_version }}
Hostname: {{ inventory_hostname }}
eth0 address: {{ ansible_eth0.ipv4.address }}

            All connections are monitored and recorded
    Disconnect IMMEDIATELY if you are not an authorized user

We can now move to the webserver role, more specifically to the roles/webserver/tasks/main.yaml file:

- name: Ensure the HTTPd package is installed
  yum:
    name: httpd
    state: present
  become: True
- name: Ensure the PHP is installed
  yum:
    name: '{{ item }}'
    state: present
  become: True
  with_items:
 - git
 - php
- name: Ensure the HTTPd service is enabled and running
  service:
    name: httpd
    state: started
    enabled: True
  become: True
- name: Ensure HTTP can pass the firewall
  firewalld:
    service: http
    state: enabled
    permanent: True
    immediate: True
  become: True
- name: Ensure HTTPd configuration is updated
  copy:
    src: website.conf
    dest: /etc/httpd/conf.d
  become: True
  notify: Restart HTTPd

We also need to create the handler in roles/webserver/handlers/main.yaml with the content:

- name: Restart HTTPd
  service:
    name: httpd
    state: restarted
  become: True

Lastly, we need to touch the roles/webserver/files/website.conf file, leaving it empty for now, but it needs to exist.

We can now provision a couple of CentOS machines (I provisioned ws01.fale.io and ws02.fale.io) and ensure that the inventory is right. We can also run the firstrun.yaml playbook to ensure that the Ansible user is present and properly configured:

ansible-playbook -i inventory/production playbooks/firstrun.yaml

The output you should receive is the following:

PLAY [localhost] *************************************************


PLAY [all] *******************************************************

TASK [setup] *****************************************************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [Ensure ansible user exists] ********************************
changed: [ws01.fale.io]
changed: [ws02.fale.io]


TASK [Ensure ansible user accepts the SSH key] *******************
changed: [ws01.fale.io]
changed: [ws02.fale.io]


TASK [Ensure the ansible user is sudoer with no password required]
changed: [ws01.fale.io]
changed: [ws02.fale.io]


PLAY RECAP *******************************************************
ws01.fale.io      : ok=4    changed=3    unreachable=0    failed=0
ws02.fale.io      : ok=4    changed=3    unreachable=0    failed=0

We can now configure those machines running their group playbook:

ansible-playbook -i inventory/production playbooks/groups/web.yaml

We will receive the following output:

PLAY [web] *******************************************************


TASK [setup] *****************************************************
ok: [ws01.fale.io]
ok: [ws02.fale.io]
    ....
PLAY RECAP *******************************************************
ws01.fale.io      : ok=20   changed=14   unreachable=0    failed=0
ws02.fale.io      : ok=20   changed=14   unreachable=0    failed=0

We can now point our browser to our nodes on port 80 to check that the HTTPd page is displayed as expected.

For this example, we are going to deploy a simple PHP application that will be composed of only a single PHP page. The source is available on the following repository: https://github.com/Fale/demo-php-app.

To deploy it, we will need the following code placed in playbooks/manual/rcs_deploy.yaml:

- hosts: web
  user: ansible
  tasks:
  - name: Install or update website
    git:
      repo: https://github.com/Fale/demo-php-app.git
      dest: /var/www/application
    become: True

We can now run the deployer with the following command:

ansible-playbook -i inventory/production playbooks/manual/rcs_deploy.yaml

This is the expected result:

PLAY [web] *******************************************************


TASK [setup] *****************************************************
ok: [ws01.fale.io]
ok: [ws02.fale.io]
    ....

PLAY RECAP *******************************************************
ws01.fale.io      : ok=2    changed=1    unreachable=0    failed=0
ws02.fale.io      : ok=2    changed=1    unreachable=0    failed=0

At the moment, our application is not yet reachable since we have no HTTPd rule for that folder. To achieve this, we will need to change the roles/webserver/files/website.conf file with the following content:

<VirtualHost *:80>
    ServerName app.fale.io
    DocumentRoot /var/www/application
    <Directory /var/www/application>
        Options None
    </Directory>
    <DirectoryMatch ".git*">
        Require all denied
    </DirectoryMatch>
</VirtualHost>

As you can see, we are just displaying this application to the users reaching our server with the app.fale.io URL and not to everyone. This will ensure that all your users will have a consistent experience. Also, you can see that we are blocking all access to the .git folder (and all its content). This is needed for security reasons we mentioned earlier in the chapter.

We can now re-run the web playbook to ensure that our HTTPd configuration gets propagated with:

ansible-playbook -i inventory/production playbooks/groups/web.yaml

This is the result we are going to receive:

PLAY [web] *******************************************************


TASK [setup] *****************************************************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure EPEL is enabled] ***************************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure libselinux-python is present] **************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure libsemanage-python is present] *************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure we have last version of every package] *****
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure NTP is installed] **************************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure the timezone is set to UTC] ****************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure the NTP service is running and enabled] ****
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure FirewallD is installed] ********************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure FirewallD is running] **********************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure SSH can pass the firewall] *****************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure the MOTD file is present and updated] ******
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [common : Ensure the hostname is the same of the inventory] *
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [webserver : Ensure the HTTPd package is installed] *********
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [webserver : Ensure the PHP is installed] *******************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [webserver : Ensure git is installed] ***********************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [webserver : Ensure the HTTPd service is enabled and running]
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [webserver : Ensure HTTP can pass the firewall] *************
ok: [ws01.fale.io]
ok: [ws02.fale.io]


TASK [webserver : Ensure HTTPd configuration is updated] *********
changed: [ws01.fale.io]
changed: [ws02.fale.io]


RUNNING HANDLER [webserver : Restart HTTPd] **********************
changed: [ws01.fale.io]
changed: [ws02.fale.io]


PLAY RECAP *******************************************************
ws01.fale.io      : ok=20   changed=2    unreachable=0    failed=0
ws02.fale.io      : ok=20   changed=2    unreachable=0    failed=0

You can now check and see that everything works properly.

In order to deploy an RPM package, we will need to create it in the first place. To do so, the first thing we need is a Spec file.

%define debug_package %{nil}
%global commit0 b49f595e023e07a8345f47a3ad62a6f50f03121e
%global shortcommit0 %(c=%{commit0}; echo ${c:0:7})

Name:       demo-php-app
Version:    0
Release:    1%{?dist}
Summary:    Demo PHP application

License:    PD
URL:        https://github.com/Fale/demo-php-app
Source0:    %{url}/archive/%{commit0}.tar.gz#/%{name}-%{shortcommit0}.tar.gz

%description
This is a demo PHP application in RPM format

%prep
%autosetup -n %{name}-%{commit0}

%build

%install
mkdir -p %{buildroot}/var/www/application
ls -alh
cp index.php %{buildroot}/var/www/application

%files
%dir /var/www/application
/var/www/application/index.php

%changelog
* Tue Oct 04 2016 Fabio Alessandro Locati - 0.1
- Initial packaging

%define debug_package %{nil}
%global commit0 b49f595e023e07a8345f47a3ad62a6f50f03121e
%global shortcommit0 %(c=%{commit0}; echo ${c:0:7})

Name:       demo-php-app
Version:    0
Release:    1%{?dist}
Summary:    Demo PHP application

License:    PD
URL:        https://github.com/Fale/demo-php-app
Source0:    %{url}/archive/%{commit0}.tar.gz#/%{name}-%{shortcommit0}.tar.gz

%description
This is a demo PHP application in RPM format

%prep
%autosetup -n %{name}-%{commit0}

%build

%install
mkdir -p %{buildroot}/var/www/application
ls -alh
cp index.php %{buildroot}/var/www/application

%files
%dir /var/www/application
/var/www/application/index.php

%changelog
* Tue Oct 04 2016 Fabio Alessandro Locati - 0.1
- Initial packaging

sudo dnf install -y fedora-packager

sudo yum install -y mock rpm-build spectool

sudo usermod -a -G mock [yourusername]

mkdir -p ~/rpmbuild/SOURCES

spectool -R -g demo-php-app.spec

rpmbuild -bs demo-php-app.spec

Wrote: /home/fale/rpmbuild/SRPMS/demo-php-app-0-1.fc24.src.rpm

mock -r epel-7-x86_64 /home/fale/rpmbuild/SRPMS/demo-php-app-0-1.fc24.src.rpm

Wrote: /builddir/build/RPMS/demo-php-app-0-1.el7.centos.x86_64.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.d4vPhr
+ umask 022
+ cd /builddir/build/BUILD
+ cd demo-php-app-b49f595e023e07a8345f47a3ad62a6f50f03121e
+ /usr/bin/rm -rf /builddir/build/BUILDROOT/demo-php-app-0-1.el7.centos.x86_64
+ exit 0
Finish: rpmbuild demo-php-app-0-1.fc24.src.rpm
Finish: build phase for demo-php-app-0-1.fc24.src.rpm
INFO: Done(/home/fale/rpmbuild/SRPMS/demo-php-app-0-1.fc24.src.rpm) Config(epel-7-x86_64) 0 minutes 58 seconds
INFO: Results and/or logs in: /var/lib/mock/epel-7-x86_64/result
Finish: run

drwxrwsr-x. 2 fale mock 4.0K Oct 10 12:26 .
drwxrwsr-x. 4 root mock 4.0K Oct 10 12:25 ..
-rw-rw-r--. 1 fale mock 4.6K Oct 10 12:26 build.log
-rw-rw-r--. 1 fale mock 3.3K Oct 10 12:26 demo-php-app-0-1.el7.centos.src.rpm
-rw-rw-r--. 1 fale mock 3.1K Oct 10 12:26 demo-php-app-0-1.el7.centos.x86_64.rpm
-rw-rw-r--. 1 fale mock 184K Oct 10 12:26 root.log
-rw-rw-r--. 1 fale mock  792 Oct 10 12:26 state.log

[web]
ws01.fale.io
ws02.fale.io

[db]
db01.fale.io

[builders]
builder01.fale.io

[production:children]
db
web
builders

- name: Install our private repository
  copy:
    src: privaterepo.repo
    dest: /etc/yum.repos.d/privaterepo.repo
  become: True

[privaterepo]
name=Private repo that will keep our apps packages
baseurl=http://repo.fale.io/
skip_if_unavailable=True
gpgcheck=0
enabled=1
enabled_metadata=1

ansible-playbook -i inventory/production playbooks/groups/web.yaml

PLAY [web] *******************************************************


TASK [setup] *****************************************************
ok: [ws01.fale.io]
ok: [ws02.fale.io]
    ....
PLAY RECAP ******************************************************
ws01.fale.io       : ok=20   changed=1    unreachable=0    failed=0
ws02.fale.io       : ok=20   changed=1    unreachable=0    failed=0