The Puppet executable can be run from the command line. Let's begin with confirming which version of Puppet we are running. We can check the version by running the following command:

puppet --version

At the time of writing, the command run on the Learning VM, it shows the Version as 3.7.3 (Puppet Enterprise 3.7.1.).

The first Version number 3.7.3 is for the open source version of Puppet that we are using. The second Version number 3.7.1 is for the Puppet Enterprise version number.

What is the difference between these two versions? The difference between them is how these products are packaged, distributed, and supported.

So now we know how to extract the version using the Puppet command-line utility. Let's shift our focus to the resources next.

Resources in Puppet are known as types. Types are operating system resources such as a file, a user, or a package. There are tens of built-in types in Puppet, and in addition to these, you can create your own custom types to manage resources.

We will learn more about custom types later in Chapter 5, Load Balancing the Cluster, but for now we will take a look at the built-in types and see how to use them.

A complete list of available built-in types is available on the Puppet Labs website at http://docs.puppetlabs.com/references/latest/type.html.

Run the puppet describe --list command in the Learning VM to list all the built-in types known to Puppet.

The output will contain about 60 resources and their descriptions. To paginate the output, you can extend the command by adding | less to the end of it.

Here is the command to view the output page by page:

puppet describe --list | less

You should now be able to scroll the output up and down using the arrow keys, and you can exit the view by pressing Q on the keyboard.

All the Puppet types have attributes that are used to describe the characteristics of the resources we want Puppet to manage.

For example, the type user has attributes such as a name for the user name and a password for the user account password.

To list the available attributes for a specific type of resource, you can use the puppet describe <type> command. For example, to list the available attributes of a type user, you can run the puppet describe user, or puppet describe user | less command to paginate the output.

If you scroll down the list of attributes, you can find a password attribute that is used to set a password for the user account. Another attribute that you can find on the list is called ensure, which defines the state of the user account. The attribute can have three values:

We can manage Puppet resources from the command line using the following syntax: puppet resource <type> <name> <attribute1>=<value> <attribute2>=<value>.

Let's create a user called Elisa on the system using Puppet. In the Learning VM terminal, type in the following command and and hit the Enter key:

puppet resource user Elisa ensure=present

When the command is executed successfully, it produces the following output:

The first line of the output displays a notice confirming that the user account Elisa was created by Puppet. Lines 2-4 show the syntax that we will be using when we declare resources in the Puppet manifest files. The manifest files will be explained more in detail later.

Let's take a look at the syntax line by line:

The name Elisa on line 2 has two use cases. Firstly, it declares the name of the Puppet resource. Each Puppet resource must have a unique name, otherwise Puppet reports an error.

Secondly, the name Elisa is used as the name of the user account that was created. If the user statement contains a name attribute (alongside the ensure attribute), then the value of the name attribute would become the name of the user account and the name Elisa would only be used as the Puppet resource name.

As the name attribute is omitted, Puppet will use the name for the Puppet resource name as well as for the user account name.

Declaring the following statement in the Puppet manifest would result in the Jakob user account being created in the system, as the name attribute would take priority over the Puppet resource name Elisa:

user { 'Elisa':
  ensure => 'present',
  name   => 'Jakob',
}

This produces an output that is similar to the following, although the numeric information in the output may vary between the systems:

uid=501(Elisa) gid=501(Elisa) groups=501(Elisa)

Next, we can remove the account Elisa from the system by setting the ensure attribute value to absent.

puppet resource user Elisa ensure=absent

Assuming that the command executes successfully, you should see the following output:

This output is very similar to how we created the user account Elisa except that line 1 confirms that the user account Elisa was removed and line 3 has the ensure attribute value set to absent, which results in the account being removed when declaring a resource in the Puppet manifest file.

To confirm whether Puppet really removed the account, you can run the command id Elisa again, which will confirm that the account no longer exists in the system.

The command id Elisa should produce the following output:

id: Elisa: No such user

Congratulations! You just did two system configuration changes using Puppet. It wasn't hard, right?

Puppet dry run

Sometimes, you want to simulate configuration changes without applying the changes to the system. This can be done by adding the --noop parameter after the resource keyword in the Puppet command.

To simulate account creation without creating the account, we can extend the previous user account creation command with the --noop option:

puppet resource user Elisa ensure=present --noop

Line1 in the output tells us that the user account does not exist in the system, and if you run the following command without the --noop option, Puppet would create an account:

Notice: /User[Elisa]/ensure: current_value absent, should be present (noop)
user { 'Elisa':
  ensure => 'absent',
}

The --noop option comes in handy when testing Puppet commands for a syntax. To demonstrate this, we declare an invalid value removed for the attribute ensure:

puppet resource user Elisa ensure=removed --noop

Puppet will return an error that tells you that we have used an invalid value for the ensure attribute:

Use Puppet to examine the current state of resources

Puppet can also be used to query resources from the system. Information produced by the query can be helpful when we are uncertain about what syntax we should be using to create a resource.

Earlier, we created a user account called Elisa. This account was created without a password, which means that the account cannot be used for interactive logins. Let's recreate the user account and use the password attribute to set a password for the account.

As passwords on Linux are encrypted, we must provide them to the password attribute in the encrypted format.

We know that the user account root on the Learning VM uses the password puppet, but we yet don't know how the password would look like in the encrypted format.

No problem, as we can query the password with the command and then use the encrypted password when recreating the user account Elisa.

The following command shows all the attributes for the root user account:

Puppet resource user root

The preceding command produces the following output:

Now we can create the user account Elisa with the password attribute, which is the same as the value of the password attribute for the user root.

The following command will create the user account Elisa that uses the password puppet:

puppet resource user Elisa ensure=present \
managehome=true \
password='$1$jrm5tnjw$h8JJ9mCZLmJvIxvDLjw1M/'

I've split the command into three lines with the backslash character at the end of the first two lines.

Note

The $1$jrm5tnjw$h8JJ9mCZLmJvIxvDLjw1M/ string is a hash of the password puppet. Please note that we have to use single quotes around the password hash '$1$jrm5tnjw$h8JJ9mCZLmJvIxvDLjw1M/' because the string contains characters that the Linux command line otherwise interprets as a control character.

Now we can test whether we can log on to the system as the user Elisa.

Log out from the system using the logout command. Then, log in with the username Elisa and password puppet. You will see the following welcome screen, which confirms that the login of Elisa was successful:

useradd Jakob

/usr/sbin/useradd: Permission denied

Puppet DSL and manifests

I've mentioned Puppet manifests earlier, but I haven't yet explained what manifests are. Puppet manifests are text files that declare one or more Puppet resources. Instead of running Puppet resource commands on the command line, you can declare resources in the manifest file and apply the manifest to the system.

Puppet manifests uses the Puppet Domain Specific Language (DSL) and resource statements in the manifest file, which are described in a syntax that looks very similar to a Hash data type in the Ruby language.

We can use our user account Elisa as a simple example of the Puppet manifest syntax.

First, log out from the user account Elisa by running the logout command. Then, log on to the system as a root user and remove the user Elisa from the system with the following command:

puppet resource user Elisa ensure=absent

Then, inspect the state of the user account Elisa with the following commands:

puppet resource user Elisa
user { 'Elisa':
  ensure => 'absent',
}

In the Ruby language, if we try to declare a hash called User that contains a key Elisa with the value of the ensure attribute as absent, we will declare it using the following syntax:

User = { 'Elisa' =>
  { 'ensure' => 'absent' }
}

If you compare the preceding two code blocks, you can see that the Puppet DSL syntax looks similar to the Ruby language syntax, but it is slightly simpler and easier to read than the Ruby equivalent.

The output of the preceding Puppet resource command is spread across three lines only in order to make it easier for us to read. The Puppet parser that reads the manifest file doesn't care about the line feed characters.

The preceding user resource can be declared on a single line as follows:

user { 'Elisa': ensure => 'absent', }

We now have the Puppet DSL representation of the user resource Elisa.

Let's run the command again, and you will notice the difference in the command output compared to the previous Puppet run:

puppet apply user.pp

This time, the output is shorter. The lines that notify us that the users Jakob and Markus were created are missing in this Puppet run:

Notice: Compiled catalog for learning.puppetlabs.vm in environment production in 0.14 seconds
Notice: Finished catalog run in 0.27 seconds

This is due to the idempotent nature of Puppet. As the users Jakob and Markus already exist in the system, Puppet doesn't attempt to recreate these accounts. Idempotency in Puppet means that you can apply the same manifest as many times as you like, and only when the state of the resource in the system is different from the state of the resource declared in the manifest, will Puppet ensure that the required configuration changes are performed according to the manifest.

To demonstrate how Puppet handles idempotency, we will remove the user Markus with the following command, which we are familiar with:

puppet resource user Markus ensure=absent

Then, apply the manifest again with the puppet apply user.pp command, and you can see that the user Jakob, which we did not remove earlier, does not appear in the output but the user Markus is recreated.

Here is the command again:

puppet apply user.pp

The output of the command is as follows:

Notice: Compiled catalog for learning.puppetlabs.vm in environment production in 0.15 seconds
Notice: /Stage[main]/Main/User[Markus]/ensure: created
Notice: Finished catalog run in 0.35 seconds

So far, we have practiced how to manage system resources from the command line with puppet resource command, and also learned how to manage resources with the Puppet manifest and puppet apply command. When we start expanding our environment with new hosts and increase the number of resources that Puppet manages on these hosts, you will notice that the Puppet command line doesn't scale very well. The Puppet command line typically manages a single resource, such as user Elisa or user Jakob. Each of these resources was created with its own command. If I have 100 user accounts to be managed, then that would result in the same amount of commands to be run, which would be a very tiring job for anyone to do.

Puppet is a configuration management and automation tool that helps you eliminate repetitive tasks, such as creating 100 user accounts. Instead of running the puppet resource command 100 times, it is better if we add all our users once to a single manifest file, call the file with a puppet apply command, and let the Puppet do the hard work for us. Puppet manifests are types of recipes for your system configuration. Once you have described your system configuration in the form of a manifest, you can easily transfer the recipe onto another host and apply the configuration with a single command.

The phrase "everything is a file" that is often associated with the Linux operating system makes it an ideal environment for Puppet to manage. Puppet is very good at managing files. Puppet's file resource can create files from static source files. You can define the file content with the content attribute, or you can create files with a dynamic content using templates. A file resource can also be used to manage directories and links.

The syntax of a file resource is very similar to the user resource syntax, only the set of available attributes is different. Here is a simple example of how to create an empty directory called /root/Documents:

file {  '/root/Documents':
  ensure => directory;
}

The first line defines the type of resource we want Puppet to manage, followed by the name of the directory that Puppet creates.

The ensure attribute on line two says that the file resource must be a directory. If we omit the ensure attribute, Puppet will create a file instead of a directory.

The closing curly brace } on the third and the last line ends the file resource statement.

Let's do a practical experiment with the file resource and write a manifest file that sets a log in greeting message when the user Jakob logs in. In order to do this, our manifest must fulfill the following two criteria:

To start with, let's remove the user Jakob from the system so that we can easily recreate the account with a password, and tell Puppet to create a home directory for the user:

puppet resource user Jakob ensure=absent

Now when the user Jakob has been removed, let's generate a user resource for Jakob and redirect the output to the file called jakobs-login.pp. Again, we use the single > character to create a new file:

puppet resource user Jakob > jakobs-login.pp

Then, using the >> notation to redirect the output to the jakobs-login.pp file, we can generate the file resource snippet for the .bash_profile file that will be placed under the home directory of Jakob with the following command:

puppet resource file /home/Jakob/.bash_profile >> jakobs-login.pp

Now that we have the manifest body ready for editing, we can open the jakobs-login.pp file in the Nano editor:

nano jakobs-login.pp

On opening the file, you should see the following file content:

user { 'Jakob':
  ensure => 'absent',
}
file { '/home/Jakob/.bash_profile':
  ensure => 'absent',
}

Let's begin by changing the ensure attribute value from absent to present for the user resource Jakob.

Then, to tell Puppet to create the home directory for the user, we need to use the managehome attribute and set its value to true. The managehome attribute is specific to a user resource, and we can use it to tell Puppet to create a home directory for the user under the /home directory. The home directory is needed to store the .bash_profile file, which we will take a look at shortly.

Finally, to enable Jakob to log in using the password puppet, we should set the encrypted password attribute value to $1$jrm5tnjw$h8JJ9mCZLmJvIxvDLjw1M/.

This is how the user resource for Jakob should look like after the changes:

user { 'Jakob':
  ensure => 'present',
  managehome => true,
  password => '$1$jrm5tnjw$h8JJ9mCZLmJvIxvDLjw1M/',
}

Before we move on to the file resource, let's save the changes with Ctrl + X and hit Enter. Then, apply the manifest to the puppet apply command:

puppet apply jakobs-login.pp

If the Puppet run was successful, you should see the following output:

Notice: Compiled catalog for learning.puppetlabs.vm in environment production in 0.63 seconds
Notice: /Stage[main]/Main/User[Jakob]/ensure: created
Notice: /Stage[main]/Main/File[/home/Jakob/.bash_profile]/ensure: removed
Notice: Finished catalog run in 0.35 seconds

If we take a look at the third line of the output, we can see that Puppet removed the /home/Jakob/.bash_profile file although we had not yet created it. This is because of the managehome attribute that we declared for the user Jakob, which results in the Linux environment to create the file when the user is created. Because we haven't yet modified the file resource for /home/Jakob/.bash_profile in the manifest, the ensure attribute value is absent. This results in Puppet removing the file.

Don't worry, as we will now tell Puppet to recreate the file with the content that we specify:

The VirtualBox Guest Additions provides a service called shared folders that enables us to share folders from the machine that VirtualBox is running on and access them directly from the virtual machine. This means that we are no longer bound to use the text editor inside the virtual machine, as we can use better text editors, such as Notepad++ on Windows or Gedit on Linux. As the files stored in the shared folders are stored outside the virtual machine on the host computer's hard drive, and not in the virtual disk file used by the VM, we are guaranteed that the files are preserved when we revert to the previous snapshot or delete the virtual machine.

Another advantage of shared folders is that folders can be attached to multiple virtual machines simultaneously. This makes the manifest distribution easy as we can write the manifest once, and we can then apply the manifest across all the virtual machines by running the puppet apply command on each VM, pointing each to the manifest file stored in the Shared Folders directory.

In addition to shared folders, VirtualBox provides the so-called host-only networking feature that enables us to attach virtual network interfaces to the virtual machine. This feature is handy as we can create a private network inside the VirtualBox environment, where virtual machines can communicate with each other directly. Virtual machines that have the so-called host-only network adapter attached to them can operate in isolation, and they are able to connect to each other using private IP addresses that are allocated by the DHCP server built into the VirtualBox. When addresses are allocated by the VirtualBox, there is no dependency to external routers. The host-only networking feature enables us to spin up the development environment when we are working offline, without any connectivity to Wi-Fi or the Internet.

The shared folders service enables us to access files that are stored on the host computer from the virtual machine. To configure shared folders, you need to decide which directory you want to share and what mount point to use inside the virtual machine to access the directory. In the following example, I'll share the /home/jussi/learning directory on the host computer and use the /learning mount point on the virtual machine.

Select the Puppet Learning VM from the list of virtual machines:

The host-only network interface enables a virtual machine to communicate with other virtual machines that are connected to the same virtual network. We will make use of this later in the book when we run multiple virtual machines in parallel and configure them to communicate with each other.

First, we should confirm that VirtualBox has the host-only network interface that we can assign to the virtual machine:

Next, we should attach the host-only network interface with the virtual machine:

Let's take a look at whether the shared folders are available on the Puppet Learning VM. Start the virtual machine and log on with the username root and the password puppet.

We can check whether the shared folder configuration was successful by running the following command:

df

If the shared folders are working, you should see a /media/sf_learning mount point appearing at the bottom of the view, as shown in the following screenshot:

Before the virtual machine can start using the host-only network interface that we added, we have to create a file that tells the operating system how the new network interface should be configured.

We will configure the host-only network interface as a device name eth1 on the virtual machine, set it to use the DHCP network address, and enable it when the system boots up.

Such a configuration can be done by creating a /etc/sysconfig/network-scripts/ifcfg-eth1 file with the following content:

DEVICE="eth1"
BOOTPROTO="dhcp"
ONBOOT="yes"

Instead of creating the file manually, we can use Puppet's file resource to create the file. Here's a file resource that does the configuration for us:

file {
  '/etc/sysconfig/network-scripts/ifcfg-eth1':
    content => 'DEVICE="eth1"
BOOTPROTO="dhcp"
ONBOOT="yes"',
}

The preceding file resource may initially look a bit messy, but let's take a look at it in detail and try to make some sense out of it:

Now add the preceding file resource statement to the network.pp manifest file and apply the manifest:

If no errors occurred while the manifest was applied, Puppet reports the following lines:

Notice: Compiled catalog for learning.puppetlabs.vm  in environment production in 0.12 seconds 
Notice: /Stage[main]/Main/File[/etc/sysconfig/network-scripts/ifcfg-eth1]/ensure: defined content as '{md5}f98d0b3852ba630aeb2b7d37f8c74b26' 
Notice: Finished catalog run in 0.27 seconds 

Restart networking

To tell the operating system to refresh the network configuration, we can use the following command:

service network restart

If the network was restarted successfully, you should see the following information printed on the virtual machine console:

We have now arrived at the point where we are going to "branch" the Puppet Learning VM to create a slimmed down version of the Puppet Learning VM, which I'll be referring to as the puppet-agent node. The slimmed down version of the Learning VM is based on the current state of the Puppet Learning VM, but it will have a smaller memory footprint than the original Puppet Learning VM, as we will remove some of the packages that are preinstalled on the Puppet Learning VM. The Puppet Learning VM in its current state will be used as the puppetmaster node that manages the puppet-agent nodes. In Chapter 6, Scaling Up the Puppet Environment, we will learn how to connect the puppet-agent nodes with the puppetmaster node, but for now, we'll focus on how to branch the virtual machine and remove packages from it using Puppet.

Purging package resources

You should power on the new virtual machine so that we can take a look at what packages can be removed to reduce the memory usage on the virtual machine.

Select the puppet-agent virtual machine from the list, and click on the Start button. Once the machine has booted up, log in using the user name root and the password puppet.

Once you have logged in, you can check the current memory consumption by running the free –m command.

The command output shows the current memory usage of 1570 MB, and the virtual machine has only 322 MB of free memory space. We should be able to halve the memory consumption by removing packages that are not needed by the puppet-agent node.

We want to remove all the packages that are used by the puppetmaster and only retain the puppet and mcollective client packages that are needed to run the puppet-agent node.

Here is the complete list of packages that we can safely remove from the system:

• pe-puppet-dashboard

• pe-puppetserver

• pe-puppet-license-cli

• pe-puppetdb

• pe-puppetdb-terminus

• pe-memcached

• pe-postgresql

• pe-activemq

• pe-console-services

Earlier, when we removed a file, we did it with an attribute called ensure. To remove packages, we will also use the ensure attribute but with a value purge. As all the package resources share the same attribute value, we should consolidate the removal of all the packages into one package resource using an array of package names as the Puppet resource name.

In the following example, I've split the package name array on multiple lines to make it easier to read:

package {
  ['pe-puppet-dashboard',
   'pe-puppetserver',
   'pe-puppet-license-cli',
   'pe-puppetdb',
   'pe-puppetdb-terminus',
   'pe-memcached',
   'pe-postgresql',
   'pe-activemq',
   'pe-console-services']:
     ensure => 'purged';
}

The preceding package resource contains the following elements:

• Line 1 begins with the package resources.

• Lines 2-10 declare the array of packages that we want Puppet to manage. You will notice that the shutdown is much faster now due to the removal of the packages.

We have already practiced quite a few times how to take a snapshot of the virtual machine, so I will just provide you with a quick overview of the steps without screenshots:

1. Ensure that the virtual machine is powered off.

2. Click on the Snapshots button in the top right-hand corner of the VirtualBox Manager window.

3. Click on the Take Snapshot button.

4. Provide a meaningful name for the snapshot. I call my snapshot puppet-agent-web.

5. Then, click on OK to create the snapshot.

The easiest way to get familiar with a module structure is to create an empty module with the puppet module generate command. As we are in the process of building a web server that runs a web application, we should give our module a meaningful name, such as learning-webapp.

A Puppet class is a container for Puppet resources. A class typically includes references to multiple different types of resources and can also reference other Puppet classes.

The syntax for declaring a Puppet class is not that different from declaring Puppet resources. A class definition begins with the keyword class, followed by the name of the class (unquoted) and an opening curly brace ({). A class definition ends with a closing curly brace (}).

Here is a generic syntax of the Puppet class:

class classname {
}

Let's take a look at the manifests/init.pp file that you just created with the puppet module generate command. Inside the file, you will find an empty Puppet class called webapp. You can view the contents of the manifests/init.pp file using the following command:

# cat /media/sf_learning/learning-webapp/manifests/init.pp

The init.pp file mostly contains the comment lines, which are prefixed with the # sign, and these lines can be ignored. At the end of the file, you can find the following declaration for the webapp class:

class webapp {
}

The webapp class is a Puppet class that does nothing as it has no resources declared inside it.

class webapp {
}

class webapp {
   notify {  'Applying class webapp':
   }
}

# cd /media/sf_learning
# mv learning-webapp webapp

# puppet apply --modulepath=./ webapp/tests/init.pp

Installing a module from Puppet Forge

Puppet Forge is a public Puppet module repository (https://forge.puppetlabs.com) for modules that are created by the community around Puppet. Making use of the modules in Puppet Forge is a great way to build a software stack quickly, without having to write all the manifests yourself from scratch.

One of the key deliverables of this chapter is to build a fully functioning web server, and we can do this very easily by building the web server from modules that are available in Puppet Forge.

The web server that we are going to install is a highly popular Apache HTTP Server (http://httpd.apache.org/), and there is a module in Puppet Forge called puppetlabs-apache that we can install. The Puppetlabs-apache module provides all the necessary Puppet resources for the Apache HTTP Server installation.

Note that the puppet module installation requires an Internet connection. To test whether the Puppet Learning VM can connect to the Internet, run the following command on the command line:

# host www.google.com

On successful completion, the command will return the following output:

www.google.com has address 216.58.211.164
www.google.com has IPv6 address 2a00:1450:400b:801::2004

Note

Note that the reported IP address may vary. As long as the host command returns www.google.com has address …, the Internet connection works.

Now that the Internet connection has been tested, you can now proceed with the module installation.

Before we install the puppetlabs-apache module, let's do a quick search to confirm that the module is available in Puppet Forge. The following command will search for the puppetlabs-apache module:

# puppet module search puppetlabs-apache

When the search is successful, it returns the following results:

Then, we can install the module. Follow these steps to install the puppetlabs-apache module:

1. In the Puppet Learning VM, go to the shared folders /media/sf_learning directory by running the cd /media/sf_learning command.

2. Then, run the following command:

   # puppet module install --modulepath=./ puppetlabs-apache
   

   The --modulepath=./ option specifies that the module should be installed in the current /media/sf_learning working directory

3. The installation will take a couple of minutes to complete, and once it is complete, you will see the following lines appear on the screen:

   Notice: Preparing to install into /media/sf_learning ...
   Notice: Downloading from https://forgeapi.puppetlabs.com ...
   Notice: Installing -- do not interrupt ...
   /media/sf_learning
   └─┬ puppetlabs-apache (v1.2.0)
      ├── puppetlabs-concat (v1.1.2)
      └── puppetlabs-stdlib (v4.8.0)
   

Let's take a look at the output line by line to fully understand what happened during the installation process:

• Line 1 tells us that the module is going to be installed in the /media/sf_learning directory, which is our current working directory. This directory was specified with the --modulepath=./ option in the puppet module install command.

• Line 2 says that the module is going to be installed from https://forgeapi.puppetlabs.com/, which is the address for Puppet Forge.

• Line 3 is fairly self-explanatory and indicates that the installation process is running.

• Lines 4 and 5 tell us that the puppetlabs-apache module was installed in the current /media/sf_learning working directory.

• Line 6 indicates that as part of the puppetlabs-apache module installation, a puppetlabs-concat dependency module was also installed.

• Line 7 lists another dependency module called puppetlabs-stdlib that got installed in the process.

Now you can run the tree -L 1 command to see what new directories got created in /media/sf_learning as a result of the puppet module install command:

# tree -L 1
├── apache
├── concat
├── stdlib
└── webapp
4 directories, 0 files

Note

The argument -L 1 in the tree command specifies that it should only traverse one level of directory hierarchy.

Installing Apache HTTP Server

Now that the puppetlabs-apache module is installed in the filesystem, we can proceed with the Apache HTTP Server installation.

Earlier, we talked about how a Puppet class can be referenced with the include keyword. Let's see how this works in practice by adding the include apache statement to our webapp class, and then applying the webapp class from the command line.

Open the webapp/manifests/init.pp file in your preferred text editor, and add the include apache statement inside the webapp class.

I like to place the include statements at the beginning of the class before any resource statement. In my text editor, the webapp class looks like the following after the include statement has been added to it:

Once you have saved the webapp/manifests/init.pp file, you can apply the webapp class with the following command:

# puppet apply --modulepath=./ webapp/tests/init.pp

This time, the command output is much longer compared to what it was when we applied the webapp class for the first time. In fact, the output is too long to be included in full, so I'm only going to show you the last two lines of the Puppet report, which shows you the step where the state of the Service[httpd] resource has changed from stopped to running:

Notice: /Stage[main]/Apache::Service/Service[httpd]/ensure: ensure changed 'stopped' to 'running'
Notice: Finished catalog run in 65.20 seconds

Testing the Apache HTTP Server

Assuming that the installation was successful, you can now test whether the HTTP interface on the Puppet Learning VM is accessible in a web browser on the host computer.

You must start by discovering what IP address the Puppet Learning VM uses for the host-only network interface called eth1. The discovery is the easiest to do with the facter utility that is installed as part of Puppet. We will take a look at the facts in detail shortly, but for now, we can just query a key called ipaddress_eth1, and this will return the IP address of the network interface, eth1.

Here is the command and output when I run it on my Puppet Learning VM:

# facter ipaddress_eth1
192.168.56.10

Facter shows that the host-only networking interface eth1 uses an IP address 192.168.56.10. Based on this information, I can type in the URL http://192.168.56.10 in the web browser on the host computer, and the following web page will load up:

Puppet templates (https://docs.puppetlabs.com/guides/templating.html) are files that produce dynamic content for files on the target machine. Templates are written in the ERB Templating Language, which is a mixture of static text, variable references, and functions. The ERB language syntax can get complex especially when using functions, but I like simple things, so we will begin by writing a template that only contains static text, and learn how to create a file resource that populates the template. Later, we will modify the template and learn how to use variables to make the file content dynamic.

Puppet templates live inside the templates directory within the Puppet module. Unfortunately, the Puppet module generate command does not create the templates directory, so we have to create this directory by hand.

Here are the steps on how to create the templates directory inside the webapp module from the command line:

After the templates directory has been created, you will see the following directory structure inside the webapp module:

# tree -L 1
├── Gemfile
├── manifests
├── metadata.json
├── Rakefile
├── README.md
├── spec
├── templates
└── tests

4 directories, 4 files

<html>
  <body>
    <h1>Welcome to Puppet Learning VM</h1>
    <p>The contents of this file was generated by Puppet</p>
  </body>
</html>

Creating a file resource for the template file

The template file has been created, and we have to now instruct Puppet where to find the template and where to populate the file on the target machine.

Open the manifests/init.pp file in the text editor on the host computer, and add the following file resource statement to the webapp class:

file { '/var/www/html/index.html':
  content => template('webapp/index.html.erb'),
  owner   => 'apache',
  require => User['apache'];
}

As we have introduced a couple of new attributes for the file resource, I think it is worth taking a look at the preceding resource declaration in detail:

• Line 1 begins with a new file resource. The name of the resource defines the path and the name of the file that Puppet will create from the template.

• Line 2 defines a content attribute, and the attribute value begins with a function called template. The function takes an argument that defines the location of the template file in the webapp module.

  The syntax of the template file location may look a bit strange, as it doesn't specify the directory called template, where the template file lives. The syntax that Puppet uses for referencing templates is in the format of <modulename>/<template>.

• Line 3 defines an owner attribute that is used to specify a user account that Puppet should make as the owner of the file on the target machine. The apache attribute value results in the file owner to become an apache user, which is the user account that runs the Apache HTTP Server process.

• Line 4 introduces a require attribute, which is used to set the order for Puppet to process resources. The User['apache'] value means that Puppet must process the apache user resource (declared in the puppetlabs-apache module) prior to processing the file resource.

  This attribute ensures that the apache user is created prior to setting the file owner as the apache user.

• Line 5 closes the file resource statement.

The webapp class should have the following content once a new file resource has been added to it:

Apply the Puppet class and visit the new landing page.

When the file resource is added to the webapp class, you can reapply the class with the following two commands:

# cd /media/sf_learning
# puppet apply --modulepath=./ webapp/tests/init.pp

After a successful Puppet run, you will see the following lines on the screen:

# puppet apply --modulepath./ webapp/tests/init.pp
Notice: Compiled catalog for web.development.vm in environment production in 5.97 seconds
Warning: The package type's allow_virtual parameter will be changing its default value from false to true in a future release. If you do not want to allow virtual packages, please explicitly set allow_virtual to false.
(at /opt/puppet/lib/ruby/site_ruby/1.9.1/puppet/type/package.rb:430:in 'block (3 levels) in <module:Puppet>')
Notice: Applying class webapp
Notice: /Stage[main]/Webapp/Notify[Applying class webapp]/message: defined 'message' as 'Applying class webapp'
Notice: /Stage[main]/Webapp/File[/var/www/html/index.html]/ensure: defined content as '{md5}751763924b26dc9a7f3e4c81bc4bf158'
Notice: Finished catalog run in 9.53 seconds

The warning message in the Puppet report output can be safely ignored.

If you open the web browser and type http://192.168.56.10/ in the address bar, you should see your newly-designed landing page loading up in the browser:

Facter is an inventory application that comes bundled with Puppet, and it provides an interface with a range of data about your system. Earlier in the chapter, we used the facter command to query the IP address of an eth1 network interface. This is just one of the many available system properties that are available via Facter.

In addition to built-in facter queries, you can also create your own facts and bundle them with the module or make Facter read values from particular files in the filesystem. We will take a look at the custom facts later in this book, but for now, we'll focus on how to make use of the built-in facts.

Facter integrates nicely with Puppet and Puppet templates. Next, we will practice how to make use of the facter to turn the currently static landing page content into a more dynamic web page and expose some of the system information on the landing page.

# facter

# facter | less

# facter operatingsystem

CentOS

Accessing facts from the Puppet template

Accessing facts from the Puppet template is done by a reference to the variable name that matches the name of the fact. For example, to reference the fact operating system, we can do it with the following statement in the Puppet template:

<%= @operatingsystem %>

Note

Since Puppet Version 3.x, local variable references must be prefixed with the @ sign. Although facts are not technically local variables, we can reference them as local variables with the @ prefix. An alternative and recommended way to reference nonlocal variables in Puppet 3.x is to use the <%= scope['::operatingsystem'] %> template syntax. I personally prefer the @ notation as it is shorter and it works just fine.

The name of the fact that we want to reference must be wrapped inside an expression that begins with the opening marker <%= and ends with the closing marker %>. When the ERB template parser reads the template and detects a character sequence <%, followed by %>, it treats its contents as a block of executable Ruby code. Any characters outside the <% and %> tags are treated as plain text.

A tag with the equals sign such as <%= means that the ERB template parser must substitute the expression with the result of the command output.

For example, in order to allow the Puppet template to produce a string My operating system is CentOS, we can use the following ERB syntax in the template file:

My operating system is <%= @operatingsystem %>

Let's add the preceding statement to the existing template, and see what happens when Puppet applies the template.

Open the index.html.erb file under webapp/templates/ in the text editor, and add the statement to it. In the following example, I've wrapped the statement inside the HTML <p> and </p> tags to allow the browser to display the text with a font that is consistent with the rest of the page:

<html>
  <body>
    <h1>Welcome to Puppet Learning VM</h1>
    <p>The contents of this file was generated by Puppet</p>
    <p>My operating system is <%= @operatingsystem %></p>
  </body>
</html>

Once you have saved the file, you can apply the changes with the following commands:

cd /media/sf_learning/
puppet apply --modulepath=./ webapp/tests/init.pp

Once Puppet has applied the new version of the template, you should refresh the browser window at http://192.168.56.10/ by pressing F5 on the keyboard. If the changes were applied successfully, you should now have the page with the following content in front of you:

Accessing facts from Puppet manifests

Facts are accessible in the Puppet manifest file through a reference to the fact name prefixed with the $:: (dollar and double colon) character sequence. For example, a reference to the fact operatingsystem can be done with the $::operatingsystem notation.

Let's create a local Puppet variable and store some facter values in it. We can then reference the local variable in the Puppet template, and use a simple for loop inside the template to populate the Facter values on the landing page.

We can choose any name for the variable, and as we are going to store the facter values in the variable, I'll call it $fact_list. A local variable in Puppet is declared by adding a dollar character as a prefix for the variable name, for example, $fact_list. In order to iterate through values in the variable, the variable type has to be an array, which is created with the following syntax:

$fact_list = []

The preceding example creates an empty array. Let's add some content to the array with double-quoted strings separated by comma characters:

$fact_list = ["IP address $::ipaddress_eth1", "Uptime $::uptime"]

The $fact_list array now contains two elements. The first element in the array begins with a string IP address, followed by a reference to the fact called ipaddress_eth1. The second array element contains an Uptime string and a reference to the fact uptime.

Before we move on to editing the template file, you should add the $fact_list array to the webapp class in the manifests/init.pp file. Once the array is added to the webapp class, it should have the following content:

A simple for loop in the Puppet template

We now have a $fact_list array with two elements but nothing yet references the variable. Next, we are going to modify the templates/index.html.erb file, and learn how to reference the local variable from the template and iterate through elements that are stored in the array.

A local variable reference in a template is done in exactly the same manner as referencing facts, by adding the @ prefix to the variable name. The iteration over the array can be done with the method each, which is a method provided by an array data type in the Ruby programming language.

The referencing and iteration can be done with the following code block:

<% @fact_list.each do | fact | %>
<p> <%= fact %> </p>
<% end %>

I'll explain what the preceding three lines do:

• Line 1 contains a reference to the variable facts, which we defined in the init.pp manifest file. Variable facts call the Ruby array method each that results in iteration over the array. Each element in the array is referenced with the local variable called fact for which you can find a reference on line 2.

• Line 2 begins with an HTML <p> tag, followed by a reference to the fact local variable that was instantiated on line 1. The line ends with closing the HTML </p> tag.

• Line 3 has the end statement that terminates the iteration created by the each method.

Now it's time to try this out in action. Once you have added the preceding code block to the templates/index.html.erb directory, the file should have the following content, where the latest code addition is highlighted in red:

After you have saved the template file, apply the template with the following commands:

# cd /media/sf_learning/
# puppet apply --modulepath=./ webapp/tests/init.pp

Once Puppet has applied the new template, you will see the IP address and the Uptime information displayed on the landing page when you reload the page.

Here's a screenshot showing you the landing page content after the changes were applied successfully:

If you wait for a minute and try rerunning the puppet apply command and reload the landing page, you should see that the Uptime value gets updated at every Puppet run.

One great thing about Puppet is that it makes the deployment process repeatable across multiple machines. Once you have described the deployment process in the form of Puppet manifests, you can quickly deploy a cluster of machines that share the common configuration. For example, we can easily build a cluster of web servers that are deployed by the webapp module. Let's try this in action and create two web servers that incorporate the webapp module.

The high-level plan is as follows:

# poweroff

Power on both the virtual machines and apply the webapp class

You should now see three virtual machines in the list of VMs in the VirtualBox Manager view. These machines are called:

• learn_puppet_centos-6.5

• puppet-agent

• puppet-agent-web-clone

Power on the puppet-agent and the puppet-agent-web-clone virtual machines by selecting the machine and then clicking on the Start button.

Once both the machines have booted up, you should find the IP address information for each host displayed on the log on the screen under the section called My IP information. Make a note of the IP address from the IP address range 192.168.56.x for both the hosts. You will need this information in a little while after you have applied the webapp class to both the hosts.

The following screenshot shows that the virtual machines that I powered on use IP addresses 192.168.56.11 and 192.168.56.12:

Now log on to both the virtual machines using these login details:

• Username: root

• Password: puppet

Then, apply the webapp class by running the following two commands on both the machines:

# cd /media/sf_learning
# puppet apply --modulepath=./ webapp/tests/init.pp

Once the webapp class has been applied to both hosts, you can check whether the landing page is available on both machines by opening the web browser on the host computer and accessing http://<ipaddress>, where the <ipaddress> should correspond to the IP address of your virtual machines.

The following screenshot shows the landing page from both the web servers, following the successful deployment:

Let's do a quick recap on the current state of virtual machines and snapshots before we create a new clone. In the VirtualBox Manager view, I can see the following three virtual machines:

I'll include a screenshot of the VirtualBox Manager view on my machine, so you can compare it with the view on yours:

In the the preceding screenshot, you can see that I've highlighted the snapshot puppet-agent-web of the virtual machine puppet-agent. This is the snapshot we should use as the base to create a new clone for the Nagios module development.

You may already be familiar with how to clone the virtual machine from the snapshot, but just in case you need a reminder, here are the steps to carry out the task:

Once the clone is created, you should see a fourth virtual machine appearing on the list called puppet-agent-nagios. Before you launch the new virtual machine, create a new snapshot called Base image that can be used to roll back the machine state in the case of an emergency. I trust the you are familiar with the snapshot creation by now so I won't instruct it again, but if you feel unsure, you may revisit the Snapshot of the virtual machine section in Chapter 1, Puppet Development in Isolation, that instructs the process.

I'd also advise to lower the system memory allocation for the puppet-agent-nagios virtual machine. This can be done in the virtual machine settings (shortcut keys Ctrl + S) by selecting the System category and reducing the Base Memory to 512 MB.

Now, let's launch the virtual machine and start writing our new Nagios module.

Once the puppet-agent-nagios virtual machine is running, you can log in with the username root and password puppet. Next, you need to change the directory to /media/sf_learning and generate the Nagios module template files. This can be done with the following command sequence:

# cd /media/sf_learning
# puppet module generate learning-nagios --skip-interview

The puppet module generate command will produce the following output:

Notice: Generating module at /media/sf_learning/learning-nagios
Notice: Populating templates...
Finished; module generated in learning-nagios.
learning-nagios/metadata.json
learning-nagios/Rakefile
learning-nagios/manifests
learning-nagios/manifests/init.pp
learning-nagios/spec
learning-nagios/spec/spec_helper.rb
learning-nagios/spec/classes
learning-nagios/spec/classes/init_spec.rb
learning-nagios/Gemfile
learning-nagios/tests
learning-nagios/tests/init.pp
learning-nagios/README.md

As in Chapter 3, My First Puppet Module, we renamed the webapp module so that Puppet is able to find the module from the module path. We must also rename the Nagios module directory so that the part learning- is not present. Use the following command to rename the module directory learning-nagios to nagios:

# mv learning-nagios nagios

Using the command, ls, you can list files and directories in the current working directory. After renaming the learning-nagios module directory, we'll have the following list of modules:

# ls
apache  concat  nagios  stdlib  webapp

Nagios Server installation is very easy with Puppet as it only requires a couple of resources to be declared in the manifest. To install Nagios Server, we need a manifest file that installs the Nagios Server package and the Apache HTTP Server provided by the Apache module that we used in Chapter 3, My First Puppet Module. The manifest file also must declare a configuration file for the Nagios administration web interface.

We will wrap these resources into the Puppet class called nagios::server and place the class into the file nagios/manifests/server.pp.

Here is the definition for the class nagios::server, which I'll explain more in detail after the snippet:

class nagios::server {
  include apache
  include apache::mod::php
package { ['nagios','nagios-plugins-nrpe']:
    require => Package['httpd'],
    ensure  => installed;
  }
  file fil { '/etc/httpd/conf.d/nagios.conf':
      require => Package['nagios'],
      notify  => Service['httpd'],
      source => "puppet:///modules/nagios/nagios.conf";
  }
  exec { 'set-default-username':
    require => Package['nagios'],
    command => '/bin/echo default_user_name=nagiosadmin >> /etc/nagios/cgi.cfg',
    unless  => '/bin/grep default_user_name=nagiosadmin /etc/nagios/cgi.cfg',
    notify  => Service['nagios'];
  }
  service { 'nagios':
    require => Package['nagios'],
    ensure  => running;
  }
}

To make it easier to explain the preceding code block, I've prefixed lines with line numbers, which you should omit when writing the file server.pp:

You hopefully now have a better understanding of what the nagios::server class does. Now, it's your turn to create the file server.pp. You can create the file inside the puppet-agent-nagios virtual machine using the Nano editor, or you can create it on the host computer using the graphical text editor.

Here is the screen capture from text editor on my computer after I've added the content to server.pp:

Before we can test the nagios::server class, we need to create the configuration source file for nagios.conf and place it inside the files directory within the Nagios module. Because the puppet module generate command doesn't create the files directory automatically, we must create it by hand. You can create the directory in the file manager program on your machine, or alternatively, create the directory on the command line inside the puppet-agent-nagios virtual machine. Here are the commands for creating the files directory on the command line:

# cd /media/sf_learning
# mkdir nagios/files

Configuring the Nagios Server web interface

The module should now have the files directory in place where we can store the nagios.conf configuration file that the nagios::server class references. The following example enables Nagios web interface on Apache HTTP Server.

For clarity, I've prefixed each line with a line number, which you should omit when creating the file nagios/files/nagios.conf:

ScriptAlias /nagios/cgi-bin/ /usr/lib/nagios/cgi-bin/
<Directory /usr/lib/nagios/cgi-bin/>
   Options ExecCGI
   Allow from all
</Directory>
Alias /nagios /usr/share/nagios/html
<Directory /usr/share/nagios/html>
   Allow from all
</Directory>

Although the preceding configuration has very little to do with Puppet, I'd still like to take a moment to explain what these configuration statements do:

• Line 1 creates an alias for script execution on the web server.

• Line 2 begins a directive for directory /usr/lib/nagios/cgi-bin that holds scripts used by Nagios Server web interface.

• Line 3 enables scripts in directory /usr/lib/nagios/cgi-bin to be executable by the web server.

• Line 4 makes the directory unrestricted and accessible from any host.

• Line 5 closes the Directory directive.

• Line 6 creates a web server resource alias /nagios. In practice, this means that when you access the URL http://192.168.56.10/nagios, the web server will provide web documents from directory /usr/share/nagios/html.

• Line 7 begins a configuration block for Directory /usr/share/nagios/html.

• Line 8 configures the directory to be accessible for everyone.

• Line 9 closes the Directory directive.

Now, go ahead and create the file nagios.conf with the preceding content. Once you have added all lines into the file, the contents should look very similar to the following screenshot:

include nagios

include nagios::server

# cd /media/sf_learning
# puppet apply --modulepath=./ nagios/tests/init.pp

# puppet apply --modulepath=./ nagios/tests/init.pp
Notice: Compiled catalog for web.development.vm in environment production in 7.23 seconds
Notice: Finished catalog run in 11.39 seconds

Verifying Nagios Server installation

If Puppet didn't report any errors (errors are highlighted in red) during the Puppet run, you should now have Nagios Server web interface accessible from your web browser.

So that we can work out the address for the web interface, we should check what IP address Nagios Server is using. You can use the ifconfig command to display the IP address information on the host. Running the command without arguments would print out IP address information from all network interfaces, but to query information from particular interface, such as the host-only network interface that uses device ID eth1, we can use the following command:

# ifconfig eth1

The IP address that we are interested in is displayed on line 2 with name inet addr: in the following output:

eth1      Link encap:Ethernet  HWaddr 08:00:27:7A:63:99  
          inet addr:192.168.56.10  Bcast:192.168.56.255  Mask:255.255.255.0

Based on the IP information that I just extracted, I can construct an URL http://192.168.56.10/nagios. When I put this address into the web browser's address bar and hit Enter, the following page will load up:

I hope you managed to get the Nagios Server installed and the web interface running on your machine. Now you can spend some time exploring the Nagios user interface using the links on the left hand side of the view. There is not much to see at this point as we yet haven't configured any resources on the server. There are some resources created by default that you can have a look right away.

If you click the Services link that can be found in the navigation pane on your left you will find a list of checks that are run on the host called localhost. The localhost is the Nagios Server and it has some checks on it that all flag up as CRITICAL.

There is a reason why all checks are in CRITICAL status, and that is because there is no Nagios Client installed on the monitoring server and that's why the Nagios Server is unable to check the state of it.

Next, we'll have a look at how to hook a Nagios Client with the server. Earlier in the chapter, I mentioned the acronym NRPE, which is the name of the package that makes a host a Nagios Client. Our mission now is to install the NRPE package, configure it, and tell Puppet to start the process.

To begin the process, you should create a new file called client.pp and save it into the directory called manifests inside the Nagios module. In the client.pp file, create a class called nagios::client that does the installation, configuration, and service management for us.

Here is how the content of the file should look like:

class nagios::client {
  package { 'nrpe'']:
      ensure => installed;
  }
    package { [
      'nagios-plugins-http', 
      'nagios-plugins-ping', 
      'nagios-plugins-ssh',
      'nagios-plugins-disk',
      'nagios-plugins-users',
      'nagios-plugins-swap',
      'nagios-plugins-procs',
      'nagios-plugins-load',
      ]:
        ensure  => installed,
        require => Package['nrpe'];
    }  
  exec { 'allowed-hosts'
    command => '/bin/sed -i 's/^allowed_hosts=127.0.0.1//g''/etc/nagios/nrpe.cfg",
    onlyif  => '/bin/grep ^allowed_hosts=127.0.0.1 /etc/nagios/nrpe.cfg',
    require => Package['nrpe'],
    notify  => Service['nrpe'];
  }
  service { 'nrpe':
    ensure => running,
    enable => true;
  }
}

Lines 1 to 20 look fairly straight forward but the exec resource on lines 21 to 27 may look a bit messy. Not to worry though, as I'll do my best to explain what each line inside the class is for.

Now it's time to start typing and to create the client.pp under the manifests directory. Here is a screenshot that shows the content of the file client.pp:

Testing the nagios::client class

Note

You will need an Internet connection to test the nagios::client class.

The nagios::client is now ready to be applied. Let's test it first on the puppet-agent-nagios virtual machine before we try to apply it on the web server host.

The easiest way to apply the class is to add an include nagios::client statement into the file tests/init.pp, that currently only includes the nagios::server class, and then apply the file with puppet apply command.

Open the nagios/tests/init.pp in text editor and add include nagios::client statement into the file. Once it's been added, the nagios/tests/init.pp should have the following content:

Once the nagios/tests/init.pp file has been saved, you can apply it with the following commands:

# cd /media/sf_learning
# puppet apply --modulepath=./ nagios/tests/init.pp

If the nagios::client class is applied successfully, Puppet will produce the following report:

Notice: Compiled catalog for web.development.vm in environment production in 6.22 seconds
Notice: /Stage[main]/Nagios::Client/Package[nrpe]/ensure: created
Notice: /Stage[main]/Nagios::Client/Exec[allowed-hosts]/returns: executed successfully
Notice: /Stage[main]/Nagios::Client/Package[nagios-plugins-all]/ensure: created
Notice: /Stage[main]/Nagios::Client/Service[nrpe]/ensure: ensure changed 'stopped' to 'running'
Notice: Finished catalog run in 53.20 seconds

If you now go back to Nagios Server web interface and click the service link again, you should see that the checks on the host localhost are slowly starting to change from state CRITICAL to state OK, as seen in the following screen capture:

Nagios resources such as hosts and checks have built-in resource types in Puppet. Nagios hosts are configured with the resource type called nagios_host and checks are configured with the resource type nagios_service.

Monitoring checks in Nagios terminology are called services.

Creating a Nagios host

Every host that we want to monitor must be configured on the Nagios Server. Nagios host resource requires two pieces of information:

• Name of the host

• IP address of the host

The name can be whatever you like, and it does not have to match the host name of the host. I like descriptive names, so I'll choose the name web-server for the web server host.

The IP address is slightly trickier as we don't know for sure what IP address the web server will get from the DHCP server when it boots up. For now, let's assume that the web server gets the next available IP address from the DHCP server, which is 192.168.56.11. The IP address ending with .10 is allocated to the Nagios Server at the moment. Later in the book we will have a look at how hosts can export resources that enable Puppet to dynamically adjust the IP address when it changes.

So, we have chosen the name web-server and the IP address 192.168.56.11, and we can now create the host with the following resource definition:

  nagios_host { 'web-server':
    host_name   => 'web-server',
    address     => '192.168.56.11',
    use         => 'linux-server',
    target      => '/etc/nagios/conf.d/hosts.cfg',
    notify      => File['/etc/nagios/conf.d'],
    require     => Package['nagios'],
  }

Let's walk through the nagios_host resource definition so that we know what each attribute does:

• Line 1 declares a nagios_host resource called the web-server.

• Line 2 defines the host_name attribute which sets the name for the Nagios host that will appear in the Nagios Server web interface.

• Line 3 defines the IP address that Nagios Server should use when contacting the agent.

• Line 4 specifies a use attribute that acts as a reference to Nagios host template that provides default values for a variety of attributes associated with the host.

• Line 5 defines the target filename where Nagios host resource should be stored in.

  Target is used to override the default location of the Nagios configuration files that Puppet uses. By default Puppet uses directory /etc/nagios to store configuration files in to. The problem is that Nagios Server is not configured to read configurations from this directory. Nagios Server, however, is configured to look for configuration files that use the .cfg file extension within the directory /etc/nagios/conf.d, and that's why we specify the target file /etc/nagios/conf.d/resources.cfg as part of the host resource definition.

• Line 6 creates a notify relationship with a file resource for directory /etc/nagios/conf.d which we will create in a moment.

• Line 7 requires the package nagios that creates the directory /etc/nagios/conf.d, to be installed prior to processing this resource. Puppet cannot create the target file /etc/nagios/conf.d/hosts.cfg if the directory does not exist.

• Line 8 closes the nagios_host resource definition.

Before we add the above nagios_host definition into the manifest file, we should have a look at the nagios_service definition that will go into the same manifest file and create a check for the web server:

  nagios_service { 'HTTP':
    host_name           => 'web-server',
    service_description => 'HTTP',
    check_command       => 'check_http',
    use                 => 'local-service',
    target              => '/etc/nagios/conf.d/services.cfg',
    notify              => File['/etc/nagios/conf.d'],
    require             => Package['nagios'],
  }

• Line 1 declares a nagios_service resource called HTTP.

• Line 2 defines a host_name attribute that associates the check with the web-server host

• Line 3 defines the service name for the check that is displayed in the Nagios Server web interfaces.

• Line 4 specifies the NRPE plug-in name to be used to perform the check. Plug-in check_http is used for checking the availability of web server interfaces.

• Line 5 references the Nagios service template called local-service.

• Line 6 defines the target file where the Nagios service resource is going to be stored in.

• Line 7 creates a notify relationship to file resource /etc/nagios/conf.d which is the directory where this nagios_service resource is stored in.

• Line 8 requires the package nagios, that creates the directory /etc/nagios/conf.d, to be installed prior to processing this resource. Puppet cannot create the target file /etc/nagios/conf.d/hosts.cfg if the directory does not exist.

• Line 9 closes the nagios_service resource definition.

There is one more resource that must accompany the nagios_host and nagios_service resources, and that is the file resource /etc/nagios/conf.d, which was referenced with the notify attribute from both Nagios resources. This resource resets the file ownership information so that the Nagios process is able to read the files stored in the directory /etc/nagios/conf.d. Here is how to define the file resource:

  file { '/etc/nagios/conf.d':
    recurse   => true,
    owner     => 'nagios',
    require   => Package['nagios'],
    notify    => Service['nagios'];
  }

• Line 1 begins the file resource statement for directory /etc/nagios/conf.d.

• Line 2 uses recurse attribute which is specific to directories that we want to manage recursively.

• Line 3 sets the directory owner as user nagios which is the user account that runs the Nagios Server process.

• Line 4 requires the package nagios to be installed prior to processing the file resource. Directory /etc/nagios/conf.d is created by the nagios installer, and that's why the installation must happen before Puppet tries to change the ownership of the directory.

• Line 5 notifies the nagios process to restart in case of any of the files inside the directory had their owner information changed.

• Line 6 closes the file resource definition.

Now, create a file nagios/manifests/resources.pp and add the preceding three resources into the file wrapped inside the class nagios::resources.

The content of file nagios/manifests/resources.pp should look like the following after all resources have been added into it:

Once the file has been created, you should add the include nagios::resources statement into the file nagios/tests/init.pp before you apply the file. Here's how the content of the nagios/tests/init.pp file should look like after modification:

Next, we can apply the class on the Nagios Server with the following two commands:

# cd /media/sf_learning
# puppet apply --modulepath=./ nagios/tests/init.pp

We are almost done with setting up the monitoring for the web server. Nagios Server has been configured to monitor the HTTP interface on the web server but the web server is not running at this point and there is no NRPE agent installed on it.

However, this issue can be easily rectified by including the class nagios::client in the class webapp which is found in the file webapp/manifests/init.pp.

This is how the beginning of the class webapp looks like once I've added the statement include nagios::client into it:

Once the file webapp/manifests/init.pp has been saved, power on the virtual machine puppet-agent in its current state, not from the snapshot, and once virtual machine is powered on, log on to the machine as a user root using the password puppet. Then, apply the webapp/tests/init.pp file with the following commands:

# cd /media/sf_learning
# puppet apply --modulepath=./ webapp/tests/init.pp

Once the webapp class has been applied on the web server node, you can go back to Nagios Server web interface at http://192.168.56.10/nagios. After refreshing the browser window by clicking the services link, a new host called web-server should appear on the list.

Here's a screenshot from the Nagios Server web interface that shows the HTTP check on the host web-server in OK state:

When calling a class without parameters, we will use the include keyword followed by the class name, for example, include apache.

When calling a class with parameters, the include key word is replaced with the key word class and the syntax used with class key word becomes analogous to any other type of Puppet resource. For example, by calling a class bicycle with the parameter wheels and the parameter value 2, we would use the following syntax:

class { 'bicycle':
    wheels => '2',
}

Perhaps you find traditional bicycles a bit dull and unexciting and decide to go for a unicycle; in which case, you call the class with parameter wheels => '1'. Or if you are a family person like I am, and you need more space for kids and groceries, you may prefer a tricycle which you can get with parameter wheels => '3'.

This is just an example of how class parameters can help to have a single implementation of a class that caters for multiple needs.

Now, we know how to call class with parameters, but before we can do this, the class must be written in way that it understands the parameters that we are passing in.

Let's implement the bicycle class and add a simple exec resource into it, which executes a fictional command that orders a bicycle from a nonfictional shop called The School Run Centre (my local bike retailer based in Cambridge, UK):

class bicycle {
  exec { 'bicycle-order':
  command => "/usr/bin/the-school-run-centre order wheels=2"
}

The class bicycle in its current form doesn't yet understand parameters passed into it, and if you're calling the class without parameters, using the include key word, it will always place an order for a traditional two-wheel bike.

To make the bicycle class accept the parameter wheels and to make use of the parameter value in exec resource, we must do the following alterations to the class:

class bicycle ($wheels='2') {
  exec { 'bicycle-order':
    command => “/usr/bin/the-school-run-centre order wheels=$wheels”
  }
}

Let's have a look at the preceding class definition more in detail:

The defined type is referenced with the exactly the same syntax as when referencing Puppet built-in resource types such as the exec, file, or the user resource.

To reference the defined type called bicycle, we'd do it in the following way:

bicycle { 'unicycle':
    wheels => '1';
}

As you can see, the syntax is almost identical to the bicycle class reference. The only difference is that here we don't begin the definition with a class key word because we are not referencing the class resource type. Instead, our resource type is called bicycle, and the name of the resource, here I used unicycle, can be whatever you like as long as the resource name is unique.

The preceding snippet would order a unicycle for you. But what if I also wanted a bicycle, one that has 3 wheels? Well, this could be done with another bicycle resource definition that looks like the following:

bicycle { 'tricycle':
    wheels => '3';
}

Now we have references to two different types of bicycles: one has 1 wheel and another has 3 wheels. That's fine as long as the bicycle resource names are unique. And this criteria is fulfilled as the first bicycle is called unicycle and the other one is called tricycle.

Now, we know how to reference a defined type, but how does it look from the other end of the tunnel? Let's have a look at how the type bicycle can be implemented.

Creating the defined type is quite similar to creating a class. There is one tricky bit to remember though when creating your own types and I've highlighted that part of the code in the following declaration that creates the type bicycle:

define bicycle ($wheels='2') {
  exec { "bicycle-order-
${name}
":
    command => “/usr/bin/the-school-run-centre order wheels=$wheels”
  }
}

The tricky bit that I mentioned is found at the end of the line 2, but I'll run you through the preceding type definition line by line and compare it to the class definition:

I believe that's all we need to know about parameterized classes and defined types before we move on and use them in real-life scenario.

If you have any virtual machines running at the moment, I'd recommend to shut them down now to reduce the memory usage on the host computer. We will begin by creating a new clean virtual machine for the load balancer module development using the following steps:

cd /media/sf_learning
puppet module generate learning-loadbalancer --skip-interview

mv learning-loadbalancer loadbalancer

[root@learn /media/sf_learning]# ls
apache concat loadbalancer nagios stdlib webapp

Installing the load balancer using class parameters

Now that the loadbalancer module has been created, we can start adding deployment logic into the class loadbalancer, which you can find inside the file loadbalancer/manifests/init.pp:

Note

To make it easier to keep track of the lines in the file init.pp, I've removed all comment lines (lines beginning with #) and enabled line numbers in Gedit text editor.

Open the file init.pp in text editor and add the following content to it:

It only takes as little as 11 lines of Puppet configuration to turn the load balancer node into a proxy server.

Let's look at the preceding loadbalancer class more in detail:

1. Line 1 begins the loadbalancer class definition.

2. Line 2 calls the apache class without passing any parameters to it. With this statement, we express that we want Puppet to do the basic installation of Apache HTTP Server.

3. Line 3 makes a reference to defined type called vhost that is provided by the class apache. To make a reference to a type vhost that is declared inside a class apache, Puppet uses notation apache::vhost followed by the name of the type, in this case named as loadbalancer.

4. Line 4 sets a parameter ip that we pass into the type vhost. The value of the parameter defines the address that the load balancer node is listening on for incoming HTTP requests. For this, we use fact called ipaddress_eth1, which is a reference to the IP address of the Host-Only Network interface on the load balancer virtual machine. In my environment load balancer, the node uses IP address 192.168.56.10 for the Host-Only Network interface.

5. Line 5 defines parameter docroot which is one of the required parameters by the apache::vhost. The value of the parameter is not important for the functionality of the load balancer, we just must provide the parameter with some value when calling the type apache::vhost.

6. Line 6 defines a parameter proxy_pass, which is used to create a route via load balancer to the web server and the monitoring server. The value of the parameter begins with [ sign, which in the Puppet language means that the parameter value is a list of items. Lines 7 and 8 specifies the items on the list.

7. Line 7 defines the first item on the list, which is a dictionary type of data where the data is presented in the format of key—value pairs. With this configuration, we tell the load balancer to forward all requests for the resource /nagios on the load balancer to the monitoring server URL http://192.168.56.11/.

   The first key is called path and the value is /nagios.

   The second key is called url and the value of the url is http://192.168.56.11/nagios.

8. Line 8 specifies another set of key—value pairs for the Web Server interface at http://192.168.56.12/.

   The dictionary entry path has a value /webapp.

   The second dictionary entry url has the value http://192.168.56.12.

9. Line 9 ends the list with a closing block bracket ].

10. Line 10 closes the block apache::vhost.

11. Line 11 ends the loadbalancer class definition.

So, now you should have rough idea of what is involved in the initial deployment of the load balancer node. Next, we can try this in action.

cd /media/sf_learning
puppet apply --modulepath=./  loadbalancer/tests/init.pp

 Notice: /Stage[main]/Apache::Service/Service[httpd]/ensure: ensure changed 'stopped' to 'running'
Notice: Finished catalog run in 17.36 seconds

Verifying the load balancer deployment

Analyzing the Puppet report is one way to know whether the deployment was successful, but what is more important is to know whether the service works in real life. To verify the load balancer functionality, we will spin up the web and monitoring servers and see whether we can access both web interfaces via load balancer node's IP address.

But before we do this, let's do a quick test that shows whether the load balancer node is trying to route request to the backend services:

Open the web browser on the host computer. Type in the URL http://192.168.56.10/webapp and hit Enter.

If you get immediate response of 404 Not Found displayed at the top of the browser window, then that's a sign that deployment was not successful. If this is the case, I'd advise you to review the content of the class loadbalancer and check the syntax of each line in the class.

However, if the request takes a few seconds to process and you then receive message 503 Service Temporarily Unavailable, this means that the load balancer attempted to route the request to the web server interface but as the node is not yet running, the request timed out. The following screenshot shows example 503 response:

Testing end-to-end functionality

Next, we will verify the functionality end-to-end and launch.

Depending on how much free memory you have on the host computer, you may have to reduce the memory allocation in the virtual machine settings, which you can access by pressing Ctrl + S and adjusting the Base Memory value under the System category. You can reduce the amount of memory down to 256 MB and virtual machines should still boot up fine.

Once the monitoring server comes up, you should see the IP address information displayed on the log in prompt. The IP information shows that the monitoring server has got IP address 192.168.56.11 from the DHCP server as shown in the following screenshot:

Similarly, in the login prompt on the web server, the IP address information shows that the web server is using the IP address 192.168.56.12:

If for any reason web and monitoring server nodes came up with different IP addresses, you should reconfigure the load balancer node by adjusting the lines 7 and 8 in the file loadbalancer/manifests/init.pp, and point the /nagios and /webapp resources to appropriate IP addresses. In case you made changes to the loadbalancer class, you should apply your changes with the command puppet apply --modulepath=./ loadbalancer/tests/init.pp on the load balancer node.

Before we do our end-to-end tests, let's do a Puppet run on the other two nodes to make sure that all services are started properly.

Log on to the web server node using the username root and password puppet. Then, run the following two commands:

cd /media/sf_learning
puppet apply --modulepath=./ webapp/tests/init.pp

Then, log on to the monitoring server node using the same login details and apply the nagios::server class with the following set of commands:

cd /media/sf_learning
puppet apply --modulepath=./ nagios/tests/init.pp

Now that all three virtual machines have been updated by Puppet, it is time to check whether we can access monitoring and web server interfaces via the load balancer node. In the class loadbalancer, we configured resource /nagios for the monitoring server and the resource /webapp for the web server.

Open the web browser on the host computer type in the URL http://192.168.56.10/nagios. This should give you the nagios server front-page:

Then, try the URL http://192.168.56.10/webapp and see whether the web server landing page loads up. The following screenshot demonstrates the proxy functionality quite nicely. If you look at the page content, it shows the web server's IP address 192.168.56.12, but the IP address in the address field on the web browser points to the IP address 192.168.56.10:

That's it. I hope your end-to-end tests were as successful as mine. Next, we will focus on how to make the loadbalancer class more configurable by turning it into the parameterized class.

To enable load balancing in the class loadbalancer, we will add couple of new variables in to the class that we can reference from the class loadbalancer::create that we will soon create.

Here's how the latest version of the class loadbalancer in file loadbalancer/manifests/init.pp looks after variables and variable references have been put in place:

Let's walk through the lines that were added or changed since the previous version of the class:

Let's not try to apply the loadbalancer class yet as it would produce an error during Puppet run due to missing the class loadbalancer::create that we need to create now.

In the text editor, create a new file loadbalancer/manifests/create.pp. The file defines the class loadbalancer::create that creates the load balancer object and adds two member nodes into it.

Here's the content for the file loadbalancer/manifests/create.pp and I'll explain its functionality more in detail after the screenshot:

Here's the break down of the file loadbalancer/manifests/create.pp:

We now have all the assets ready for load balancer deployment. Make sure that the class loadbalancer::create is saved as create.pp in the directory loadbalancer/manifests so that Puppet is able to locate the class. Next, we will apply the class loadbalancer and see how load balancing works in practice.

Let's apply the class loadbalancer first to see that we got the syntax right before we join the second web server node in the cluster.

On the load balancer node, run the following two commands:

cd /media/sf_learning/
puppet apply --modulepath=./ loadbalancer/tests/init.pp

This time the puppet apply command will produce couple of warning messages, which you can safely ignore.

The first message Warning: You cannot collect exported resources without storeconfigs being set is related to exported resource collection, which we haven't yet enabled in our environment.

The second message Critical: Scope(Concat::Fragment[01-webapp-proxyset]): No content, source or symlink specified is produced by the type concat::fragmentWhat. This message doesn't have functional impact on the deployment so you can safely ignore this.

What you should ensure is that Puppet bounces the service HTTPD at the end of the Puppet run, which is an indication that deployment was successful. On successful run, the Puppet report will include the following two lines at the end:

Notice: /Stage[main]/Apache::Service/Service[httpd]: Triggered 'refresh' from 1 events
Notice: Finished catalog run in 13.42 seconds

Before we launch the second web server node, let's do a quick test to ensure that the first web server is accessible via load balancer in the URL http://192.168.56.10/webapp.

I've checked it and it works for me. Does it also work in your environment?

The Puppet binary running inside the Apache HTTP Server is responsible for distribution of Puppet resources to the clients. But what about the other services that I briefly mentioned in the previous paragraph? Well, there is a separate component for each of the services, which we should discuss now in few words.

The module bootstrap will be created inside the shared folders, which is mounted as /media/sf_learning across all nodes. You can use any of the existing virtual machines to create the module, but I've decided that I will create a new module on the Puppet Master node, which is called learn_puppet_centos-6.5 in VirtualBox Manager.

First, launch the virtual machine, and then run the following commands to create the bootstrap module:

The first task for us is to change the network configuration on Puppet Master node in a way that it uses static IP address instead of dynamic IP address. When the IP address is fixed, it is easier for us to configure Puppet Agent nodes to connect to known IP address.

Resource processing order can be defined within the resource with require and notify attributes. If you have many resources that requires strict ordering, it may be difficult to manage resources with these attributes.

Puppet provides an alternative syntax, which I call the arrow notation, to order resources outside the resource definition block.

Let's take a look at an example where we have two Puppet resources: file { 'myfile':} and service { 'myservice': }.

If we want the file resource to be processed before the service resource, we need to traditionally declare the require attribute inside the service resource in the following way:

file { 'myfile': }
service { 'myservice': 
  require => File['myfile'];
}

With the arrow notation, we can express the same ordering by removing the require attribute and adding an arrow in between resources:

file { 'myfile': } -> service { 'myservice': }

Another form of the preceding arrow notation is an arrow with a tilde character ~>. The arrow using tilde is an analog to the notify attribute. If we would like the service { 'myservice': } to be restarted only in the case that the resource file { 'myfile': } changes, we can express it with the notify attribute in the following way:

file { 'myfile': 
  notify => Service['myservice']
}
service { 'myservice': }

The same results can be achieved with the tilde arrow notation like this:

file { 'myfile': }~> service { 'myservice': }

Next, we will try this in practice by ordering resources in the class bootstrap::master using the arrow notation.

All Puppet Master-related configuration can be contained within a subclass called bootstrap::master, which we will create in the file bootstrap/manifests/master.pp.

Now, create a new file in the text editor with the following content and save it as master.pp under the manifests directory.

The preceding master.pp file creates a class bootstrap::master that contains a bunch of Puppet resources. Let's look at them more in detail:

Once you have added all these lines into the file, save it as master.pp inside the directory bootstrap/manifests directory.

Next, we'll have to create the template file that the bootstrap::master class is referencing (on line 24 in master.pp). In this template file, we will use the so-called out-of-scope variable reference, which in Puppet terminology means a variable that is declared in an external Puppet class.

You can find the out-of-scope variable references on lines 4 and 5 in the following screenshot:

Let's have a look at the content of the template file bootstrap/templates/ifcfg-eth1.erb:

Now, go ahead and create new file in text editor with the the preceding content and save the file as ifcfg-eth1.erb under the directory bootstrap/templates/. Then, we can move on to adjust the main bootstrap class before we apply the class on the Puppet Master node.

Let's have a look at how we can add more logic into Puppet manifests by introducing conditional statements. We just created the class bootstrap::master, which we will soon apply on the Puppet Master node. A little later, we'll create a class called bootstrap::agent, which will be applied on the Puppet Agent nodes. This means that we'll have two classes for different Puppet "roles" to choose from.

In Chapter 4, Monitoring Your Web Server, we had a similar decision to make when we applied the nagios::server class on the Nagios Server node and the nagios::client class on the Nagios Client nodes. The decision on which class to apply on each node was done by us at the time when we ran the puppet apply command.

With conditional statements, we can hand over this decision-making task to Puppet and let Puppet to evaluate which class to apply on each node.

The two most commonly used types of conditional statements are the if and case statements. Both of these statements are used to evaluate whether the given condition is true or false. Based on the outcome of the evaluation, Puppet selects the block of code to execute.

The if statement

If you have ever done programming or scripting, you are probably familiar with the if statement; this is used to evaluate whether given the condition is true or false. When the if condition is true, the program executes a block of code. Otherwise, this block of code is ignored.

The syntax of the if statement in Puppet is as follows:

if condition1 {
  statement1
}

The if statement is often combined with the else statement that defines what to do when condition is false or not true. Or, to evaluate multiple conditions, we can add an elsif block in between else and if blocks in the following way:

if condition1 {
  statement1
}
elsif condition2 {
  statement2
}
else {
  statement3
}

Let's see how we can utilize the if statement in the bootstrap module to conditionally apply a class based on the value of variable role. Open the file bootstrap/manifests/init.pp in the text editor and add the following content to it:

The file init.pp defines a class bootstrap that has three input parameters and conditional statements for checking the role of the node.

Let's look at the input parameters first on lines 2, 3, and 4.

• Line 2: The $puppetmaster_ip parameter defines the IP address of the Puppet Master node. This parameter is referenced on line 4 in the template file bootstrap/templates/ifcfg-eth1.erb, which we created a moment ago. If we don't provide this parameter at the time when applying, the class Puppet Master will be configured with the IP address 192.168.56.2.

• Line 3: The $puppetmaster_netmask parameter defines the subnet mask for Puppet Master node. This variable is referenced in the template file ifcfg-eth1.erb on line 5 and the default subnet mask value 255.255.255.0.

• Line 4: The $role variable defines the role of the Puppet node. The default value is unquoted false, which means that the value is a so-called Boolean value. The variable $role is referenced by the if statement on line 6, where Puppet evaluates whether the role is master.

The conditional statement that begins on line 6 and ends on line 11 has the following logic:

• Line 6: The if $role == 'master' statement checks whether the input parameter value is the master. This condition is true only if we provide this parameter at the time when the class is applied, if we don't, it defaults to false and the statement inside the if block is ignored.

  We can provide the variable value by applying the class using syntax class { 'bootstrap': role => 'master' }.

• Line 7: If the condition on line 6 is true, then Puppet includes the class bootstrap::master.

• Line 8: The if statement is closed with a closing curly bracket.

• Line 9: The else block (lines 9-11) is processed only if the if statement is false. In practice, this means that if we don't set the value master for the variable $role, the statement class { 'bootstrap::agent': role => $role } inside the else block is processed.

  This statement passes the parameter role into the class bootstrap::agent, and we can use the role parameter to configure the certname of the Puppet Agent.

• Line 12 closes the class bootstrap.

Next, we should try to apply the class on the Puppet Master node. Before we apply the class, I'd recommend that you revert Puppet Master virtual machine state to snapshot host-only-networking.

Here are the steps required to restore the snapshot:

1. Select the virtual machine learn_puppet_centos-6.5 and click on the Snapshots button.

2. Select the snapshot host-only-networking and click on the Restore Snapshot button.

3. Uncheck the box that says Create a snapshot of the current machine state, and then click on the Restore button:

   

We have already done node classification from the command line when applying Puppet manifests. In Puppet terminology, node classification means to associate a set of Puppet classes with the Puppet Agent node. Once Puppet Agent is linked with the Puppet Master, we no longer run Puppet from the command line, instead, we run Puppet Agent in so-called daemon mode. In the daemon mode, Puppet Agent connects to Puppet Master periodically to check whether Puppet Master has new manifests that should be applied on the agent.

The site.pp file defines what classes should be applied on the Puppet Agent node when it connects to the Puppet Master. This file is stored under the directory called manifests that lives in the root of the environment directory on the Puppet Master. We defined an environment directory called development inside the class bootstrap::master (line 7 to 9) that we created in the Creating class bootstrap::master section. In the same class, we also created the directory manifests as a link (line 12 to 15) and pointed the link to the directory /media/sf_learning/manifests in the shared folders, which doesn't exist yet.

Here you can find the example site.pp file that contains node classifications for three nodes.

Node classification begins with the keyword node followed by the certname of the node, which is wrapped in single quotation characters. After certname, there comes an opening curly brace { and a list of classes that should be applied on the Puppet Agent. Node classification ends with the closing curly brace character }, as shown in the following screenshot:

Before you create the site.pp file in text editor, create a new directory called manifests under the Shared Folders directory (for example /home/jussi/learning). Then, create site.pp with the preceding contents and save it inside the directory manifests.

Now, we are ready to apply the bootstrap class with role master:

Once Puppet has successfully applied the bootstrap class on the Puppet Master, we must reboot the host so that Puppet Master services register the new IP address 192.168.56.2 and the new environment called development.
You can reboot the system by running the command reboot on the command line. Alternatively, you can stop and start the virtual machine from the VirtualBox Manager.

Puppet Enterprise Console is a web application that runs on the Puppet Master node. We will have a look at it more in detail in Chapter 9, The Puppet Enterprise Console, but in this chapter, we will use it to create a node group and sign certificates for Puppet Agent nodes.

Bypassing the certificate warning message

After rebooting the Puppet Master, you should be able to access Puppet Enterprise Console from https://192.168.56.2. When you open the Enterprise Console for the first time, you may see an SSL certificate warning message displayed in the browser. The warning message is caused by a self-signed SSL certificate used by Puppet Enterprise Console. Here is the certificate warning message that appears when you're using Chromium web browser. To proceed to the login screen, we need to first click the Advanced link, and then click the link that says Proceed to 192.168.56.2(unsafe):

Once you have got past the certificate warning messages, you should see a login form in front of you. You can log on to the Puppet Enterprise Console using the username admin and password learningpuppet.

# service pe-console-services restart
# service pe-httpd restart

Puppet node group is a method to group Puppet Agents and link the group with an environment. The class bootstrap::master (lines 7 to 9 in file bootstrap/manifests/master.pp) that we applied on the Puppet Master created an environment called development. Now, we need to add some Puppet Agents into this environment, and that can be done with a node group that we'll create next.

We now have a node group called development that doesn't contain any nodes yet. We can leave the Puppet Master node running while we bootstrap the Web Server node. Once Puppet Agent node is bootstrapped, we will return to Puppet Enterprise Console to sign the Agent's certificate and add it to the node group.

The else block inside the class bootstrap (line 9 to 11) references the class bootstrap::agent, which we haven't yet created. The bootstrap process on the Agent should perform the following two tasks:

The bootstrap::agent class definition shown in the following screenshot will perform these tasks for us:

The class bootstrap::agent contains two Puppet resources and an if statement. Let's inspect the class line by line:

That's everything we need to bootstrap Puppet Agent nodes. Now, it's your turn to create the file bootstrap/manifests/agent.pp with the preceding content. Once you are done with it, it's time to apply it on the Web Server node.

We will first apply the class bootstrap::agent on the Web Server node. The name of the Web Server virtual machine is puppet-agent and it contains a snapshot called puppet-agent-web, which we should restore prior to launching the virtual machine.

Here are the steps to bootstrap the Web Server node:

Please note that we don't call the class bootstrap::agent directly, but instead we use the bootstrap class as the entry point. If we called bootstrap::agent directly, the out-of-scope variable ${bootstrap::puppetmaster_ip} would not be visible to the bootstrap::agent class that references it.

Initiate a connection with the Puppet Master:

# puppet agent -t

When Puppet runs properly and you've run the puppet agent -t command, the following activity is printed on the screen:

The command puppet agent -t that we executed as the last step of the bootstrapping process initiates a connection with the Puppet Master. This command creates a new certificate signing request, which the Agent sends to the Puppet Master. You will also see a message Exiting: no certificate found and waitforcert is disabled, which we will rectify by signing the Puppet Agent's certificate on the Puppet Master.

The command puppet agent -t that we just ran on the Puppet Agent node resulted in a certificate signing request the to the Puppet Master. Now, we can sign the Puppet Agent certificate on the Puppet Enterprise Console by following these steps:

Next, we'll add the node web.development.vm into the development node group.

Now that the certificate has been signed, we should add the Web Server node into the node group called development. As mentioned earlier, node group is a collection of nodes and each node group has an environment, which is also called development. Once the node has been added to the node group, we can rerun the command puppet agent -t on the node web.development.vm to trigger deployment.

Here are the steps on how to add the node web.development.vm to the node group development:

We have now signed the Puppet Agent certificate and added the node web.development.vm into the node group called development. Now, we can deploy the node using the command puppet agent -t. The option -t stands for test and it enables the Puppet Agent process to run on the foreground and to report Puppet Agent activity on screen.

Here are the steps the Puppet in the agent mode:

During the Puppet Agent run, you will see the following messages, which you can safely ignore:

Warning: Local environment: "production" doesn't match server specified node environment "development", switching agent to "development".

This message means that the default Puppet environment production is overridden by the environment development that we configured on the Puppet Master.

Warning: The package type's allow_virtual parameter will be changing its default value from false to true in a future release. If you do not want to allow virtual packages, please explicitly set allow_virtual to false.
   (at /opt/puppet/lib/ruby/site_ruby/1.9.1/puppet/type/package.rb:430:in 'block (3 levels) in <module:Puppet>')

This warning comes from the puppetlabs-apache module and it can be safely ignored.

When Puppet run finishes successfully, you should see the following lines at the end of the Puppet run:

Notice: /Stage[main]/Apache::Service/Service[httpd]/ensure: ensure changed 'stopped' to 'running'
Finished catalog run in 96.83 seconds
Info: /Stage[main]/Apache::Service/Service[httpd]: Unscheduling refresh on Service[httpd]

The best way to verify whether the Web Server deployment was successful is to open the Web Server URL http://192.168.56.11. Please note that Web Server IP address may be different from 192.168.56.11 in your environment. You can check the Web Server's IP address with commands hostname -I and ifconfig eth1.

Congratulations for successfully connecting your first node with the Puppet Master. Now, we should repeat the bootstrap process on the load balancer and the Nagios nodes.

I feel that I don't have to describe this process in detail as it is identical to bootstrapping process on the Web Server node. Just as a reminder: here is an overview of the process of bootstrapping the load balancer and Nagios Server nodes:

One of the services that PuppetDB provides is called exported resources, also known as storeconfigs. Exported resources enable the Puppet Agent nodes to exchange resources between them by exporting a resource on one node and importing the resource to another node. We can export any type of a Puppet resource, including the custom types. In this chapter, we will learn how to export and import the built-in resource types, nagios_host and nagios_service.

Before we dive into this, let's briefly take a look at the benefits of using exported resources.

Exported resources are useful when configuring a cluster of nodes that share services between them, and the nodes need to be able to discover services from each other automatically. If the formation of the cluster is static (a fixed number of nodes in the cluster), its configuration can be managed without exported resources. However, clusters these days are not static but dynamic. Modern clusters are scaled up and down according to demand. To manage a configuration that can adjust to changes as they happen, we need tools such as exported resources that dynamically adjust the configuration of the cluster.

To demonstrate how exported resources enable you to turn a static configuration into a dynamic one, we will revisit the nagios module, which we created in Chapter 4, Monitoring Your Web Server, and modify it slightly.

Before we add logic to export and import resources, let's take a look at the nagios::resources class in its current form:

Lines 2 to 9 include the nagios_host definition of our web server. The nagios_service resource on lines 10 to 18 defines the HTTP monitoring check, which is associated with the web server node.

This configuration works fine as long as the web server node uses the IP address 192.168.56.11 (set by the address attribute on line 4). When the IP address of the web server changes, or we add another web server for monitoring, we are required to manually update the manifest to reflect the changes in the cluster.

A better way to do this is to make the nagios_host and nagios_service resources as exported resources, and let the nagios server node import them. This way, the resources get automatically updated when the IP address changes or when we add more nodes to the cluster, and we don't have to manually update the manifest every time the change happens.

To achieve this, we are going to move the nagios_host and nagios_service resources to a new class called nagios::check_http, and make the resources exported.

Exporting resources

Let's create a new file called check_http.pp under the nagios/manifests directory, and declare a nagios::check_http class with the following content:

Note

You can cut lines 2 to 18 from the nagios::resources class, paste them into the nagios::check_http class, and remove the target, notify, and require attributes.

The nagios::check_http class declares the nagios_host and nagios_service resources that are exported. To export a resource, we prefix the type with a double at sign notation, as in @@nagios_host and @@nagios_service.

Let's take a look at the nagios::check_http class more in detail:

• Line 1 begins with the nagios::check_http class definition.

• Line 2 declares an exported resource @@nagios_host.

  The name of the resource is a reference to facter ipaddress_eth1. The reason why we use facter as the exported resource name is because the name of the resource must be unique for every resource that is exported. If we use a fixed string, for example, web-server, for the resource name, this would cause a Duplicate declaration error when the resources are collected from more than one node. Using facter ipaddress_eth1 as the resource name, we can be sure that the resource name is unique when multiple nodes export the same resource.

• Lines 3 and 4 set the host_name and address attribute values based on facter ipaddress_eth1.

• Line 5 associates nagios_host with the host template called linux-server.

• Line 6 introduces a tag attribute, which is often used with exported resources.

  We tag the nagios_host resource with the name of the Puppet environment (the ${::environment} variable reference) of the node that exports the resource. When the resource is imported, which we will do on the Nagios Server host shortly, we will use the tag as a filter to only collect the resources that are exported on hosts that belong to a specific environment.

• Line 7 closes the nagios_host resource definition.

• Line 8 declares another exported resource called nagios_service, which creates a Nagios HTTP check.

  As every Puppet resource name must be unique, we construct a unique name by joining the value of facter ipaddress_eth1 and a string _HTTP. By creating a unique name this way, we can associate this check with multiple hosts and not have to worry about the resource name clashes.

• Line 9 associates the check with the nagios_host resource.

• Lines 10 to 12 define attributes that must be supplied with the nagios_command resource.

  Refer to Chapter 4, Monitoring Your Web Server, for a description of the attributes.

• Line 13 tags the nagios_service resources with the name of the environment.

• Line 14 closes the nagios_service definition.

• Line 15 closes the nagios::check_http class.

Once the nagios::check_http class has been created, save the file as check_http.pp under the manifests directory.

Then we need to associate the class with the web server node. You can choose whether you want to do it by adding an include nagios::check_http statement to the webapp class in the learning/webapp/manifests/init.pp file or by associating the nagios::check_http class via the node definition file learning/manifests/site.pp.

I would prefer to include the class in the node definition file learning/manifests/site.pp. This is how the definition of the web.learning.vm node will look like after I've associated the nagios::check_http class with the node on line 4:

Importing resources

Before we can see how exported resources work, we must configure a resource collector that imports the nagios_host and nagios_service resources from PuppetDB to the nagios server host. Importing resources is done with the Resource_type <<| tag == search_expression |>> syntax.

For example, to collect all the nagios_host resources that belong to our environment, we do it with the following statement:

Nagios_host <<| tag == $::environment |>>

The statement begins with a reference to the type of the resource we wish to collect. When referencing a Puppet resource, the resource type must begin with a capital letter (Nagios_host instead of nagios_host). After the resource type declaration, we have the so-called spaceship notation <<| |>> with an optional tag parameter inside it. Although the tag parameter is optional, I recommend that you use it; otherwise, the collection is done for every nagios_host resource that is stored in PuppetDB.

We can collect the exported nagios_host and nagios_service resources by altering the nagios::resources class in the file resources.pp file, as shown in the following screenshot:

The nagios::resources class contains two file resources and a resource for importing the nagios_host and nagios_service resources.

File resources (on lines 2 to 9) create the so-called symbolic links that are needed when purging Nagios resources with Puppet. We will cover Nagios resource purging in detail later in this chapter, but here is some background on why these symbolic links are needed.

By default, Puppet stores the nagios_host resources in the /etc/nagios/nagios_host.cfg configuration file and the nagios_service resources in the /etc/nagios/nagios_service.cfg file. The issue with these file locations is that the nagios server does not process these files unless it is explicitly told to do so in the main nagios server configuration file, /etc/nagios/nagios.cfg. Symbolic links are created to workaround this problem.

By default, the nagios server processes all the files, including symbolic links, that have the .cfg file extension in the /etc/nagios/conf.d directory. To make Puppet play nicely with the default nagios server configuration, we create two links in the /etc/nagios/conf.d directory, which points to the /etc/nagios/nagios_host.cfg and /etc/nagios/nagios_service.cfg files.

Once the links are created, we declare the resource collection for the nagios_host and nagios_service resources.

Resource definitions for nagios_host and nagios_service that were previously present in this class have been replaced by the resource collectors for Nagios_host and Nagios_service.

There are two import statements:

• Lines 10 to 13 collect all the Nagios_host resources that are tagged with the name of our environment:

  • Using the require attribute, we tell Puppet that the nagios package must be installed prior to collecting the Nagios_host resources

  • The notify attribute sends a restart signal to the nagios service after the collection is finished

• Lines 14 to 17 repeat the same steps as lines 10 to 13 but for the Nagios_service resource

Now that we have created the nagios::check_http class for exporting resources and a nagios::resources class that collects resources, we can test how the resource collection works in practice.

Before we apply manifests, let's move the /etc/nagios/conf.d/hosts.cfg and /etc/nagios/conf.d/services.cfg files to the /etc/nagios/ directory and rename them. This can be done by running the following two commands on the command line on the nagios server:

mv /etc/nagios/conf.d/hosts.cfg /etc/nagios/nagios_host.cfg
mv /etc/nagios/conf.d/services.cfg /etc/nagios/nagios_service.cfg

Here is a screenshot that shows you the output of the mv commands followed by the ls command to list the files in a new location:

For testing, we need the following three virtual machines running:

When working with exported resources, it is important to know in which order to run Puppet on nodes. For the nagios server node to successfully import resources from the web server node, the Puppet run that exports the resources must happen before the run that imports resources. As resources are exported on the web server node, we must run Puppet on it before we run Puppet on the nagios server node that imports resources.

Run the puppet agent -t command on nodes in the following sequence:

The Puppet run on the web server node doesn't show any nagios-related activity in the Puppet report but, when Puppet is run on the nagios server node, you will see the new Nagios_host and Nagios_service resources being imported on the server.

Here is a screenshot of the Puppet run that shows the resources that are imported on the nagios server:

After the Puppet run is complete on the nagios server, open the nagios web interface at http://192.168.56.12/nagios. Click on the Services link in the navigation pane, which is on the left-hand side of the page, to list all the hosts, and check whether they are currently configured on the nagios server as shown in the following screenshot:.

The list should contain the following three hosts:

As you probably noticed that the Services view in the nagios web interface contains two nagios host resources for the web server. There is a resource for the host 192.168.56.11 that we created on the web server using exported resources a moment ago. There is also a resource for the nagios host called web-server, which was created in Chapter 4, Monitoring Your Web Server. Although we just removed the nagios_host record from the nagios::resources class and reran Puppet, the resource didn't get removed. Why is that?

The reason for this is because Puppet does not automatically remove resources when resources are removed from the manifest file. Instead, we have to explicitly tell Puppet to purge resources of a certain type that are no longer present in the manifests. Puppet has a special resource type called resources that can be used to purge resources that are no longer needed. To purge old resources using type resources, we can use the following syntax:

resources { 'type_of_resource':
  purge => true;
}

To purge the old nagios_host and nagios_service resources from the nagios server, we will create a new class that contains all the purging-related logic, and associate this class with the nagios server.

Let's create a new class called nagios::purge with the following content:

On lines 2 to 4, we declare a resource called resources that is applied to the nagios_host and nagios_service resource types. The resource contains a purge => true attribute that tells Puppet to delete all the nagios_host and nagios_service resources that have been removed from the manifests.

Once Puppet has purged resources, we tell it to reload the nagios process using the exec resource type that is defined on lines 7 to 10.

The notify arrow notation (tilde + greater-than) on line 5 means that only if the nagios_host and/or nagios_service resources are purged, then the notify signal is sent to the exec resource. Note the refreshonly => true attribute on line 8. This attribute is specific to the exec resource, and it means that the resource will only be processed if the notify signal is received.

In other words, the reload-nagios exec resource will only run the /etc/init.d/nagios reload command, in case the nagios_host or nagios_service resources were purged.

Once you have created the nagios::purge class, save the file as nagios/manifests/purge.pp.

We are now ready to try resource purging on the nagios server. Before we do this, let's add the nagios::purge class to the catalog of the nagios server.

Open the node classification file, manifests/site.pp, and add an include nagios::purge statement to the nagios.development.vm node definition block, as shown on line 14 in the following screenshot:

Once you've saved the file, go to the nagios server console and run the puppet agent -t command. The following output confirms that the nagios_host and nagios_service resources were removed successfully, and the nagios server process got reloaded:

Now go to the nagios web interface at http://192.168.56.12/nagios, and refresh the page by clicking on the Services link in the navigation pane on the left-hand side of the page. Did the web server nagios host disappear from the list?

You should now have a fairly good understanding of how exported resources can be used to exchange Puppet resources between nodes. Next, we will take a look at how to query the node information from PuppetDB.

Where exported resources are used to exchange Puppet resources between nodes, the PuppetDB query is used to query information about the Puppet agent nodes.

For example, if we wish to find out the IP address that belongs to the nagios server node, we can make a query that asks for the value of the ipaddress_eth1 facter on the node that has the nagios::server class in its catalog.

Or if we want to know what certnames belong to Puppet agents in our environment, we can make a query that returns a list of certnames that belong to an environment called development.

There are a a couple of aspects to the PuppetDB query that some users may find a bit off-putting.

Firstly, the PuppetDB query syntax is quite complex, and the PuppetDB documentation available online (http://docs.puppetlabs.com/puppetdb/) signally fails to make it easier to learn. For example, to query the value of the ipaddress_eth1 fact of the nagios.development.vm node, we use the following command on the Puppet Master node:

curl -G http://localhost:8080/v3/facts/ipaddress_eth1 --data-urlencode 'query=["=", "certname", "nagios.development.vm"]'

Let's take a look at this command more in detail:

In English, this query can be described as "Please give me the value of fact ipaddress_eth1 that belongs to the Puppet agent that has the certname value nagios.development.vm".

The query response is the following set of key-value pairs.

[ {
  "value" : "192.168.56.12",
  "name" : "ipaddress_eth1",
  "certname" : "nagios.development.vm"
} ]

With this query, we learned that the node with certname nagios.development.vm has an ipaddress_eth1 facter with value 192.168.56.12. That's nice, but how can the query results be integrated with the Puppet deployment when the only documented interface is from the command line using Curl?

This brings us to the second issue with PuppetDB. There is no built-in interface to the PuppetDB query from the Puppet manifest, which is similar to what we have for exported resources, that uses the @@ notation to export resources and the <<||>> notation to import resources in the Puppet manifest.

Don't give up hope yet as there is a simple solution that provides a remedy for both of the previously mentioned issues. This is the Puppet community add-on module, which we will take a look at next.

The Puppetdbquery module, created by Mr. Erik Dalén (documentation is available at https://github.com/dalen/puppet-puppetdbquery), is a handy tool that provides a simple command-line interface to PuppetDB as well as the functions to access PuppetDB from the manifests.

The Puppetdbquery module is available in Puppet Forge, and it can be installed with the following command:

puppet module install dalen-puppetdbquery --modulepath=/media/sf_learning/

The following screenshot shows the output of the preceding command:

We have installed the dalen-puppetdbquery module in the /media/sf_learning/puppetdbquery directory. Before we can start experimenting with it, we need to add the module to the command-line environment on Puppet Master.

We can achieve this by adding the /media/sf_learning/puppetdbquery/lib path to the RUBYLIB environment variable using the following command:

export RUBYLIB=/media/sf_learning/puppetdbquery/lib

To make this change permanent, we don't have to manually run the command every time we log on to the Puppet Master. We can configure the bootstrap::master class to create an /etc/profile.d/puppetdb.sh file, which contains the preceding command.

Open the /media/sf_learning/bootstrap/manifests/master.pp file and add lines 30 to 33, as shown in the following screenshot:

Once you have saved the file, reapply the bootstrap::master class on the Puppet Master node using the following two commands:

cd /media/sf_learning
puppet apply --modulepath=./ -e "class { 'bootstrap': role => 'master'}"

The puppet apply command produces the following report, confirming that the /etc/profile.d/puppetdb.sh file was created:

To make the change effective, you must log out and log in to the Puppet Master node. Once you are logged in again, you can verify that the puppetdbquery module is available by running the puppet help query command, which displays the help menu, as shown in the following screenshot:

Now we are ready to start experimenting with the puppet query command. As seen in the preceding screenshot, puppet query has three actions: events, facts, and nodes.

Querying certname with action nodes

Let's first try the nodes action and make a PuppetDB query that returns the certname of the node that includes the nagios::server class in its catalog. We can do the query using the following command:

puppet query nodes 'Class["nagios::server"]'

A query returns the string nagios.development.vm, which is the certname of the nagios server node.

To expand the scope of the query, we need to also include the certname of the node that has the webapp class in its catalog. We can do this using the or operator to join two search patterns:

puppet query nodes 'Class["nagios::server"] or Class["webapp"]'

In addition to the string nagios.development.vm, the query returns a string web.development.vm, which is the certname of the web server node, as shown in the following screenshot:

Querying facts with action facts

Querying facts works in a similar fashion to querying nodes. The syntax for querying facts is the puppet query facts 'search pattern'. This will print out all the facts from the nodes that match the search pattern. To filter the query results so that the command only returns the selected facts instead of all the facts, we can use the --facts parameter to specify a list of facts that the command should return.

Here is a sample command on how to query the ipaddres_eth1 and uptime facts from the nagios server node:

puppet query facts 'Class["nagios::server"]' --facts ipaddress_eth1,uptime

This command will return the following hash of facts from the node:

nagios.development.vm.
nagios.development.vm {"ipaddress_eth1":"192.168.56.12","uptime":"2:30 hours"}

To choose a different output format, we can add the –render-as yaml parameter to make the query return the yaml document instead of the standard PuppetDB query hash:

puppet query facts 'Class["nagios::server"]' --facts ipaddress_eth1,uptime –render-as yaml

Here are the query results in the yaml document format:

---
nagios.development.vm:
  uptime: "2:30 hours"
  ipaddress_eth1: "192.168.56.12"

Following is the screenshot of the preceding output:

The Puppetdbquery module on the command line is a useful tool for testing query patterns and discovering information about nodes in PuppetDB. How can these queries be made part of the Puppet deployment?

The Puppetdbquery module comes with a couple of handy Puppet functions called query_nodes and query_facts that allow us to make the PuppetDB queries from the Puppet manifest. We can make use of these functions, for example, when configuring a load balancer node to balance the HTTP requests between the web server nodes. Instead of using static IP addresses for the load balancer configuration (like we currently do in the loadbalancer/manifests/create.pp file), we can use the query_nodes function to discover the IP addresses of all the web server nodes and configure the load balancer accordingly. Using this method, the load balancer configuration will automatically update if the web server node's IP address changes or if we add more web servers to the environment.

Before we dive into this, let's take a look at the syntax of the functions.

['nagios.development.vm', 'web.development.vm']

query_nodes('Class["nagios::server"] or Class["webapp"]', ipaddress_eth1)

puppet query facts 'Class["nagios::server"]' --facts ipaddress_eth1,uptime

query_facts('Class["nagios::server"]', [ipaddress_eth1,uptime])

{ 'nagios.development.vm' => {"ipaddress_eth1":"192.168.56.12","uptime":"1:30 hours"} }

Creating a custom type for testing PuppetDB queries

Getting the PuppetDB query syntax right can often be tricky. When creating queries, it is important to test them properly to ensure that the queries return the desired results. To test PuppetDB queries, I usually create a short manifest that I run on the Puppet Master node, and examine the PuppetDB query results.

Create a new file in a text editor called puppetdb.pp with the following content:

The preceding manifest contains a custom defined type (lines 1-3), three variables (lines 4-6), and two references to a defined type (lines 7-8). Let's take a look at the manifest in detail:

• Line 1 begins with a defined type called puppetdb_query.

  This type is used for printing out PuppetDB query results.

• Line 2 uses a Puppet function called notice (https://docs.puppetlabs.com/references/latest/function.html#notice) that outputs a string value joined with the value of the ${name} variable, which is set at the time when the type is called (lines 7-8).

• Line 3 closes the custom defined type block.

• Line 4 creates a variable called $list, where we store the PuppetDB query results from the query_nodes function.

  The query_nodes function makes a PuppetDB query that returns a list of node names, such as a certname, that has the nagios::server or webapp class in its catalog.

• Line 5 creates a $hash variable, where we store the results from the query_facts function.

  The $hash variable name implies that the query_facts function returns a hash type of data (which means key-value pairs). The hash contains the values of facts ipaddress_eth1 and the uptime.

• Line 6 defines the third and last variable that is used to store the hash content of the $hash variable in the format of a string, hence the name $hash_to_string.

  The hash data type has to be converted to the string format so that we can use it when we call the type puppetdb_query (on line 8), which prints out the results of this query.

• Line 7 calls the puppetdb_query defined type with the $list variable as the name of the resource.

  Calling the defined type with a data type list as the name of the resource results in Puppet to call the type once per each element on the list.

  Assuming that the query_nodes function (line 4) returns a list of two node names, the type puppetdb_query is called twice.

• Line 8 makes another call to puppetdb_query, but this time using the $hash_to_string variable as the resource name.

  In this case, puppetdb_query is only called once, and the whole content of the hash produced by the query_facts function (line 5) is printed on the screen.

Once you have created the file, save it as puppetdb.pp under the Puppet module directory learning (on the virtual machine /media/sf_learning/puppetdb.pp).

Then, we can try to apply the manifest to the Puppet Master node. Log on to the Puppet Master and run the following two commands:

cd /media/sf_learning
puppet apply --modulepath=./ puppetdb.pp

On a successful Puppet run, you should see the following events being reported.

The output from the Puppet run may look a little bit messy, but when we take a look at it more closely, we can identify the following three Puppet events:

1. nagios.development.vm: This is the first record returned from PuppetDB by the query_nodes function, and it matches the query pattern Class["nagios::server"] (line 4 in puppetdb.pp).

2. web.development.vm: This is the second record returned from PuppetDB by the query_nodes function, and it matches the query pattern Class["webapp"] (line 4 in puppetdb.pp).

3. nagios.development.vm is {"uptime"=>"0:30 hours", "ipaddress_eth1"=>"192.168.56.12"}: This is the content of the hash returned from PuppetDB by the query_facts function (line 5 in puppetdb.pp).

We now have a better understanding of how to interact with PuppetDB using the query_nodes and query_facts functions. Let's see how we can make use of these functions to configure the load balancer node.

In Chapter 5, Load Balancing the Cluster, we created the loadbalancer and loadbalancer::create classes. The loadbalancer class created an HTTP proxy that routes the requests to the nagios server as well as to web server nodes via a load balancer that was created by the loadbalancer::create class.

One problem with these classes is that they are currently using the static IP configuration for the nagios server and web server connectivity. The loadbalancer class assumes that the nagios server has an IP address 192.168.56.11, and in the loadbalancer::create class, we assume that the web server nodes have the IP addresses 192.168.56.12 and 192.168.56.13.

Since Chapter 5, Load Balancing the Cluster, I've been bouncing all the virtual machines a number of times, and VirtualBox has allocated different IP addresses to virtual machines. At this moment, my web server has an IP address 192.168.56.11, and the nagios server's IP address is 192.168.56.12. I guess in your environment, virtual machines have a different set of IP addresses.

Unfortunately the virtual machine's IP addresses have changed, which means that the load balancer configuration in my environment is currently broken. The good news is that we can easily rectify the situation by making a couple of minor changes to the loadbalancer and loadbalancer::create classes.

Let's first tackle the loadbalancer class, and add a couple of PuppetDB queries to it. Here is a screenshot of the loadbalancer class in its current state:

The $nagios_url variable on line 2 defines the nagios server address as http://192.168.56.11/nagios, and the variable is referenced on line 13 by the proxy_pass attribute.

Instead of using a predefined IP address in the $nagios_url variable, we can use the query_nodes function to extract the nagios server IP address from PuppetDB, and pass the IP address to the $nagios_url variable. We'll also add a similar PuppetDB query for web server nodes, which we can reference from the loadbalancer::create class.

Here is a redesigned loadbalancer class that introduces you to two new PuppetDB queries and constructs the $nagios_url variable value from the query results:

Let's take a look at what has changed in the loadbalancer class:

Before we try running this manifest on a load balancer, let's make a couple of changes to the loadbalancer::create class, which currently has the following content:

The loadbalancer::create class creates two apache::balancermember records, which we are going to substitute with a defined type that creates the apache::balancermember record for each web server IP address that the PuppetDB query returns (line 4 in the loadbalancer/manifests/init.pp file). Here is the screenshot of the newly designed loadbalancer::create class:

Let's take a look at what has changed in the loadbalancer::create class:

Once you are done with the changes made to the init.pp and create.pp files and have saved the files in the loadbalancer/manifests directory, we can move on to apply manifests on the load balancer node.

Testing the PuppetDB query manifests on the load balancer node

To test our manifest changes, you need to boot up the load balancer node and have Puppet Master running of course.

Once the load balancer node is running, log in with the username root and password puppet.

Then, execute the following two commands:

cd /media/sf_learning
puppet agent -t

When the Puppet run completes successfully, you will see the following events displayed on the screen:

If you have the nagios server and web server nodes running, you should be able to access both of them via the load balancer node at http://<loadbalancer_ip>/nagios and http://<loadbalancer_ip>/webapp.

You can find out the value of <loadbalancer_ip > by running the hostname -I or ifconfig eth1 commands on the load balancer node.

Although functions can be distributed from any Puppet module, I prefer to store functions (and facts) in a dedicated Puppet module, which I have included in the Puppet's modulepath, and I can then reference these functions from the other Puppet modules.

The name of the module is not that important, but I usually include the word lib in the name of the module, which indicates that the module contains a library of functions. A suitable name for our functions module can be, for example, learning-flib.

Let's create a new module for functions with the following set of commands:

# cd /media/learning
# puppet module generate learning-flib –skip-interview

This command creates an empty Puppet module. Then, we rename the directory from learning-flib to flib:

# mv learning-flib flib

Now we use the mkdir command to create a directory structure to store functions, facts, and tests:

# mkdir -p flib/lib/puppet/parser/functions flib/lib/facter flib/tests

Once all the preceding commands have been executed, we can view the folder structure with the tree flib/ command, as shown in the following screenshot:

Now that we have created a module to store our custom functions, we can write our first function. Let's create a function called cli_command, which accepts a command-line command as an argument, executes the command, and returns the results.

Here is the content of the cli_command function:

The syntax of the cli_command function looks very different from the syntax that we have been using so far in Puppet manifests. This is because functions are written in the Ruby programming language instead of Puppet DSL. Let's take a look at the flib/lib/puppet/parser/functions/cli_command.rb file line by line: