Another useful command-line switch for puppet apply is the --noop option. It instructs Puppet to refrain from taking any action on unsynced resources. Instead, you only get a log output that indicates what will change without the switch. This is useful in determining whether a manifest would possibly break anything on your system:

root@puppetmaster:~# puppet apply puppet_service.pp --noop
Notice: Compiled catalog for puppetmaster.example.net in environment production in 0.63 seconds
Notice: /Stage[main]/Main/Service[puppet]/enable: current_value true, should be false (noop)
Notice: Class[Main]: Would have triggered 'refresh' from 1 events
Notice: Stage[main]: Would have triggered 'refresh' from 1 events
Notice: Applied catalog in 0.06 seconds

Note that the output format is the same as before, with a (noop) marker trailing the notice about the sync action. This log can be considered a preview of what will happen when the manifest is applied without the --noop switch.

The additional notices about triggered refreshes will be described later and can be ignored for the moment. You will have a better understanding of their significance after finishing this chapter and Chapter 4, Modularizing Manifests with Classes and Defined Types.

As of Puppet 3.x, there are only four variable types: Strings, Arrays, Hashes, and Boolean. Puppet 4 introduces a rich type system. The new type system will be explained in Chapter 7, New Features from Puppet 4. The basic variable types work much like their respective counterparts in other languages. Depending on your background, you might be familiar with using associative arrays or dictionaries as semantic equivalents to Puppet's hash type:

$a_bool = true
$a_string = 'This is a string value'
$an_array = [ 'This', 'forms', 'an', 'array' ]
$a_hash = { 
  'subject'   => 'Hashes',
  'predicate' => 'are written',
  'object'    => 'like this',
  'note'      => 'not actual grammar!',
  'also note' => [ 'nesting is',
    { 'allowed' => ' of course' } ], 
}

Accessing the values is equally simple. Note that the hash syntax is similar to that of Ruby, not Perl's:

$x = $a_string
$y = $an_array[1]
$z = $a_hash['object']

Strings can be used as resource attribute values, but it's worth noting that a resource title can also be a variable reference:

package { $apache_package:
  ensure => 'installed'
}

It's intuitively clear what a string value means in this context. But you can also pass arrays here to declare a whole set of resources in one statement. The following manifest manages three packages, making sure that they are all installed:

$packages = [ 'apache2',
  'libapache2-mod-php5',
  'libapache2-mod-passenger', ]
package { $packages:
  ensure => 'installed'
}

You will learn how to make efficient use of hash values in later chapters.

The array does not need to be stored in a variable to be used, but it is a good practice in some cases.

The easiest way to bring order to such a straightforward manifest is resource chaining. The syntax for this is a simple ASCII arrow between two resources:

package { 'haproxy':
  ensure => 'installed',
}
->
file { '/etc/haproxy/haproxy.cfg':
  ensure => file,
  owner  => 'root',
  group  => 'root',
  mode   => '0644',
  source => 'puppet:///modules/haproxy/etc/haproxy/haproxy.cfg',
}
->
service {'haproxy':
  ensure => 'running',
}

This is only viable if all the related resources can be written next to each other. In other words, if the graphic representation of the dependencies does not form a straight chain, but more of a tree, star, or any other shape, this syntax is not sufficient.

A more generic and flexible way to declare dependencies is through special metaparameters—parameters that are eligible for use with any resource type. There are different metaparameters, most of which have nothing to do with ordering (you have seen provider in an earlier example).

For resource ordering, puppet offers the metaparameters, require and before. Both take one or more references to a declared resource as their value. Puppet references have a special syntax, as was previously mentioned:

Type['title']
e.g.
Package['haproxy']

Here is the HAproxy manifest, ordered using the require metaparameter:

package { 'haproxy':
  ensure => 'installed',
}
file {'/etc/haproxy/haproxy.cfg':
  ensure  => file,
  owner   => 'root',
  group   => 'root',
  mode    => '0644',
  source  => 'puppet:///modules/haproxy/etc/haproxy/haproxy.cfg',
  require => Package['haproxy'],
}
service {'haproxy':
  ensure  => 'running',
  require => File['/etc/haproxy/haproxy.cfg'],
}

The following manifest is semantically identical, but relies on the before metaparameter rather than require:

package { 'haproxy':
  ensure => 'installed',
  before => File['/etc/haproxy/haproxy.cfg'],
}
file { '/etc/haproxy/haproxy.cfg':
  ensure => file,
  owner  => 'root',
  group  => 'root',
  mode   => '0644',
  source => 'puppet:///modules/haproxy/etc/haproxy/haproxy.cfg',
  before => Service['haproxy'],
}
service { 'haproxy':
  ensure => 'running',
}

The require metaparameter usually leads to more understandable code, because it expresses the dependency of the annotated resource on another resource. The before parameter, on the other hand, implies a dependency that a referenced resource forms upon the current resource. This can be counter-intuitive, especially for frequent users of packaging systems (which usually implement a require style dependency declaration).

Sometimes, it might be difficult to decide whether to use require or before. In simple cases, most people prefer require. In some cases, it is easier to use before. Think of services that have multiple configuration files. Keeping information about the configuration file and the requirement in a single place reduces errors caused by forgetting to also adopt changes to the service, when adding or removing additional configuration files. Take a look at the following example code:

file { '/etc/apache2/apache2.conf':
  ensure => file,
  before => Service['apache2'],
}
file { '/etc/apache2/httpd.conf':
  ensure => file,
  before => Service['apache2'],
}
service { 'apache2':
  ensure  => running,
  enable  => true,
}

In the example, all dependencies are declared within the file resource declarations. If you use the require parameter instead, you will always need to touch at least two resources in case of changes:

file { '/etc/apache2/apache2.conf':
  ensure  => file,
}
file { '/etc/apache2/httpd.conf':
  ensure  => file,
}
service { 'apache2':
  ensure  => running,
  enable  => true,
  require => [
    File['/etc/apache2/apache2.conf'],
    File['/etc/apache2/httpd.conf'],
  ],
}

Will you remember to update the service resource declaration whenever you add a new file to be managed by Puppet?

Consider another simpler example:

if $os_family == 'Debian' {
  file { '/etc/apt/preferences.d/example.net.prefs':
    content => '…',
    before  => Package['apache2'],
  }
}
package { 'apache2':
  ensure => 'installed',
}

The file in the preferences.d directory only makes sense for Debian-like systems. That's why the package cannot safely require it. If the manifest is applied on a different OS, such as CentOS, the apt preferences file will not appear in the catalog thanks to the if clause. If the package had it as a requirement regardless, the resulting catalog would be inconsistent, and Puppet would not apply it. Specifying before in the file resource is safe, and semantically equivalent.

The before metaparameter is outright necessary in situations like this one, and can make the manifest code more elegant and straightforward in other scenarios. Familiarity with both before and require is advisable.

Defining requirements serves another important purpose. References on declared resources will only be validated as successful references if the depended upon resource was finished successfully. This can be seen like a stop point inside Puppet DSL code, when a required resource is not synchronized successfully.

For example, a file resource will fail if the URL of the source file is broken:

file { '/etc/haproxy/haproxy.cfg':
  ensure => file,
  source => 'puppet:///modules/haproxy/etc/haproxy.cfg',
} 

One path segment is missing here. Puppet will report that the file resource could not be synchronized:

root@puppetmaster:~# puppet apply typo.pp
Notice: Compiled catalog for puppetmaster.example.net in environment production in 0.62 seconds
Error: /Stage[main]/Main/File[/etc/haproxy/haproxy.cfg]: Could not evaluate: Could not retrieve information from environment production source(s) puppet:///modules/haproxy/etc/haproxy.cfg
Notice: /Stage[main]/Main/Service[haproxy]: Dependency File[/etc/haproxy/haproxy.cfg] has failures: true
Warning: /Stage[main]/Main/Service[haproxy]: Skipping because of failed dependencies
Notice: Applied catalog in 0.06 seconds

In this example, the Error line describes the error caused by the broken URL. The error propagation is represented by the Notice and Warning lines below it.

Puppet failed to apply changes to the configuration file; it cannot compare the current state to the nonexistent source. As the service depends on the configuration file, Puppet will not even try to start it. This is for safety: if any dependencies cannot be put into the defined state, Puppet must assume that the system is not fit for application of the dependent resource.

This is another important reason to make consequent use of resource dependencies. Remember that both the chaining arrow and the before metaparameter imply error propagation as well.

Before you learn about another way in which resources can interrelate, there is an issue that you should be aware of: dependencies must not form circles. Let's visualize this in an example:

file { '/etc/haproxy':
  ensure => 'directory',
  owner  => 'root',
  group  => 'root',
  mode   => '0644',
}
file { '/etc/haproxy/haproxy.cfg':
  ensure => file',
  owner  => 'root',
  group  => 'root',
  mode   => '0644',
  source => 'puppet:///modules/haproxy/etc/haproxy/haproxy.cfg',
}
service { 'haproxy':
  ensure  => 'running',
  require => File['/etc/haproxy/haproxy.cfg'],
  before  => File['/etc/haproxy'],
}

The dependency circle in this manifest is somewhat hidden (as will likely be the case for many such circles that you will encounter during regular use of Puppet). It is formed by the following relations:

Granted, the preceding example is contrived—it will not make sense to manage the service before the configuration directory. Nevertheless, even a manifest design that is apparently sound can result in circular dependencies. This is how Puppet will react to that:

root@puppetmaster:~# puppet apply circle.pp
Notice: Compiled catalog for puppetmaster.example.net in environment production in 0.62 seconds
Error: Failed to apply catalog: Found 1 dependency cycle:
(File[/etc/haproxy/haproxy.cfg] => Service[haproxy] => File[/etc/haproxy] => File[/etc/haproxy/haproxy.cfg])
Try the '--graph' option and opening the resulting '.dot' file in OmniGraffle or GraphViz

The output helps you locate the offending relation(s). For very wide dependency circles with lots of involved resources, the textual rendering is difficult to analyze. Therefore, Puppet also gives you the opportunity to get a graphical representation of the dependency graph through the --graph option.

If you do this, Puppet will include the full path to the newly created .dot file in its output. Its content looks similar to Puppet's output:

digraph Resource_Cycles {
label = "Resource Cycles"
"File[/etc/haproxy/haproxy.cfg]" ->"Service[haproxy]" ->"File[/etc/haproxy]" ->"File[/etc/haproxy/haproxy.cfg]"
}

This is not helpful by itself, but it can be fed directly into tools such as dotty to produce an actual diagram.

To summarize, resource dependencies are helpful in keeping Puppet from acting upon resources in unexpected or uncontrolled situations. They are also useful in restricting the order of resource evaluation.

Especially in the absence of central registries such as LDAP, it is useful to be able to manage user accounts on each of your machines. There are providers for all supported platforms, however, the available attributes vary. On Linux, the useradd provider is the most common. It allows the management of all fields in /etc/passwd, such as uid and shell, and also group memberships:

group { 'proxy-admins':
  ensure => present,
  gid    => 4002,
}
user { 'john':
  ensure           => present,
  uid              => 2014,
  home             => '/home/john'
  managehome       => true, # <- adds -m to useradd
  gid              => 1000,
  shell            => '/bin/zsh',
  groups           => [ 'proxy-admins' ],
}

As with all resources, Puppet will not only make sure that the user and group exist, but also fix any divergent properties, such as the home directory.

Even though the user depends on the group, (because they cannot be added before the group exists) it need not be expressed in the manifest. The user automatically requires all necessary groups, similar to a file auto requiring its parent directory.

It was mentioned earlier that there are different attributes available, depending on the Operating System. Linux (and the useradd provider) support setting a password, whereas on HP-UX (using the hp-ux provider) the user password cannot be set via Puppet.

In this case, Puppet will only show a warning saying that the user resource type is making use of an unsupported attribute, and will continue managing all other attributes. In other words, using an unsupported attribute in your Puppet DSL code will not break your Puppet run.

There is one oddball resource type in the Puppet core. Remember our earlier assertion that Puppet is not a specialized scripting engine, but instead, a tool that allows you to model part of your system state in a compelling DSL, and which is capable of altering your system to meet the defined goal. This is why you declare user and group, instead of invoking groupadd and useradd in order. You can do this because Puppet comes with support to manage such entities. This is vastly beneficial because Puppet also knows that on different platforms, other commands are used for account management, and that the arguments can be subtly different on some systems.

Of course, Puppet does not have knowledge of all the conceivable particulars of any supported system. Say that you wish to manage an OpenAFS file server. There are no specific resource types to aid you with this. The ideal solution is to exploit Puppet's plugin system and to write your own types and providers so that your manifests can just reflect the AFS-specific configuration. This is not simple though, and also not worthwhile in cases where you only need Puppet to invoke some exotic commands from very few places in your manifest.

For such cases, Puppet ships with the exec resource type, which allows the execution of custom commands in lieu of an abstract sync action.

For example, it can be used to unpack a tarball in the absence of a proper package:

exec { 'tar cjf /opt/packages/homebrewn-3.2.tar.bz2':
  cwd     => '/opt',
  path    => '/bin:/usr/bin',
  creates => '/opt/homebrewn-3.2',
}

The creates parameter is important for Puppet to tell whether the command needs running—once the specified path exists, the resource counts as synchronized. For commands that do not create a telltale file or directory, there are the alternative parameters, onlyif and unless, to allow Puppet to query the sync state:

exec { 'perl -MCPAN -e "install YAML"': path   => '/bin:/usr/bin', 
  unless => 'cpan -l | grep -qP ^YAML\\b',
}

The query command's exit code determines the state. In the case of unless, the exec command runs if the query fails. This is how the exec type maintains idempotency. Puppet does this automatically for most resource types, but this is not possible for exec, because synchronization is defined so arbitrarily. It becomes your responsibility as the user to define the appropriate queries per resource.

Finally, the exec type resources are the second notable case of receivers for events using notify and subscribe:

exec { 'apt-get update': path        => '/bin:/usr/bin',
  subscribe   => File['/etc/apt/sources.list.d/jenkins.list'],
  refreshonly => true,
}

You can even chain multiple exec resources in this fashion so that each invocation triggers the next one. However, this is a bad practice and degrades Puppet to a (rather flawed) scripting engine. The exec resources should be avoided in favor of regular resources whenever possible. Some resource types that are not part of the core are available as plugins from the Puppet Forge. You will learn more about this topic in Chapter 5, Extending Your Puppet Infrastructure with Modules.

Since exec resources can be used to perform virtually any operation, they are sometimes abused to stand in for more proper resource types. This is a typical antipattern in Puppet manifests. It is safer to regard exec resources as the last resort that is only to be used if all other alternatives have been exhausted.

puppet describe <type> [-s]

puppet describe --list

Let's briefly discuss two more types that are supported out of the box. They allow the management of cron jobs, mounted partitions, and shares respectively, which are all frequent requirements in server operation.

A cron job mainly consists of a command and the recurring time and date at which to run the command. Puppet models the command and each date particle as a property of a resource with the cron type:

cron { 'clean-files':
  ensure      => present,
  user        => 'root',
  command     => '/usr/local/bin/clean-files',
  minute      => '1',
  hour        => '3',
  weekday     => [ '2', '6' ],
  environment => 'MAILTO=felix@example.net',
}

The environment property allows you to specify one or more variable bindings for cron to add to the job.

Finally, Puppet will manage all aspects of mountable filesystems for you—including their basic attributes such as the source device and mount point, the mount options, and the current state. A line from the fstab file translates quite literally to a Puppet manifest:

mount { '/media/gluster-data':
  ensure  => 'mounted',
  device  => 'gluster01:/data',
  fstype  => 'glusterfs',
  options => 'defaults,_netdev',
  dump    => 0,
  pass    => 0,
}

For this resource, Puppet will make sure that the filesystem is indeed mounted after the run. Ensuring the unmounted state is also possible, of course; Puppet can also just make sure the entry is present in the fstab file, or absent from the system altogether.

The terminology around the master software might be a little confusing. That's because both the terms, Puppet Master and Puppet Server, are floating around, and they are closely related too. Let's consider some technological background in order to give you a better understanding of what is what.

Puppet's master service mainly comprises a RESTful HTTP API. Agents initiate the HTTPS transactions, with both sides identifying each other using trusted SSL certificates. During the time when Puppet 3 and older versions were current, the HTTPS layer was typically handled by Apache. Puppet's Ruby core was invoked through the Passenger module. This approach offered good stability and scalability.

Puppet Labs has improved upon this standard solution with a specialized software called puppetserver. The Ruby-based core of the master remains basically unchanged, although it now runs on JRuby instead of Ruby's own MRI. The HTTPS layer is run by Jetty, sharing the same Java Virtual Machine with the master.

By cutting out some middlemen, puppetserver is faster and more scalable than a Passenger solution. It is also significantly easier to set up. Towards the end of this chapter, the two approaches are recapped and visually compared.

Getting the puppetserver software onto a Linux machine is just as simple as the agent package (which you did at the very beginning of Chapter 1, Writing Your First Manifests). At the time of writing this, distribution packages were available on Red Hat Enterprise Linux and its derivatives. For Debian and Ubuntu, it was still necessary to rely on Puppet Labs' own repositories to get the software.

A great way to get Puppet Labs packages on any platform is the Puppet Collection. Shortly after the release of Puppet 4, Puppet Labs created this new way of supplying software. This can be considered as a distribution in its own right. Unlike Linux distributions, it does not contain a Kernel, system tools, and libraries. Instead, it comprises various software from the Puppet ecosystem. Software versions that are available from the same Puppet Collection are guaranteed to work well together.

Use the following commands to install puppetserver from the first Puppet Collection (PC1) on a Debian 7 machine. (The Collection for Debian 8 has not yet received a puppetserver package at the time of writing this.)

root@puppetmaster# wget http://apt.puppetlabs.com/puppetlabs-release-pc1-wheezy.deb
root@puppetmaster# dpkg -i puppetlabs-release-pc1-wheezy.deb
root@puppetmaster# apt-get update
root@puppetmaster# apt-get install puppetserver

The puppetserver package comprises only the Jetty server and the Clojure API, but the all-in-one puppet-agent package is pulled in as a dependency.

Specifically, you can use the puppet command on the master node. You will soon learn how this is useful. However, when using the packages from Puppet Labs, everything gets installed under /opt/puppetlabs. It is advisable to make sure that your PATH variable always includes the /opt/puppetlabs/bin directory so that the puppet command is found here.

Regardless of this, once the puppetserver package is installed, you can start the master service:

root@puppetmaster# service puppetserver start

Depending on the power of your machine, the startup can take a few minutes. Once initialization completes, the server will operate very smoothly though. As soon as the master port 8140 is open, your Puppet master is ready to serve requests.

puppet master –-no-daemonize

When you used Puppet locally in Chapter 1, Writing Your First Manifests, you specified a manifest file that puppet apply should compile. The master compiles manifests for many machines, but the agent does not get to choose which source file is to be used—this is completely at the master's discretion. The starting point for any compilation by the master is always the site manifest, which can be found in /opt/puppetlabs/code/environments/production/manifests/.

Each connecting agent will use all the manifests found here. Of course, you don't want to manage only one identical set of resources on all your machines. To define a piece of manifest exclusively for a specific agent, put it in a node block. This block's contents will only be considered when the calling agent has a matching common name in its SSL certificate. You can dedicate a piece of the manifest to a machine with the name of agent, for example:

node 'agent' {
  $packages = [ 'apache2', 
    'libapache2-mod-php5', 
    'libapache2-mod-passenger', ] 
  package { $packages:
    ensure => 'installed', 
    before => Service['apache2'], 
  } 
  service { 'apache2': 
    ensure => 'running', 
    enable => true, 
  }
}

Before you set up and connect your first agent to the master, step back and think about how the master should be addressed. By default, agents will try to resolve the unqualified puppet hostname in order to get the master's address. If you have a default domain that is being searched by your machines, you can use this as a default and add a record for puppet as a subdomain (such as puppet.example.net). Otherwise, pick a domain name that seems fitting to you, such as master.example.net or adm01.example.net. What's important is the following:

The mode of resolution depends on your circumstances—the hosts file on each machine is one ubiquitous possibility. The Puppet server listens on all the available addresses by default.

This leaves the task of creating a suitable certificate, which is simple. Configure the master to use the appropriate certificate name and restart the service. If the certificate does not exist yet, Puppet will take the necessary steps to create it. Put the following setting into your /etc/puppetlabs/puppet/puppet.conf file on the master machine:

[master]
certname=master.example.net

Upon its next start, the master will use the appropriate certificate for all SSL connections. The automatic proliferation of SSL data is not dangerous even in an existing setup, except for the certification authority. If the master were to generate a new CA certificate at any point in time, it would break the trust of all existing agents.

All the customization of the master's parameters can be made in the puppet.conf file. The operating system packages ship with some settings that are deemed sensible by the respective maintainers. Apart from these explicit settings, Puppet relies on defaults that are either built-in or derived from the environment (details on how this works follow in the next chapter):

root@puppetmaster # puppet master --configprint manifest
/etc/puppetlabs/code/environments/production/manifests

Most users will want to rely on these defaults for as many settings as possible. This is possible without any drawbacks because Puppet makes all settings fully transparent using the --configprint parameter. For example, you can find out where the master manifest files are located.

To get an overview of all available settings and their values, use the following command:

root@puppetmaster# puppet master --configprint all | less

While this command is especially useful on the master side, the same introspection is available for puppet apply and puppet agent.

In a Puppet-centric workflow, you typically want all changes to the configuration of servers (perhaps even workstations) to originate on the Puppet master and propagate to the agents automatically. Each new machine gets integrated into the Puppet infrastructure with the master at its center and gets removed during the decommissioning, as shown in the following diagram:

The very first step—generating a key and a certificate signing request—is always performed implicitly and automatically at the start of an agent run if no local SSL data exists yet. Puppet creates the required data if no appropriate files are found. There will be a short description on how to trigger this behavior manually later in this section.

The next step is usually the signing of the agent's certificate, which is performed on the master. It is a good practice to monitor the pending requests by listing them on the console:

root@puppetmaster# puppet cert list
root@puppetmaster# puppet cert sign '<agent fqdn>'

From this point on, the agent will periodically check with the master to load updated catalogs. The default interval for this is 30 minutes. The agent will perform a run of a catalog each time and check the sync state of all the contained resources. The run is performed for unchanged catalogs as well, because the sync states can change between runs.

Launching this background process can be done manually through a simple command:

root@agent# puppet agent

However, it is preferable to do this through the puppet system service.

When an agent machine is taken out of active service, its certificate should be invalidated. As is customary with SSL, this is done through revocation. The master adds the serial number of the certificate to its certificate revocation list. This list, too, is shared with each agent machine. Revocation is initiated on the master through the puppet cert command:

root@puppetmaster# puppet cert revoke agent

The agent can then no longer use its old certificate:

root@agent# puppet agent --test
Warning: Unable to fetch my node definition, but the agent run will continue:
Warning: SSL_connect SYSCALL returned=5 errno=0 state=unknown state
[...]
Error: Could not retrieve catalog from remote server: SSL_connect SYSCALL returned=5 errno=0 state=unknown state
[...]

Sometimes, it is necessary during an agent machine's life cycle to regenerate its certificate and related data—the reasons can include data loss, human error, or certificate expiration, among others. Performing the regeneration is quite simple: all relevant files are kept at /etc/puppetlabs/puppet/ssl (for Puppet 3.x, this is /var/lib/puppet/ssl) on the agent machine. Once these files are removed (or rather, the whole ssl/ directory tree), Puppet will renew everything on the next agent run. Of course, a new certificate must be signed. This requires some preparation—just initiating the request from the agent will fail:

root@agent# puppet agent –test
Info: Creating a new SSL key for agent
Info: Caching certificate for ca
Info: Caching certificate for agent.example.net
Error: Could not request certificate: The certificate retrieved from the master does not match the agent's private key.
Certificate fingerprint: 6A:9F:12:C8:75:C0:B6:10:45:ED:C3:97:24:CC:98:F2:B6:1A:B5: 4C:E3:98:96:4F:DA:CD:5B:59:E0:7F:F5:E6

The master still has the old certificate cached. This is a simple protection against the impersonation of your agents by unauthorized entities. To fix this, remove the certificate from both the master and the agent and then start a puppet run, which will automatically regenerate a certificate.

On the master:

puppet cert clean agent.example.net

On the agent:

puppet agent –t
Exiting; failed to retrieve certificate and waitforcert is disabled

Once you perform the cleanup operation on the master, as advised in the preceding output, and remove the indicated file from the agent machine, the agent will be able to successfully place its new CSR:

root@puppetmaster# puppet cert clean agent
Notice: Revoked certificate with serial 18
Notice: Removing file Puppet::SSL::Certificate agent at '/etc/puppetlabs/ puppet/ssl/ca/signed/agent.pem'
Notice: Removing file Puppet::SSL::Certificate agent at '/etc/puppetlabs/ puppet/ssl/certs/agent.pem'

The rest of the process is identical to the original certificate creation. The agent uploads its CSR to the master, where the certificate is created through the puppet cert sign command.

There is an alternative way to operate the agent. We covered starting one long-running puppet agent process that does its work in set intervals and then goes back to sleep. However, it is also possible to have cron launch a discrete agent process in the same interval. This agent will contact the master once, run the received catalog, and then terminate. This has several advantages as follows:

Setting Puppet to run the agent from cron is also very easy to do—with Puppet! You can use a manifest such as the following:

service { 'puppet': enable => false }
cron { 'puppet-agent-run':
  user    => 'root',
  command =>
    'puppet agent --no-daemonize --onetime --logdest=syslog',
  minute  => fqdn_rand(60),
  hour    => absent,
} 

The fqdn_rand function computes a distinct minute for each of your agents. Setting the hour property to absent means that the job should run every hour.

puppetserver is a great way to run the master service. It is simple to set up and maintain, and it also has great performance during operation. Starting up can take a little while to initialize everything to that end.

There are only few customizable settings that can impact performance. Seeing as puppetserver runs in the JVM, the most important tuning approach is to scale the heap. A small heap will increase the overhead for garbage collection. Therefore, you should use the -Xmx and -Xms Java options to allow the JVM to use large parts of your available memory for the mentioned heap.

On Debian, these settings are found in /etc/default/puppetserver. It is sensible to pass the same value to both. A dynamic heap has little benefit, because you cannot safely use any saved memory.

Both Passenger and puppetserver have their share of complexity. These are much less visible to the user in the case of puppetserver, however. All that's needed is the software package and a Java runtime. The internals are well hidden.

The following diagram highlights the differences:

With puppetserver, both the web service and the Ruby runtimes share a single JVM. This allows a better overall performance, and is easier to set up. All new setups should therefore, prefer puppetserver over Passenger.

You have already seen the use of the processorcount fact in an example. In the manifest, each fact value is available as a global variable value. That is why you can just use the ::processorcount expression where you need it.

Some helpful facts have already been mentioned. The processorcount fact might play a role for your configuration. When configuring some services, you will want to use the machine's ipaddress value in a configuration file or as an argument value:

file { '/etc/mysql/conf.d/bind-address':
  ensure  => 'file',
  mode    => '0644',
  content => "[mysqld]\nbind-address=${::ipaddress}\n",
}

Besides the hostname, your manifest can also make use of the fully qualified domain name (FQDN) of the agent machine.

file { '/etc/my-secret':
  ensure => 'file',
  mode   => '0600',
  owner  => 'root',
  source => "puppet:///modules/secrets/${::clientcert}/key",
}

There is a whole group of facts to describe the operating system. Each fact is useful in different situations. The operatingsystem fact takes values such as Debian or CentOS:

if $::operatingsystem != 'Ubuntu' {
  package { 'avahi-daemon':
    ensure => absent
  }
}

If your manifest will behave identically on RHEL, CentOS, and Fedora (but not on Debian and Ubuntu), you should make use of the osfamily fact instead:

if $::osfamily == 'RedHat' {
  $kernel_package = 'kernel'
}

The operatingsystemrelease fact allows you to tailor your manifests to differences between the versions of your OS:

if $::operatingsystem == 'Debian' {
  if versioncmp($::operatingsystemrelease, '7.0') >= 0 {
    $ssh_ecdsa_support = true
  }
}

Facts such as mac address, the different SSH host keys, fingerprints, and others make it easy to use Puppet for keeping inventory of your hardware. There are a slew of other useful facts. Of course, the collection will not suit every possible need of every user out there. That is why Facter comes readily extendible.

Technically, nothing is stopping you from adding your own fact code right next to the core facts, either by maintaining your own Facter package, or even by deploying the Ruby code files to your agents directly through Puppet management. However, Puppet offers a much more convenient alternative in the form of custom facts.

We have still not covered Puppet modules yet. They will be thoroughly introduced in Chapter 5, Extending Your Puppet Infrastructure with Modules. For now, just create a Ruby file at /etc/puppetlabs/code/environments/production/modules/hello_world/lib/facter/hello.rb on the master machine. Puppet will recognize this as a custom fact of the name, hello. (For Puppet 3 or older versions, the path should be /etc/puppet/modules/hello_world/lib/facter/hello.rb.)

The inner workings of Facter are very straightforward and goal oriented. There is one block of Ruby code for each fact, and the return value of the block becomes the fact value. Many facts are self-sufficient, but others will rely on the values of one or more basic facts. For example, the method to determine the IP address(es) of the local machine is highly dependent upon the operating system.

The hello fact is very simple though:

Facter.add(:hello) do
  setcode { "Hello, world!" }
end

The return value of the setcode block is the string, Hello, world!, and you can use this fact as $::hello in a Puppet manifest.

It is important for the pluginsync option to be enabled on the agent side. This has been the default for a long time and should not require any customization. The agent will synchronize all custom facts whenever checking in to the master. They are permanently available on the agent machine after that. You can then retrieve the hello fact from the command line using the following line:

# puppet facts | grep hello

By just invoking the following command without an argument, you request a list of all fact names and values.

# puppet facts

There is also a facter command. It does roughly the same as puppet facts, but will only show built in facts, not your custom facts.

This book will not cover all aspects of Facter's API, but there is one facility that is quite essential. Many of your custom facts will only be useful on Unix-like systems, and others will only be useful on your Windows boxen. You can retrieve such values using a construct like the following:

if Facter.value(:kernel) != "windows"
  nil
else
  # actual fact code here
end

This would be quite tedious and repetitive though. Instead, you can invoke the confine method within the Facter.add(name) { … } block:

Facter.add(:msvs_version) do
  confine :kernel => :windows
  setcode do
    # …
  end
end

You can confine a fact to several alternative values as well:

confine :kernel => [ :linux, :sunos ]

Finally, if a fact does make sense in different circumstances, but requires drastically different code in each respective case, you can add the same fact several times, each with a different set of confine values. Core facts such as ipaddress use this often:

Facter.add(:ipaddress) do
  confine :kernel => :linux
  …
end
Facter.add(:ipaddress) do
  confine :kernel => %w{FreeBSD OpenBSD Darwin DragonFly}
  …
end
…

You can confine facts based on any combination of other facts, not just kernel. It is a very popular choice though. The operatingsystem or osfamily facts can be more appropriate in certain situations. Technically, you can even confine some of your facts to certain processorcount values and so forth.

# site-facts.txt
workgroup=CT4Site2
domain_psk=nm56DxLp%

# site-facts.yaml
workgroup: CT4Site2
domain_psk: nm56DxLp%

# site-facts.json
{ 'workgroup': 'CT4Site2', 'domain_psk': 'nm56DxLp%' }

file { '/etc/puppetlabs/facter/facts.d/site-facts.yaml':
  ensure => 'file',
  source => 'puppet:///…',
}

#!/bin/sh

echo hello=Hello, world\!

The whole structure and philosophy of Facter serves the goal of allowing for platform-agnostic usage and development. The same collection of facts (roughly) is available on all supported platforms. This allows Puppet users to keep a coherent development style through manifests for all those different systems.

Facter forms a layer of abstraction over the characteristics of both hardware and software. It is an important piece of Puppet's platform-independent architecture. Another piece that was mentioned before is the type and provider subsystem. Types and providers are explored in greater detail in the following sections.

Once the compilation has succeeded, the master hands out the catalog and the agent enters the catalog validation phase. Each resource type can define some Ruby methods that ensure that the passed values make sense. This happens on two levels of granularity: each attribute can validate its input value, and then the resource as a whole can be checked for consistency.

One example for attribute value validation can be found in the ssh_authorized_key resource type. A resource of this type fails if its key value contains a whitespace character, because SSH keys cannot comprise multiple strings.

Validation of whole resources happens with the cron type, for example. It makes sure that the time fields make sense together. The following resource would not pass, because special times, such as midgnight, cannot be combined with numeric fields:

cron { 'invalid-resource':
  command => 'apt-get update',
  special => 'midnight',
  weekday => [ '2', '5' ],
}

Another task during this phase is the transformation of input values to more suitable internal representations. The resource type code refers to this as a munge action. Typical examples of munging are the removal of leading and trailing whitespace from string values, or the conversion of array values to an appropriate string format—this can be a comma-separated list, but for search paths, the separator should be a colon instead. Other kinds of values will use different representations.

Next up is the prefetching phase. Some resource types allow the agent to create an internal list of resource instances that are present on the system. These types are referred to as being enumerable types. For example, this is possible (and makes sense) for installed packages—Puppet can just invoke the package manager to produce the list. For other types, such as file, this would not be prudent. Creating a list of all reachable paths in the whole filesystem can be arbitrarily expensive, depending on the system on which the agent is running.

# puppet resource user root
user { 'root':
ensure           => 'present',
comment          => 'root',
gid              => '0',
home             => '/root',
password         => '$6$17/7FtU/$TvYEDtFgGr0SaS7xOVloWXVTqQxxDUgH.eBKJ7bgHJ.hdoc03Xrvm2ru0HFKpu1QSpVW/7o.rLdk/9MZANEGt/',
password_max_age => '99999',
password_min_age => '0',
shell            => '/bin/bash',
uid              => '0',
}

Finally, the agent starts walking its internal graph of interdependent resources. Each resource is brought in sync, if necessary. This happens separately for each individual property, for the most part.

There are some notable aspects of the whole agent process. For one, attributes are handled independently. Each can define its own methods for the different phases. There are quite a number of hooks, which allow a resource type author to add a lot of flexibility to the model.

It is also worth noting that the whole validation process is performed by the agent, not the master. This is beneficial in terms of performance. The master saves a lot of work, which gets distributed to the network of agents (which scales with your needs automatically).

There are some resource types that use no providers, but they are rare among the core types. Most of the interesting management tasks that Puppet makes easy just work differently among operating systems, and providers enable this in a most elegant fashion.

Even for straightforward tasks that are the same on all platforms, there might be a provider. For example, there is a host type to manage entries in the /etc/hosts file. Its syntax is universal, so the code can technically just be implemented in the type. However, there are actual abstract base classes for certain kinds of providers in the Puppet code base. One of them makes it very easy to build providers that edit files, if those files consist of single-line records with ordered fields. Therefore, it makes sense to implement a provider for the host type and base it on this provider class.

host { 'puppet':
  ip           => '10.144.12.100',
  host_aliases => [ 'puppet.example.net', 'master' ],
}

Puppet's resource types and their providers work together to form a solid abstraction layer over software configuration details. The type system is an extendable basis for Puppet's powerful DSL. It forms an elaborate interface for the polymorphous provider layer.

The providers flexibly implement the actual management actions that Puppet is supposed to perform. They map the necessary synchronization steps to commands and system interactions. Many providers cannot satisfy every nuance that the resource type models. The feature system takes care of these disparities in a transparent fashion.

A Puppet class can be considered to be a container for resources. It is defined once and selected by all nodes that need to make use of the prepared functionality. Each class represents a well-known subset of a system's configuration.

For example, a classic use case is a class that installs the Apache web server and applies some basic settings. This class will look like the following:

class apache {
  package { 'apache2':
    ensure => 'installed',
  }
  file { '/etc/apache2/apache2.conf':
    ensure => 'file',
    source =>
      'puppet:///modules/apache/etc/apache2/apache2.conf', 
    require => Package['apache2'], 
  } 
  service { 'apache2': 
    enable    => true, 
    subscribe => File['/etc/apache2/apache2.conf', 
  } 
} 

All web server nodes will make use of this class. To this end, their manifests need to contain a simple statement:

include apache

This is referred to as including a class, or declaring it. If your apache class is powerful enough to do all that is needed, this line might fully comprise a node block's content:

node 'webserver01' {
  include apache
}

This already concludes our tour of classes in a nutshell. There is yet some more to discuss, of course, but let's take a look at defined types before that.

A defined type can be imagined like a blueprint for a piece of manifest. Like a class, it mainly consists of a body of the manifest code. However, a defined type takes arguments and makes their values available in its body as local variables.

Here is another typical example of a defined type, the Apache virtual host configuration:

define virtual_host(
  String $content,
  String[3,3] $priority = '050'
) {
  file { "/etc/apache2/sites-available/${name}":
    ensure  => 'file',
    owner   => 'root',
    group   => 'root',
    mode    => '0644',
    content => $content
  }
  file { "/etc/apache2/sites-enabled/${priority}-${name}": 
    ensure => 'link', 
    target => "../sites-available/${name}";
  }
}

This code might still seem pretty cryptic. It will get clearer in the context of how it is actually used from other places in your manifest; the following code shows you how:

virtual_host { 'example.net': 
  content => file('apache/vhosts/example.net') 
}
virtual_host{ 'fallback': 
  priority => '999', 
  content  => file('apache/vhosts/fallback') 
} 

This is why the construct is called a defined type—you can now place what appear to be resources in your manifest, but you really call your own manifest code construct.

virtual_host {
  'example.net':
    content  => 'foo';
  'fallback':
    priority => '999',
    content  => ...,
}

The virtual_host type takes two arguments: the content argument is mandatory and is used verbatim in the configuration file resource. Puppet will synchronize that file's content to what is specified in the manifest. The priority argument is optional and its value becomes the file name prefix. If omitted, the respective virtual host definition uses the default priority of 050.

Both parameters of this example type are of the String type. For details about Puppet's variable type system, see Chapter 7, New Features from Puppet 4. It suffices to say that you can restrict parameters to certain value types. This is optional, however. You can omit the type name, and Puppet will accept any value for the parameter in question.

Also, each defined type can implicitly refer to the name (or title) by which it was called. In other words, each instance of your define gets a name, and you can access it through the $name or $title variable.

The respective purposes of Puppet's class and defined type are very specific and they usually don't overlap.

The class declares resources and properties that are in some way centric to the system. A class is a finalized description of one or sometimes more aspects of your system as a whole. Whatever the class represents, it can only ever exist in one form; to Puppet, each class is implicitly a singleton, a fixed set of information that either applies to your system (the class is included), or not.

The typical resources you will encapsulate in a class for convenient inclusion in a manifest are as follows:

The define is used for all things that exist in multiple instances. All aspects that appear in varying quantities in your system can possibly be modeled using this language construct. In this regard, the define is very similar to the full-fledged resource it mimics with its declaration syntax. Some of the typical contents of defined types are:

The class's singleton nature is especially valuable because it prevents clashes in the form of multiple resource declarations. Remember that each resource must be unique to a catalog. For example, consider a second declaration of the Apache package:

package { 'apache2': }

This declaration can be anywhere in the manifest of one of your web servers (say, right in the node block, next to include apache); this additional declaration will prevent the successful compilation of a catalog.

The virtue of the class is that there can be an arbitrary number of include statements for the same class strewn throughout the manifest. Puppet will commit the class's contents to the catalog exactly once.

virtual_host { 'wordpress': 
  content  => file(...), 
  priority => '011', 
}
virtual_host { 'wordpress': 
  content  => '# Dummy vhost', 
  priority => '600', 
}   

Many classes are written to make Puppet perform momentous tasks on the agent platform. Of these, the Apache class is probably one of the more modest examples. You can conceive a class that can be included from any machine's manifest and make sure that the following conditions are met:

There are two general ways you can go about the task of creating a class of this magnitude. It can either become what one might call a monolithic implementation, a class with a large body that comprises all resources that work together to form the desired security baseline. On the other hand, you could aim for a composite design, with few resources (or none at all) in the class body, and a number of include statements for simpler classes instead. The functionality is compartmentalized, and the central class acts as a collector.

We have not yet touched on the ability of classes to include other classes. That's because it's quite simple. The body of a class can comprise almost any manifest, and the include statement is no exception. Among the few things that cannot appear in a class are node blocks.

Adding some life to the descriptions, this is how the respective classes will roughly look like:

class monolithic_security { 
  package { [ 'iptables', 'rkhunter', 'postfix' ]:
    ensure => 'installed';
  } 
  cron { 'run-rkhunter': 
    ... 
  } 
  file { '/etc/init.d/iptables-firewall': 
    source => ... 
    mode   => 755 
  }
  file { '/etc/postfix/main.cf': 
    ensure  => 'file', 
    content => ... 
  } 
  service { [ 'postfix', 'iptables-firewall' ]: 
    ensure => 'running', 
    enable => true 
  } 
}

class divided_security {
  include iptables_firewall
  include rkhunter
  include postfix
}

When developing your own functional classes, you should not try to pick either of these extremes. Most classes will end up anywhere on the spectrum in between. The choice can be largely based on your personal preference. The technical implications are subtle, but these are the respective drawbacks:

Neither of these aspects is of critical importance at most times. The case-by-case design choices will be based on each author's experience and preference. When in doubt, lean towards composite designs at first.

There is another common use case for classes. Instead of filling a class with lots of aspects that work together to achieve a complex goal, you can also limit the class to a very specific purpose. Some classes will contain but one resource. The class wraps the resource, so to speak.

This is useful for resources that are needed in different contexts. By wrapping them away in a class, you can make sure that those contexts do not create multiple declarations of the same resource.

For example, the netcat package can be useful to firewall servers, but also to web application servers. There is probably a firewall class and an appserver class. Both declare the netcat package:

package { 'netcat': 
  ensure => 'installed' 
} 

If any server ever has both roles (this might happen for budget reasons or in other unforeseen circumstances), it is a problem; when both the firewall and appserver classes are included, the resulting manifest declares the netcat package twice. This is forbidden. To resolve this situation, the package resource can be wrapped in a netcat class, which is included by both the firewall and appserver classes:

class netcat {
  package { 'netcat': 
    ensure => 'installed' 
  } 
} 

Let's consider another typical example for component classes that ensures the presence of some common file path. Assume your IT policy requires all custom scripts and applications to be installed in /opt/company/bin. Many classes, such as firewall and appserver from the previous example, will need some relevant content there. Each class needs to make sure that the directories exist before a script can be deployed inside it. This will be implemented by including a component class that wraps the file resources of the directory tree:

class scripts_directory { 
  file { [ '/opt/company/', '/opt/company/bin' ]: 
    ensure => 'directory', 
    owner  => 'root', 
    group  => 'root', 
    mode   => '0644', 
  } 
}

The component class is a pretty precise concept. However, as you have seen in the previous section about the more powerful classes, the whole range of possible class designs forms a fine-grained scale between the presented examples. All manifests you write will likely comprise more than a few classes. The best way to get a feeling for the best practices is to just go ahead and use classes to build the manifests you need.

Next, let's look at some uses for defined types.

For all their apparent similarity to classes, defined types are used in different ways. For example, the component class was described as wrapping a resource. This is accurate in a very specific context—the wrapped resource is a singleton, and it can only appear in one form throughout the manifest.

When wrapping a resource in a defined type instead, you end up with a variation on the respective resource type. The manifest can contain an arbitrary number of instances of the defined type, and each will wrap a distinct resource.

Here is yet another typical example from the many manifests out there: most users who make use of Puppet's file serving capabilities will want to wrap the file type at some point so that the respective URL need not be typed for each file:

define module_file(String $module) { 
  file { $name: 
    source => "puppet:///modules/${module}/${name}"
  }
}

This makes it easy to get Puppet to sync files from the master to the agent. The master copy must be properly placed in the named modules on the master:

module_file { '/etc/ntpd.conf': 
  module => 'ntp':
} 

This resource will make Puppet retrieve the ntp.conf file from the ntp module. The preceding declaration is more concise and less redundant than the fully written file resource with the Puppet URL (especially for the large number of files you might need to synchronize), which would resemble the following:

file { '/etc/ntpd.conf': 
  source => 'puppet:///modules/ntp/etc/ntpd.conf':
} 

For a wrapper such as module_file, which will likely be used very widely, you will want to make sure that it supports all attributes of the wrapped resource type. In this case, the module_file wrapper should accept all file attributes. For example, this is how you add the mode attribute to the wrapper type:

define module_file(
  String $module,
  Optional[String] $mode = undef
) { 
  if $mode != undef { 
    File { mode => $mode } 
  } 
  file { $name: 
    source => "puppet:///modules/${module}/${name}" 
  } 
}

The File { ... } block declares some default values for all file resource attributes in the same scope. The undef value is similar to Ruby's nil, and is a convenient parameter default value, because it is very unlikely that a user will need to pass it as an actual value for the wrapped resource.

File[$name] { mode => $mode }

Wrapping single resources with a defined type is useful, but sometimes you will want to add functionality beyond the resource type you are wrapping. At other times, you might wish for your defined type to unify a lot of functionality, just like the comprehensive classes from the beginning of the section.

For both scenarios, what you want to have is multiple resources in the body of your defined type. There is a classic example for this as well:

define user_with_key(
  String $key, 
  Optional[String] $uid = undef, 
  String $group = 'users'
) {
  user { $title: 
    ensure     => present
    gid        => $group, 
    managehome => true, 
  } -> 
  ssh_authorized_key { "key for ${title}": 
    ensure => present, 
    user   => $title, 
    type   => 'rsa', 
    key    => $key, 
  } 
}

This code allows you to create user accounts with authorized SSH keys in one resource declaration. This code sample has some notable aspects:

Some source code requires many repetitive tasks. Assume that your site uses a subsystem that relies on symbolic links at a certain location to enable configuration files, just like init does with the symlinks in rc2.d/ and its siblings, which point back to ../init.d/<service>.

A manifest that enables a large number of configuration snippets might look like this:

file { '/etc/example_app/conf.d.enabled/england': 
  ensure => 'link', 
  target => '../conf.d.available/england' 
}
file { '/etc/example_app/conf.d.enabled/ireland': 
  ensure => 'link', 
  target => '../conf.d.available/ireland' 
}
file { '/etc/example_app/conf.d.enabled/germany': 
  ensure => 'link', 
  target => '../conf.d.available/germany' 
  ... 
}

This is tiring to read and somewhat painful to maintain. In a C program, one would use a preprocessor macro that just takes the base name of both link and target and expands to the three lines of each resource description. Puppet does not use a preprocessor, but you can use defined types to achieve a similar result:

define example_app_config { 
  file { "/etc/example_app/conf.d.enabled/${name}": 
    ensure => 'link', 
    target => "../conf.d.available/${name}", 
  }
}

The define requires no arguments—it can rely solely on its resource name, so the preceding code can now be simplified to the following:

example_app_config {'england': }
example_app_config {'ireland': }
example_app_config {'germany': }
...

Alternatively, the following code is even more terse:

example_app_config { [ 'england', 'ireland', 'germany', ... ]: 
}

This array notation leads us to another use of defined types.

One of the more common scenarios in programming is the requirement to accept an array value from some source and perform some task on each value. Puppet manifests are not exempt from this.

Let's assume that the symbolic links from the previous example actually led to directories, and that each such directory would contain a subdirectory to hold optional links to regions. Puppet should manage those links as well.

Of course, after learning about the macro aspect of defined types, you would not want to add each of those regions as distinct resources to your manifest. However, you will need to devise a way to map region names to countries. Seeing as there is already a defined resource type for countries, there is a very direct approach to this: make the list of regions an attribute (or rather, a parameter) of the defined type:

define example_app_config (
  Array $regions = []
) {
  file { "/etc/example_app/conf.d.enabled/${name}":
    ensure => link,
    target => "../conf.d.available/${name}",
  }
  # to do: add functionality for $regions
}

Using the parameter is straightforward:

example_app_config { 'england':
  regions => [ 'South East', 'London' ],
}
example_app_config { 'ireland':
  regions => [ 'Connacht', 'Ulster' ],
}
example_app_config { 'germany':
  regions => [ 'Berlin', 'Bayern', 'Hamburg' ],
}
...

The actual challenge is putting these values to use. A naïve approach is to add the following to the definition of example_app_config:

file { $regions:
  path   => "/etc/example_app/conf.d.enabled/${title}/ regions/${name}",
  ensure => 'link',
  target => "../../regions.available/${name}";
}

However, this will not work. The $name variable does not refer to the title of the file resource that is being declared. It actually refers, just like $title, to the name of the enclosing class or defined type (in this case, the country name). Still, the actual construct will seem quite familiar to you. The only missing piece here is yet another defined type:

define example_app_region(String $country) { 
  file { "/etc/example_app/conf.d.enabled/${country}/regions/${name}":
    ensure => 'link', 
    target => "../../regions.available/${name}",
  } 
} 

The complete definition of the example_app_config defined type should look like this then:

define example_app_config(Array $regions = []) {
  file { "/etc/example_app/conf.d.enabled/${name}": 
    ensure => 'link', 
    target => "../conf.d.available/${name}", 
  } 
  example_app_region { $regions: 
    country => $name,
  } 
}

The outer defined type adapts the behavior of the example_app_region type to its respective needs by passing its own resource name as a parameter value.

[ 'england', 'ireland', 'germany' ].each |$country| { 
  file { "/etc/example_app/conf.d.enabled/${country}": 
    ensure => 'link', 
    target => "../conf.d.available/${country}",
  } 
} 

$region_data = { 
  'england' => [ 'South East', 'London' ], 
  'ireland' => [ 'Connacht', 'Ulster' ], 
  'germany' => [ 'Berlin', 'Bayern', 'Hamburg' ], 
}
$region_data.each |$country, $region_array| {
  $region_array.each |$region| {
    file { "/etc/example_app/conf.d.enabled/${country}/ regions/${region}":
        ensure => link,
        target => "../../regions.available/${region}",
    }
  }
}

By sorting all resources into classes, you make it unnecessary (for your co-workers or other collaborators) to know about each single resource. This is beneficial. You can think of the collection of classes and defined types as your interface. You would not want to read all manifests that anyone on your project ever wrote.

However, the encapsulation is inconvenient for passing resource events. Say you have some daemon that creates live statistics from your Apache logfiles. It should subscribe to Apache's configuration files so that it can restart if there are any changes (which might be of consequence to this daemon's operation). In another scenario, you might have Puppet manage some external data for a self-compiled Apache module. If Puppet updates such data, you will want to trigger a restart of the Apache service to reload everything.

Armed with the knowledge that there is a service, Service['apache2'], defined somewhere in the apache class, you can just go ahead and have your module data files notify that resource. It would work—Puppet does not apply any sort of protection to resources that are declared in foreign classes. However, it would pose a minor maintainability issue.

The reference to the resource is located far from the resource itself. When maintaining the manifest later, you or a coworker might wish to look at the resource when encountering the reference. In the case of Apache, it's not difficult to figure out where to look, but in other scenarios, the location of the reference target can be less obvious.

Besides, this approach will not work for the other scenario, in which your daemon needs to subscribe to configuration changes. You could blindly subscribe the central apache2.conf file, of course. However, this would not yield the desired results if the responsible class opted to do most of the configuration work inside snippets in /etc/apache2/conf.d.

Both scenarios can be addressed cleanly and elegantly by directing the notify or subscribe parameters at the whole class that is managing the entity in question:

file { '/var/lib/apache2/sample-module/data01.bin':
  source => '...',
  notify => Class['apache'],
}
service { 'apache-logwatch': 
  enable    => true, 
  subscribe => Class['apache'], 
} 

Of course, the signals are now sent (or received) indiscriminately—the file not only notifies Service['apache2'], but also every other resource in the apache class. This is usually acceptable, because most resources ignore events.

As for the logwatch daemon, it might refresh itself needlessly if some resource in the apache class needs a sync action. The odds for this occurrence depend on the implementation of the class. For ideal results, it might be sensible to relocate the configuration file resources into their own class so that the daemon can subscribe to that instead.

With your defined types, you can apply the same rules: subscribe to and notify them as required. Doing so feels quite natural, because they are declared like native resources anyway. This is how you subscribe several instances of the defined type, symlink:

$active_countries = [ 'England', 'Ireland', 'Germany' ]
service { 'example-app': 
  enable    => true, 
  subscribe => Symlink[$active_countries], 
}

Granted, this very example is a bit awkward, because it requires all symlink resource titles to be available in an array variable. In this case, it would be more natural to make the defined type instances notify the service instead:

symlink { [ 'England', 'Ireland', 'Germany' ]:
  notify => Service['example-app'],
}

This notation passes a metaparameter to a defined type. The result is that this parameter value is applied to all resources declared inside the define.

If a defined type wraps or contains a service or exec type resource, it can also be desirable to notify an instance of that define to refresh the contained resource. The following example assumes that the service type is wrapped by a defined type called protected_service:

file { '/etc/example_app/main.conf': 
  source => '...', 
  notify => Protected_service['example-app'], 
}

The notify and subscribe metaparameters are not the only ones that you can direct at classes and instances of defined types—the same holds true for their siblings, before and require. These allow you to define an order for your resources relative to classes, order instances of your defined types, and even order classes among themselves.

The latter works by virtue of the chaining operator:

include firewall
include loadbalancing
Class['firewall'] -> Class['loadbalancing']

The effect of this code is that all resources from the firewall class will be synchronized before any resource from the loadbalancing class, and failure of any resource in the former class will prevent all resources in the latter from being synchronized.

Because of these ordering semantics, it is actually quite wholesome to require a whole class. You effectively mark the resource in question as being dependent on the class. As a result, it will only be synchronized if the entire subsystem that the class models is successfully synchronized first.

Sadly, there is a rather substantial issue with both the ordering of containers and the distribution of refresh events: both will not transcend the include statements of further classes. Consider the following example:

class apache {
  include apache::service
  include apache::package
  include apache::config
}
file { '/etc/apache2/conf.d/passwords.conf':
  source  => '...', 
  require => Class['apache'], 
} 

I often mentioned how the comprehensive apache class models everything about the Apache server subsystem, and in the previous section, I went on to explain that directing a require parameter at such a class will make sure that Puppet only touches the dependent resource if the subsystem has been successfully configured.

This is mostly true, but due to the limitation concerning class boundaries, it doesn't achieve the desired effect in this scenario. The dependent configuration file should actually require the Package['apache'] package, declared in class apache::package. However, the relationship does not span multiple class inclusions, so this particular dependency will not be part of the resulting catalog at all.

Similarly, any refresh events sent to the apache class will have no effect—they are distributed to resources declared in the class's body (of which there are none), but are not passed on to included classes. Subscribing to the class will make no sense either, because any resource events generated inside the included classes will not be forwarded by the apache class.

The bottom line is that relationships to classes cannot be built in utter ignorance of their implementation. If in doubt, you need to make sure that the resources that are of interest are actually declared directly inside the class you are targeting.

There is a bright side to this as well. A more correct implementation of the Apache configuration file from the example explained would depend on the package, but would also synchronize itself before the service, and perhaps even notify it (so that Apache restarts if necessary). When all resources are part of the apache class and you want to adhere to the pattern of interacting with the container only, it would lead to the following declaration:

file { '/etc/apache2/conf.d/passwords.conf': 
  source  => '...', 
  require => Class['apache'], 
  notify  => Class['apache'], 
} 

This forms an instant dependency circle: the file resource requires all parts of the apache class to be synchronized before it gets processed, but to notify them, they must all be put after the file resource in the order graph. This cannot work. With the knowledge of the inner structure of the apache class, the user can pick metaparameter values that actually work:

file { '/etc/apache2/conf.d/passwords.conf':
  source  => '...', 
  require => Class['apache::package'], 
  notify  => Class['apache::service'], 
} 

For the curious the preceding code shows what the inner classes look like, roughly.

The other good news is that invoking defined types does not pose the same kind of issue that an include statement of a class does. Events are passed to resources inside defined types just fine, transcending an arbitrary number of stacked invocations. Ordering also works just as expected. Let's keep the example brief:

class apache { 
  virtual_host { 'example.net': ... }
  ... 
}

This apache class also creates a virtual host using the defined type, virtual_host. A resource that requires this class will implicitly require all resources from within this virtual_host instance. A subscriber to the class will receive events from those resources, and events directed at the class will reach the resources of this virtual_host.

There is another aspect that you should keep in mind whenever you are referencing a container type to build a relationship to it. The Puppet agent will have to build a dependency graph from this. This graph contains all resources as nodes and all relationships as edges. Classes and defined types get expanded to all their declared resources. All relationships to the container are expanded to relationships to each resource.

This is mostly harmless, if the other end of the relationship is a native resource. A file that requires a class with five declared resources leads to five dependencies. That does not hurt. It gets more interesting if the same class is required by an instance of a defined type that comprises three resources. Each of these builds a relationship to each of the class's resources, so you end up with 15 edges in the graph.

It gets even more expensive when a container invokes complex defined types, perhaps even recursively.

A more complex graph means more work for the Puppet agent, and its runs will take longer. This is especially annoying when running agents interactively during the debugging or development of your manifest. To avoid the unnecessary effort, consider your relationship declarations carefully, and use them only when they are really appropriate.

The architects of the Puppet language have devised two alternative approaches to solve the ordering issues. We will consider both, because you might encounter them in existing manifests. In new setups, you should always choose the latter variant.

class example_app { 
  anchor { 'example_app::begin': 
    notify => Class['example_app_config'], 
  } 
  include example_app_config 
  anchor { 'example_app::end': 
    require => Class['example_app_config'],
  } 
} 

class apache {
  contain apache::service
  contain apache::package
  contain apache::config
}

The consequence of allowing class parameters is almost obvious: you lose the singleton characteristic. Well, that's not entirely true either, but your freedom in declaring the class gets limited drastically.

Classes that define default values for all parameters can still be declared with the include statement. This can still be done an arbitrary number of times in the same manifest.

However, the resource-like declaration of class { 'name': } cannot appear more than once for any given class in the same manifest. This is in keeping with the rules for resources and should not be very surprising—after all, it would be very awkward to try and bind different values to a class's parameters in different locations throughout the manifest.

Things become very confusing when mixing include with the alternative syntax though. It is valid to include a class an arbitrary number of times after it has been declared using the resource-like notation. However, you cannot use the resource style declaration after a class has been declared using include. That's because the parameters are then determined to assume their default values, and a class { 'name': } declaration clashes with that.

In a nutshell, the following code works:

class { 'apache::config': }
include apache::config

However, the following code does not work:

include apache::config
class { 'apache::config': }

As a consequence, you effectively cannot add parameters to component classes, because the include statement is no longer safe to use in large quantities. Therefore, parameters are essentially only useful for comprehensive classes, which usually don't get included from different parts of the manifest.

Ever since class parameters have been available, some Puppet users have felt compelled to write (example) code that would make it a point to forgo the include keyword in favor of resource-like class declarations, such as this:

class apache {
  class { 'apache::service': }
  class { 'apache::package': }
  class { 'apache::config': }
}

Doing this is a very bad idea. We cannot stress this enough: one of the most powerful concepts about Puppet's classes is their singleton aspect—the ability to include a class in a manifest arbitrarily and without worrying about clashes with other code. The mentioned declaration syntax deprives you of this power, even when the classes in question don't support parameters.

The safest route is to use include whenever possible, and to avoid the alternate syntax whenever you can. In fact, Chapter 8, Separating Data from Code Using Hiera, introduces the ability to use class parameters without the resource-like class declaration. This way, you can rely solely on include, even when parameters are in play. These are the safest recommended practices to keep you out of trouble from incompatible class declarations.

For most modules, manifests form the most important part - the core functionality. The manifests consist of classes and defined types, which all share a namespace, rooted at the module name. For example, an ntp module will contain only classes and defines whose names start with the ntp:: prefix.

Many modules contain files that can be synced to the agent's filesystem. This is often used for configuration files or snippets. You have seen examples of this, but let's repeat them. A frequent occurrence in many manifests is file resources such as the following:

file { '/etc/ntp.conf': 
  source => 'puppet:///modules/ntp/ntp.conf', 
} 

The above resource references a file that ships with a hypothetical ntp module. It has been prepared to provide generally suitable configuration data. However, there is often a need to tweak some parameters inside such a file, so that the node manifests can declare customized config settings for the respective agent. The tool of choice for this is templates, which will be discussed in the next chapter.

Another possible component of a module that you have already read about is Custom Facts—code that gets synchronized to the agent and runs before a catalog is requested, so that the output becomes available as facts about the agent system.

These facts are not the only Puppet plugins that can be shipped with modules. There are also parser functions (also called custom functions), for one. These are actual functions that you can use in your manifests. In many situations, they are the most convenient way, if not the only way, to build some specific implementations.

The final plugin type that has also been hinted at in an earlier chapter is the custom native types and providers, which are conveniently placed in modules as well.

All the mentioned components need to be located in specific filesystem locations for the master to pick them up. Each module forms a directory tree. Its root is named after the module itself. For example, the ntp module is stored in a directory called ntp/.

All manifests are stored in a subdirectory called manifests/. Each class and defined type has its own respective file. The ntp::package class will be found in manifests/package.pp, and the defined type called ntp::monitoring::nagios will be found in manifests/monitoring/nagios.pp. The first particle of the container name (ntp) is always the module name, and the rest describes the location under manifests/. You can refer to the module tree in the following paragraphs for more examples.

The manifests/init.pp file is special. It can be thought of as a default manifest location, because it is looked up for any definition from the module in question. Both the examples that were just mentioned can be put into init.pp and will still work. Doing this makes it harder to locate the definitions, though.

In practice, init.pp should only hold one class, which is named after the module (such as the ntp class), if your module implements such a class. This is a common practice, as it allows the manifests to use a simple statement to tap the core functionality of the module:

include ntp

You can refer to the Modules' best practices section for some more notes on this subject.

The files and templates that a module serves to the agents are not this strictly sorted into specific locations. It is only important that they be placed in the files/ and templates/ subdirectories, respectively. The contents of these subtrees can be structured to the module author's liking, and the manifest must reference them correctly. Static files should always be addressed through URLs, such as these:

puppet:///modules/ntp/ntp.conf
puppet:///modules/my_app/opt/scripts/find_my_app.sh

These files are found in the corresponding subdirectories of files/:

.../modules/ntp/files/ntp.conf
.../modules/my_app/files/opt/scripts/find_my_app.sh

The modules prefix in the URI is mandatory and is always followed by the module name. The rest of the path translates directly to the contents of the files/ directory. There are similar rules for templates. You can refer to Chapter 6, Leveraging the Full Toolset of the Language, for the details.

Finally, all plugins are located in the lib/ subtree. Custom facts are Ruby files in lib/facter/. Parser functions are stored in lib/puppet/parser/functions/, and for custom resource types and providers, there is lib/puppet/type/ and lib/puppet/provider/, respectively. This is not a coincidence; these Ruby libraries are looked up by the master and the agent in the according namespaces. There are examples for all these components later in this chapter.

In short, following are the contents of a possible module in a tree view:

/opt/puppetlabs/code/environments/production/modules/my_app
  templates      # templates are covered in the next chapter
  files
    subdir1      # puppet:///modules/my_app/subdir1/<filename>
    subdir2      # puppet:///modules/my_app/subdir2/<filename>
      subsubdir  # puppet:///modules/my_app/subdir2/subsubdir/...
  manifests
    init.pp      # class my_app             is defined here
    params.pp    # class my_app::params     is defined here
    config
      detail.pp  # my_app::config::detail   is defined here
      basics.pp  # my_app::config::basics   is defined here
  lib
    facter         # contains .rb files with custom facts
    puppet
      functions    # contains .rb files with Puppet 4 functions
      parser
        functions  # contains .rb files with parser functions
      type         # contains .rb files with custom types
      provider     # contains .rb files with custom providers

A module can and should include documentation. The Puppet master does not process any module documentation by itself. As such, it is largely up to the authors to decide how to structure the documentation of the modules that are created for their specific site only. That being said, there are some common practices and it's a good idea to adhere to them. Besides, if a module should end up being published on the Forge, appropriate documentation should be considered mandatory.

For many modules, the main focus of the documentation is centered on the README file, which is located right in the module's root directory. It is customarily formatted in Markdown as README.md or README.markdown. The README file should contain explanations, and often, there is a reference documentation as well.

Puppet DSL interfaces can also be documented right in the manifest, in the rdoc and YARD format. This applies to classes and defined types:

# Class: my_app::firewall
#
# This class adds firewall rules to allow access to my_app.
#
# Parameters: none
class my_app::firewall {
  # class code here
}

You can generate HTML documentation (including navigation) for all your modules using the puppet doc subcommand. This practice is somewhat obscure, so it won't be discussed here in great detail. However, if this option is attractive to you, we encourage you to peruse the documentation.

The following command is a good starting point:

puppet help doc

Another useful resource is the puppetlabs-strings module (https://forge.puppetlabs.com/puppetlabs/strings), which will eventually supersede puppet doc.

Plugins are documented right in their Ruby code. There are examples for this in the following sections.

Puppet uses the production environment by default. This and the other environments are expected to be located in /opt/puppetlabs/code/environments. You can override this default by setting the environmentpath option in puppet.conf:

[main]
environmentpath = /etc/local/puppet/environments

With Puppet 3, you are required to set this option, as it is not yet the default. It is available from version 3.5. The earlier versions needed to configure the environments right in puppet.conf with their respective manifest and modulepath settings. These work just like the settings from environment.conf:

# in puppet.conf (obsolete since 4.0)
[testing]
manifest = /etc/puppet/environments/testing/manifests
modulepath = /etc/puppet/environments/testing/modules

For the special production environment, these Puppet setups use the manifest and modulepath settings from the [main] or [master] section. The old default configuration had the production environment look for the manifests and the modules right in /etc/puppet:

# Obsolete! Works only with Puppet 3.x!
[main]
manifest = /etc/puppet/site.pp
modulepath = /etc/puppet/modules

The sites that operate like this today should be migrated to the aforementioned directory environments in /opt/puppetlabs/code/environments or similar locations.

Downloading existing modules is very common. Puppet Labs hosts a dedicated site for sharing and obtaining the modules - the Puppet Forge. It works just like RubyGems or CPAN and makes it simple for the user to retrieve a given module through a command-line interface. In the Forge, the modules are fully named by prefixing the actual module name with the author's name, such as puppetlabs-stdlib or ffrank-constraints.

The puppet module install command installs a module in the active environment:

root@puppetmaster# puppet module install puppetlabs-stdlib

The current release of the stdlib module (authored by the user puppetlabs) is downloaded from the Forge and installed in the standard modules' location. This is the first location in the current environment's modulepath, which is usually the modules subdirectory. Specifically, the modules will most likely end up in the environments/production/modules directory.

With all the current versions of Puppet, you should make it a habit to put all the manifest code into modules, with only the following few exceptions:

This section provides details on how to organize your manifests accordingly. It also advises some design practices and strategies in order to test the changes to the modules.

You might find some manifests in very old installations that gather lots of manifest files in one or more directories and use the import statements in the site.pp file, such as:

import '/etc/puppet/manifests/custom/*.pp'

All classes and defined types in these files are then available globally.

It is far more efficient to give meaningful names to the classes and defined types so that Puppet can look them up in the collection of modules. The scheme has been discussed in an earlier section already, so let's just look at another example where the Puppet compiler encounters a class name, such as:

include ntp::server::component::watchdog

Puppet will go ahead and locate the ntp module in all the configured module locations of the active environment (path names in the modulepath setting). It will then try and read the ntp/manifests/server/component/watchdog.pp file in order to find the class definition. Failing this, it will try ntp/manifests/init.pp.

This makes compilation very efficient. Puppet dynamically identifies the required manifest files and includes only those for parsing. It also aids code inspection and development, as it is abundantly clear where you should look for specific definitions.

Each module should ideally serve a specific purpose. On a site that relies on Puppet to manage a diverse server infrastructure, there are likely modules for each respective service, such as apache, ssh, nagios, nginx, and so forth. There can also be site-specific modules such as users or shell_settings if the operations require this kind of fine-grained control. Such customized modules are sometimes just named after the group or the company that owns them.

The ideal granularity depends on the individual requirements of your setup. What you generally want to avoid are modules with names such as utilities or helpers, which serve as a melting pot for ideas that don't fit in any of the existing modules. Such a lack of organization can be detrimental to discipline and can lead to chaotic modules that include definitions that should have become their own respective modules instead.

Adding more modules is cheap. A module generally incurs no cost for the Puppet master operation, and your user experience will usually become more efficient with more modules, not less so. Of course, this balance can tip if your site imposes special documentation or other handling prerequisites on each module. Such rulings must then be weighed into the decisions about module organization.

Depending on the size of your agent network, some or many of your modules can be used by a large variety of nodes. Despite these commonalities, these nodes can be quite different from one another. A change to a central module, such as ssh or ntp, which are likely used by a large number of agents, can have quite extensive consequences.

The first and the most important tool for testing your work is the --noop option for Puppet. It works for puppet agent, as well as puppet apply. If it is given on the command line, Puppet will not perform any necessary sync actions, and merely present the respective line of output to you instead. There is an example of this in Chapter 1, Writing Your First Manifests.

When using a master instead of working locally with puppet apply, a new problem arises, though. The master is queried by all your agents. Unless all the agents are disabled while you are testing a new manifest, it is very likely that one will check in and accidentally run the untested code.

It is very important to guard against such uncontrolled manifest applications. A small mistake can damage a number of agent machines in a short time period. The best way to go about this is to define multiple environments on the master.

root@agent# puppet agent --test --noop --env testing

Module names should be concise and to the point. If you manage a specific piece of software, name your module after it - apache, java, mysql, and so forth. Avoid verbs such as install_cacti or manage_cacti. If your module name does need to consist of several words (because the target subsystem has a long name), they should be divided by underscore characters. Spaces, dashes, and other non-alphanumeric characters are not allowed.

In our example, the module should just be named cacti.

To use your own module, you don't need to make it available for installation through puppet module. For that, you will need to upload the module to the Forge first, which will require quite some additional effort. Luckily, a module will work just fine without all this preparation, if you just put the source code in the proper location on your master.

To create your own cacti module, create the basic directories:

root@puppetmaster# mkdir -p /opt/puppetlabs/code/environments/testing/cacti/{manifests,files}

Don't forget to synchronize all the changes to production once the agents use them.

Most modules perform all of their work through their manifests.

When planning the classes for your module, it is most straightforward to think about how you would like to use the finished module. There is a wide range of possible interface designs. The de facto standard stipulates that the managed subsystem is initialized on the agent system by including the module's main class - the class that bears the same name as the module and is implemented in the module's init.pp file.

For our Cacti module, the user should use the following:

include cacti

As a result, Puppet would take all the required steps in order to install the software and if necessary, perform any additional initialization.

Start by creating the cacti class and implementing the setup in the way you would from the command line, replacing the commands with appropriate Puppet resources. On a Debian system, installing the cacti package is enough. Other required software is brought in through the dependencies (completing the LAMP stack), and after the package installation, the interface becomes available through the web URI /cacti/ on the server machine:

# …/modules/cacti/manifests/init.pp
class cacti {
  package { 'cacti':
    ensure => installed,
  }
}

Your module is now ready for testing. Invoke it from your agent's manifest in site.pp or nodes.pp of the testing environment:

node 'agent' {
  include cacti
}

Apply it on your agent directly:

root@agent# puppet agent --test --environment testing

This will work on Debian, and Cacti is reachable via http://<address>/cacti/.

It's unfortunate that the Cacti web interface will not come up when the home page is requested through the / URI. To enable this, give the module the ability to configure an appropriate redirection. Prepare an Apache configuration snippet in the module in /opt/puppetlabs/code/environments/testing/cacti/files/etc/apache2/conf.d/cacti-redirect.conf:

# Do not edit this file – it is managed by Puppet!
RedirectMatch permanent ^/$ /cacti/

It makes sense to add a dedicated class that will sync this file to the agent machine:

# …/modules/cacti/manifests/redirect.pp
class cacti::redirect {
  file { '/etc/apache2/conf.d/cacti-redirect.conf': 
    ensure  => file, 
    source  => 'puppet:///modules/cacti/etc/apache2/conf.d/cacti-redirect.conf',
    require => Package['cacti']; 
  }
}

$puppet_warning = '# Do not edit – managed by Puppet!'
$line = 'RedirectMatch permanent ^/$ /cacti/'
file { '/etc/apache2/conf.d/cacti-redirect.conf': 
  ensure  => file, 
  content => "${puppet_warning}\n${line}\n", 
} 

The module now allows the user to include cacti::redirect in order to get this functionality. This is not a bad interface as such, but this kind of modification is actually well-suited to become a parameter of the cacti class:

class cacti($redirect = true ) {
  if $redirect {
    contain cacti::redirect
  }
  package { 'cacti':
    ensure => installed,
  }
}

The redirect is now installed by default when a manifest uses include cacti. If the web server has other virtual hosts that serve things that are not Cacti, this might be undesirable. In such cases, the manifest will declare the class with this following parameter:

class { 'cacti': redirect => false }

Speaking of best practices, most modules will also separate the installation routine into a class of its own. In our case, this is hardly helpful, because the installation status is ensured through a single resource, but let's do it anyway:

class cacti( $redirect = true ) {
  contain cacti::install
  if $redirect {
    contain cacti::redirect
  }
}

It's sensible to use contain here in order to make the Cacti management a solid unit. The cacti::install class is put into a separate install.pp manifest file:

# …/modules/cacti/manifests/install.pp
class cacti::install {
  package { 'cacti':
    ensure => 'installed'
  }
}

On Debian, the installation process of the cacti package copies another Apache configuration file to /etc/apache2/conf.d. Since Puppet performs a normal apt installation, this result will be achieved. However, Puppet does not make sure that the configuration stays in this desired state.

It is prudent to add a file resource to the manifest that keeps the configuration snippet in its post-installation state. Usually with Puppet, this will require you to copy the config file contents to the module, just like the redirect configuration is in a file on the master. However, since the Debian package for Cacti includes a copy of the snippet in /usr/share/doc/cacti/cacti.apache.conf, you can instruct the agent to sync the actual configuration with that. Perform this in yet another de facto standard for modules - the config class:

# …/modules/cacti/manifests/config.pp
class cacti::config { 
  file { '/etc/apache2/conf.d/cacti.conf': 
    mode   => '0644', 
    source => '/usr/share/doc/cacti/cacti.apache.conf' 
  }
}

This class should be contained by the cacti class as well. Running the agent again will now have no effect, because the configuration is already in place.

You have now created several classes that compartmentalize the basic installation and configuration work for your module. Classes lend themselves very well to implement global settings that are relevant for the managed software as a whole.

However, just installing Cacti and making its web interface available is not an especially powerful capability - after all, the module does little beyond what a user can achieve by installing Cacti through the package manager. The much greater pain point with Cacti is that it usually requires configuration via its web interface; adding servers as well as choosing and configuring graphs for each server can be an arduous task and require dozens of clicks per server, depending on the complexity of your graphing needs.

This is where Puppet can be the most helpful. A textual representation of the desired states allows for quick copy-and-paste repetition and name substitution through regular expressions. Better yet, once there is a Puppet interface, the users can devise their own defined types in order to save themselves from the copy and paste work.

Speaking of defined types, they are what is required for your module to allow this kind of configuration. Each machine in Cacti's configuration should be an instance of a defined type. The graphs can have their own type as well.

As with the implementation of the classes, the first thing you always need to ask yourself is how this task would be done from the command line.

Cacti comes with a set of CLI scripts. The Debian package makes these available in /usr/share/cacti/cli. Let's discover these while we step through the implementation of the Puppet interface. The goals are defined types that will effectively wrap the command-line tools so that Puppet can always maintain the defined configuration state through appropriate queries and update commands.

# …/modules/cacti/manifests/device.pp
define cacti::device ($ip) {
  $cli = '/usr/share/cacti/cli'
  $options = "--description='${name}' --ip='${ip}'"
  exec { "add-cacti-device-${name}":
    command => "${cli}/add_device.php ${options}",
    require => Class['cacti'],
}

$search = "sed 1d | cut -f4- | grep -q '^${name}\$'"
exec { "add-cacti-device-${name}":
  command => "${cli}/add_device.php ${options}",
  path    => '/bin:/usr/bin',
  unless  => "${cli}/add_graphs.php --list-hosts | ${search}",
  require => Class[cacti],
}

# in manifests/nodes.pp
node 'agent' {
  include cacti
  cacti::device { 'Puppet test agent (Debian 7)': 
    ip => $ipaddress,
  }
}

define cacti::device(
  $ip,
  $ping_method='icmp')
{
  $cli = '/usr/share/cacti/cli'
  $base_opt = "--description='${name}' --ip='${ip}'"
  $ping_opt = "--ping_method=${ping_method}"
  $options = "${base_opt} ${ping_opt}"
  $search = "sed 1d | cut -f4- | grep -q '^${name}\$'"
  exec { "add-cacti-device-${name}":
    command => "${cli}/add_device.php ${options}",
    path    => '/bin:/usr/bin',
    unless  => "${cli}/add_graphs.php --list-hosts  | ${search}",
    require => Class[cacti], 
  }
}

file { '/usr/share/cacti/cli/remove_device.php':
    mode    => 755,
    source  => 'puppet:///modules/cacti/usr/share/cacti/cli/remove_device.php',
    require => Package['cacti'],
}

define cacti::device(
  $ensure='present',
  $ip,
  $ping_method='icmp')
{
  $cli = '/usr/share/cacti/cli'
  $search = "sed 1d | cut -f4- | grep -q '^${name}\$'"
  case $ensure {
  'present': {
    # existing cacti::device code goes here
  }
  'absent': {
    $remove = "${cli}/remove_device.php"
    $get_id = "${remove} --list-devices | awk -F'\\t' '\$4==\"${name}\" { print \$1 }'"
    exec { "remove-cacti-device-${name}":
        command => "${remove} --device-id=\$( ${get_id} )",
        path    => '/bin:/usr/bin',
        onlyif  => "${cli}/add_graphs.php --list-hosts | ${search}",
        require => Class[cacti],
      }
    }
  }
}

#!/bin/bash
DEVICE_DESCR=$1
GRAPH_DESCR=$2
DEVICE_ID=` #scriptlet to retrieve numeric device ID`
GRAPH_ID=`  #scriptlet to retrieve numeric graph ID`
GRAPH_TYPE=`#scriptlet to determine the graph type`
/usr/share/cacti/cli/add_graphs.php \
  --graph-type=$GRAPH_TYPE \
  --graph-template-id=$GRAPH_ID \
  --host-id=$DEVICE_ID

define cacti::graph($device,$graph=$name) {
  $add = '/usr/local/bin/cacti-add-graph'
  $find = '/usr/local/bin/cacti-find-graph'
  exec { "add-graph-${name}-to-${device}":
    command => "${add} '${device}' '${graph}'",
    path    => '/bin:/usr/bin',
    unless  => "${find} '${device}' '${graph}'",
  }
}

The reusable classes and defines give manifests that use your module much more expressive power. Installing and configuring Cacti now works concisely, and the manifest to do this becomes very readable and maintainable.

It's time to tap into the even more powerful aspect of modules - Puppet plugins. The different types of plugins are custom facts (which were discussed in Chapter 3, A Peek under the Hood – Facts, Types, and Providers), parser functions, resource types, and providers. All these plugins are stored in the modules on the master and get synchronized to all the agents. The agent will not use the parser functions (they are available to the users of puppet apply on the agent machine once they are synchronized, however); instead the facts and resource types do most of their work on the agent. Let's concentrate on the types and providers for now - the other plugins will be discussed in dedicated sections later.

While the custom resource types are functional on both the master and the agent, the provider will do all its work on the agent side. Although the resource types also perform mainly through the agent, they have one effect on the master: they enable manifests to declare resources of the type. The code not only describes what properties and parameters exist, but it can also include the validation and transformation code for the respective values. This part is invoked by the agent. Some resource types even do the synchronization and queries themselves, although there is usually at least one provider that takes care of this.

In the previous section, you implemented a defined type that did all its synchronization by wrapping some exec resources. By installing binaries and scripts through Puppet, you can implement almost any kind of functionality this way and extend Puppet without ever writing one plugin. This does have some disadvantages, however:

In short, you pay a price, both in terms of usability and performance. Consider the cacti::device type. For each declared resource, Puppet will have to run an exec resource's unless query on each run (or onlyif when ensure => absent is specified). This consists of one call to a PHP script (which can be expensive) as well as several core utilities that have to parse the output. On a Cacti server with dozens or hundreds of managed devices, these calls add up and make the agent spend a lot of time forking off and waiting for these child processes.

Consider a provider, on the other hand. It can implement an instances hook, which will create an internal list of configured Cacti devices once during initialization. This requires only one PHP call in total, and all the processing of the output can be done in the Ruby code directly inside the agent process. These savings alone will make each run much less expensive: resources that are already synchronized will incur no penalty, because no additional external commands need to be run.

Let's take a quick look at the agent output before we go ahead and implement a simple type/provider pair. Following is the output of the cacti::device type when it creates a device:

Notice: /Stage[main]/Main/Node[agent]/Cacti::Device[Agent_VM_Debian_7]/Exec[add-cacti-device-Agent_VM_Debian_7]/returns: executed successfully

The native types express such actions in a much cleaner manner, such as the output from a file resource:

Notice: /Stage[main]/Main/File[/usr/local/bin/cacti-search-graph]/ensure: created

Puppet::Type.newtype(:cacti_device) do
  @doc = <<-EOD
    Manages Cacti devices.
    EOD
end

ensurable

require 'ipaddr'
newparam(:ip) do
  desc "The IP address of the device."
  isrequired
  validate do |value|
    begin
      IPAddr.new(value)
    rescue ArgumentError
      fail "'#{value}' is not a valid IP address"
    end
  end
  munge do |value|
    value.downcase
  end
end

newparam(:ping_method) do
  desc "How the device's reachability is determined.
    One of `tcp` (default), `udp` or `icmp`."
  validate do |value|
    [ :tcp, :udp, :icmp ].include?(value.downcase.to_sym)
  end
  munge do |value|
    value.downcase.to_sym
  end
  defaultto :tcp
end

exec { '/bin/true': }
# same effect:
exec { 'some custom name': command => '/bin/true' }

newparam(:name) do
  desc "The name of the device."
  #isnamevar # → commented because automatically assumed
end

Puppet::Type.type(:cacti_device).provide(
  :cli,
  :parent => Puppet::Provider
  ) do
end

commands :php        => 'php'
commands :add_device => '/usr/share/cacti/cli/add_device.php'
commands :add_graphs => '/usr/share/cacti/cli/add_graphs.php'
commands :rm_device  => '/usr/share/cacti/cli/remove_device.php'

def create
  args = []
  args << "--description=#{resource[:name]}"
  args << "--ip=#{resource[:ip]}"
  args << "--ping_method=#{resource[:ping_method]}"
  add_device(*args)
end

args << "--description='#{resource[:name]}'"

def destroy
  rm_device("--device-id=#{@property_hash[:id]}")
end

def exists?
  self.class.instances.find do |provider|
    provider.name == resource[:name]
  end
end

def self.instances
  return @instances ||= add_graphs("--list-hosts").
    split("\n").
    drop(1).
    collect do |line|
      fields = line.split(/\t/, 4)
      Puppet.debug "prefetching cacti_device #{fields[3]} " +
                   "with ID #{fields[0]}"
      new(:ensure => :present,
          :name   => fields[3],
          :id     => fields[0])
    end
end

def self.prefetch(resources)
  instances.each do |provider|
    if res = resources[provider.name]
      res.provider = provider
    end
  end
end

node 'agent' {
  include cacti
    cacti_device { 'Puppet test agent (Debian 7)':
      ensure => present,
      ip     => $ipaddress,
  }
}

autorequire :package do
  catalog.resource(:package, 'cacti')
end

When facts were introduced in Chapter 3, A Peek under the Hood – Facts, Types, and Providers, you got a small tour of the process of creating your own custom facts. We hinted at modules at that point, and now, we can take a closer look at how the fact code is deployed, using the example of the Cacti module. Let's focus on native Ruby facts - they are more portable than the external facts. As the latter are easy to create, there is no need to discuss them in depth here.

Facts are part of the Puppet plugins that a module can contain, just like the types and providers from the previous sections. They belong in the lib/facter/ subtree. For the users of the cacti module, it might be helpful to learn which graph templates are available on a given Cacti server (once the graph management is implemented, that is). The complete list can be passed through a fact. The following code in cacti/lib/facter/cacti_graph_templates.rb will do just this job:

Facter.add(:cacti_graph_templates) do
  setcode do
    cmd = '/usr/share/cacti/cli/add_graphs.php'
    Facter::Core::Execution.exec("#{cmd} --list-graph-templates").
      split("\n").
      drop(1).
      collect do |line|
        line.split(/\t/)[1]
      end
  end
end

The code will call the CLI script, skip its first line of output, and join the values from the second column of each remaining line in a list. Manifests can access this list through the global $cacti_graph_templates variable, just like any other fact.

Functions can be of great help in keeping your manifest clean and maintainable, and some tasks cannot even be implemented without resorting to a Ruby function.

A frequent use of the custom functions (especially in Puppet 3) is input validation. You can do this in the manifest itself, but it can be a frustrating exercise because of the limitations of the language. The resulting Puppet DSL code can be hard to read and maintain. The stdlib module comes with the validate_X functions for many basic data types, such as validate_bool. Typed parameters in Puppet 4 and later versions make this more convenient and natural, because for the supported variable types, no validation function is needed anymore.

As with all the plugins, the functions need not be specific to the module's domain, and they instantly become available for all the manifests. Point in case is the cacti module that can use the validation functions for the cacti::device parameters. Checking whether a string contains a valid IP address is not at all specific to Cacti. On the other hand, checking whether ping_method is one of those that Cacti recognizes is not that generic.

To see how it works, let's just implement a function that does the job of the validate and munge hooks from the custom cacti_device type for the IP address parameter of cacti::device. This should fail the compilation if the address is invalid; otherwise, it should return the unified address value:

module Puppet::Parser::Functions
  require 'ipaddr'
  newfunction(:cacti_canonical_ip, :type => :rvalue) do |args|
    ip = args[0]
    begin
      IPAddr.new(ip)
    rescue ArgumentError
      raise "#{@resource.ref}: invalid IP address '#{ip}'"
    end
    ip.downcase
  end
end

In the exception message, @resource.ref is expanded to the textual reference of the offending resource type instance, such as Cacti::Device[Edge Switch 03].

The following example illustrates the use of the function in the simple version of cacti::device without the ensure parameter:

define cacti::device($ip) {
  $cli = '/usr/share/cacti/cli'
  $c_ip = cacti_canonical_ip(${ip})
  $options = "--description='${name}' --ip='${c_ip}'"
  exec { "add-cacti-device-${name}":
    command => "${cli}/add_device.php ${options}",
    require => Class[cacti],
  }
}

The manifest will then fail to compile if an IP address has (conveniently) transposed digits:

ip => '912.168.12.13'

IPv6 addresses will be converted to all lowercase letters.

Sadly, our Cacti module is very specific to the Debian package. It expects to find the CLI at a certain place and the Apache configuration snippet at another. These locations are most likely specific to the Debian package. It will be useful for the module to work on the Red Hat derivatives as well.

The first step is to get an overview of the differences by performing a manual installation. I chose to test this with a virtual machine running Fedora 18. The basic installation is identical to Debian, except using yum instead of apt-get, of course. Puppet will automatically do the right thing here. The puppet::install class also contains a CLI file, though. The Red Hat package installs the CLI in /var/lib/cacti/cli rather than /usr/share/cacti/cli.

If the module is supposed to support both platforms, the target location for the remove_device.php script is no longer fixed. Therefore, it's best to deploy the script from a central location in the module, while the target location on the agent system becomes a module parameter, if you will. Such values are customarily gathered in a params class:

# …/cacti/manifests/params.pp
class cacti::params {
  case $osfamily {
    'Debian': {
      $cli_path = '/usr/share/cacti/cli'
    }
    'RedHat': {
      $cli_path = '/var/lib/cacti/cli'
    }
    default: {
      fail "the cacti module does not yet support the ${osfamily} platform"
    }
  }
}

It is best to fail the compilation for unsupported agent platforms. The users will have to remove the declaration of the cacti class from their module rather than have Puppet try untested installation steps that most likely will not work (this might concern Gentoo or a BSD variant).

Classes that need to access the variable value must include the params class:

class cacti::install {
  include cacti::params
  file { 'remove_device.php':
    ensure => file,
    path   => "${cacti::params::cli_path}/remove_device.php',
    source => 'puppet:///modules/cacti/cli/remove_device.php',
    mode   => '0755',
  }
}

Similar transformations will be required for the cacti::redirect class and the cacti::config class. Just add more variables to the params class. This is not limited to the manifests, either; the facts and providers must behave in accordance with the agent platform as well.

You will often see that the params class is inherited rather than included:

class cacti($redirect = ${cacti::params::redirect})
  inherits cacti::params{
  # ...
}

This is done because an include statement in the class body won't allow the use of variable values from the params class as the class parameter's default values, such as the $redirect parameter in this example.

The portability practices are often not required for your own custom modules. In the ideal case, you won't use them on more than one platform. The practice should be considered mandatory if you intend to share them on the Forge, though. For most of your Puppet needs, you will not want to write modules anyways, but download existing solutions from the Forge instead.

When navigating to a module's details in the Forge, you are presented with its README file. An empty or very sparse documentation speaks of little care taken by the module author. A sample manifest in the README file is often a good starting point in order to put a module to work quickly.

If you are looking for a module that will enhance your agents through additional resource types and providers, look for the Types tab on the module details page. It can also be enlightening to click on the Project URL link near the top of the module description. This usually leads to GitHub. Here, you can conveniently browse not only the plugins in the lib/ subtree, but also get a feel of how the module's manifests are structured.

Another sign of a carefully maintained module are unit tests. These are found in the spec/ subtree. This tree does exist for most of the Forge modules. It tends to be devoid of actual tests, though. There can be test code files for all the classes and the defined types that are part of the module's manifest - these are typically in the spec/classes/ and spec/defines/ subdirectories, respectively. For plugins, there will ideally be unit tests in spec/unit/ and spec/functions/.

Some README files of the modules contain a small greenish tag saying Build passing. This can turn red on occasions, stating Build failing. These modules use the Travis CI through GitHub, so they are likely to have at least a few unit tests.

Templates make short work of such scenarios. If you are familiar with ERB templates already, you can safely skip to the next section. If you know your way around PHP or JSP, you will quickly get the hang of ERB—it's basically the same but with Ruby inside the code tags. The following template will produce Hello,world! three times:

<% ( 1 .. 3 ).each do %>
Hello, world!
<% end %>

This template will also produce lots of empty lines, because the text between the <% and %> tags gets removed from the output but the final line breaks do not. To make the ERB engine do just that, change the closing tag to -%>:

<% ( 1 .. 3 ).each do -%>
Hello, world!
<% end -%>

This example is not very helpful for configuration files, of course. To include dynamic values in the output, enclose Ruby expressions in a <%=tag pair:

<% ( 1 .. 3 ).each do |index| -%>
Hello, world #<%= index %> !
<% end -%>

Now, the iterator value is part of each line of the output. You can also use member variables that are prefixed with @.

These variables are populated with the values from the Puppet manifest variables:

<IfModule mpm_worker_module>
ServerLimit         <%= @apache_server_limit %>
StartServers        <%= @apache_start_servers %>
MaxClients          <%= @apache_max_clients %>
</IfModule>
<% @apache_ports.each do |port| -%>
Listen <%= port %>
NameVirtualHost *:<%= port %>
<% end -%>

Variables that are used in a template must be defined in the same scope or scopes from which the template is used. The next section explains how this works.

In Puppet 3.x, variable values are mostly strings, arrays, or hashes. To write efficient templates, it is helpful to occasionally glance at the methods available for the respective Ruby classes. In Puppet 4, variables have more diverse values.

There are several ways to use Puppet variables in templates:

Templates have their own place in modules. You can place them freely in the templates/ subtree of the module. The template function locates them using a simple descriptor:

template('cacti/apache/cacti.conf.erb')

This expression evaluates the content of the template found in modules/cacti/templates/apache/cacti.conf.erb. The first path element (without a leading slash) is the module name. The rest of the path gets translated to the templates/ tree in the module. The function is commonly used to generate the value of a file resource's content property:

file { '/etc/apache2/conf.d/cacti.conf': 
  content => template('cacti/apache/cacti.conf.erb'), 
} 

Many templates expect some variables to be defined in their scope. The easiest way to make sure that this happens is to wrap the respective file resource in a parameterized container. Files that are singletons with a well-known name, such as /etc/ssh/sshd_config, should be managed through a parameterized class. Configuration items that can inhabit multiple files, such as /etc/logrotate.d/* or /etc/apache2/conf.d/*, are well suited to be wrapped in defined types:

define logrotate::conf(
  String $pattern,
  Integer $max_days=7,
  Array $options=[]
) {
  file { "/etc/logrotate.d/$name": 
    mode    => '0644', 
    content => template('logrotate/config-snippet.erb') 
  }
}

In the preceding example, the template uses the parameters as @pattern, @max_days, and @options, respectively.

For a quick and dirty string transformation of your data, you can also use the inline_template function in your manifest. This is often found on the right-hand side of a variable assignment:

$comma_seperated_list = inline_template('<%= @my_array * "," %>')

This example assumes that the $my_array Puppet variable holds an array value.

When using templates, both through the template and inline_template functions, be aware that each invocation implies a performance penalty for your Puppet master. During the compilation of the catalog, Puppet must initialize the ERB engine for any template it encounters. The ERB evaluation happens in an individual environment that is derived from the respective scope of the template function invocation.

It is, therefore, not even important how complex your templates are. If your manifest requires frequent expansion of a very short template, it generates an enormous overhead for each initialization. Especially in the case of an easy inline_template function, such as the one mentioned previously, it can be worthwhile to invest some more effort in creating a parser function instead, as seen in Chapter 5, Extending Your Puppet Infrastructure with Modules. A function can perform variable value transformation without incurring the cumulative penalty.

On the bright side, using templates is quite economic for the agent, who receives the whole textual file content right inside the catalog. There is no need to make an additional call to the master and retrieve file metadata. On a high-latency network, this can be a noticeable saving.

There is no silver bullet here. Don't let the performance implications deter you from turning specific configuration files into templates. Template-based solutions will often make your module more maintainable, which will usually offset performance implications—hardware is constantly getting cheaper, after all. Just don't be wasteful with frequent (and simple) expansions.

Instead of invoking the realize function, you can also rely on a different syntactic construct, which is the collector:

Yumrepo<| title == 'team_ninja_stable' |>

This is more flexible than the function call at the cost of a slight performance penalty. It can be used as a reference to the realized resource(s) in certain contexts. For example, you can add ordering constraints with the chaining operator:

Yumrepo<| title == 'team_ninja_stable' |> -> Class['...']

It is even possible to change values of resource attributes during collection. There is a whole section dedicated to such overrides later in this chapter.

As the collector is based on an expression, you can conveniently realize a whole range of resources. This can be quite dynamic—sometimes, you will create virtual resources that are already being realized by a rather indiscriminate collector. Let's look at a common example:

User<| |>

With no expression, the collection encompasses all virtual resources of the given type. This allows you to collect them all, without worrying about their concrete titles or attributes. This might seem redundant, because then it makes no sense to declare the resources as virtual in the first place. However, keep in mind that the collector might appear in some select manifests only, while the virtual resources can be safely added to all your nodes.

To be a little more selective, it can be useful to group virtual resources based on their tags. We haven't discussed tags yet. Each resource is tagged with several identifiers. Each tag is just a simple string. You can tag a resource manually by defining the tag metaparameter:

file { '/etc/sysctl.conf': tag => 'security' }

The named tag is then added to the resource. Puppet implicitly tags all resources with the name of the declaring class, the containing module, and a range of other useful meta information. For example, if your user module divides the user resources in classes such as administrators, developers, qa, and other roles, you can make certain nodes or classes select all users of a given role with a collection based on the class name tag:

User<| tag == 'developers' |>

Note that the tags actually form an array. The == comparison will look for the presence of the developers element in the tag array in this context. Have a look at another example to make this more clear:

@user { 'felix': 
  ensure => present, 
  groups => [ 'power', 'sys' ], 
}
User<| groups == 'sys' |>

This way, you can collect all users who are members of the sys group.

If you prefer function calls over the more cryptic collector syntax, you can keep using the realize function alongside collectors. This works without issues. Remember that each resource can be realized multiple times, even in both ways, simultaneously.

If you are wondering, the manifest for a given agent can only realize virtual resources that are declared inside this same agent's manifest. Virtual resources do not leak into other manifests. Consequently, there can be no deliberate transfer of resources from one manifest to another, either. However, there is yet another concept that allows such an exchange; this is described in the next section.

Puppet approaches the problem of sharing configuration information among multiple agent nodes by way of exported resources. The concept is simple. The manifest of node A can contain one or more resources that are purely virtual and not for realization in the manifest of this node A. Other nodes, such as B and C, can import some or all of these resources. Then, the resources become part of the catalogs of these remote nodes.

The syntax to import and export resources is very similar to that of virtual resources. An exported resource is declared by prepending the resource type name with two @ characters:

@@file { 'my-app-psk': 
  ensure  => file,
  path    => '/etc/my-app/psk', 
  content => 'nwNFgzsn9n3sDfnFANfoinaAEF', 
  tag     => 'cluster02', 
}

The importing manifests collect these resources using an expression, which is again similar to the collection of virtual resources but with double-angled brackets, < and >:

File<<| tag == 'cluster02' |>>

Tags are a very common way to take fine-grained control over the distribution of such exported resources.

The only recommendable way to enable support for exported resources is PuppetDB. It is a domain-specific database implementation that stores different kinds of data that the Puppet master deals with during regular operation. This includes catalog requests from agents (including their valuable facts), reports from catalog applications, and exported resources.

Chapter 2, The Master and Its Agents, detailed a manual installation of the master. Let's add the PuppetDB with more style—through Puppet! On the Forge, you will find a convenient module that will make this easy:

puppet module install puppetlabs-puppetdb

On the master node, the setup now becomes a one-line invocation:

puppet apply -e 'include puppetdb, puppetdb::master::config'

As our test master uses a nonstandard SSL certificate that is named master.example.net (instead of its FQDN), it must be configured for puppetdb as well:

include puppetdb
class { 
  'puppetdb::master::config': 
    puppetdb_server => 'master.example.net' 
}

The ensuing catalog run is quite impressive. Puppet installs the PostgreSQL backend, the Jetty server, and the actual PuppetDB package, and it configures everything and starts the services up—all in one go. After applying this short manifest, you have added a complex piece of infrastructure to your Puppet setup. You can now use exported resources for a variety of helpful tasks.

For homegrown interactions between clustered machines, SSH can be an invaluable tool. File transfer and remote execution of arbitrary commands is easily possible thanks to the ubiquitous sshd service. For security reasons, each host generates a unique key in order to identify itself. Of course, such public key authentication systems can only really work with a trust network, or the presharing of the public keys.

Puppet can do the latter quite nicely:

@@sshkey { $::fqdn: 
  host_aliases => $::hostname, 
  key          => $::sshecdsakey, 
  tag          => 'san-nyc' 
} 

Interested nodes collect keys with the known pattern:

Sshkey<<| tag == 'san-nyc' |>>

Now, SSH servers can be authenticated through the respective keys that Puppet safely stores in its database. As always, the Puppet master is the fulcrum of security.

Many sites can rely on a local DNS infrastructure. Resolving names to local IP addresses is easy with such setups. However, small networks or sites that consist of many independent clusters with little shared infrastructure will have to rely on names in /etc/hosts instead.

You can maintain a central hosts file per network cell, or you can make Puppet maintain each entry in each hosts file separately. The latter approach has some advantages:

A manually maintained registry is prone to become outdated every once in a while. It will also obliterate local additions in any hosts files on the agent machines.

The manifest implementation of the superior approach with exported resources is very similar to the sshkey example from the previous section:

@@host { $::fqdn: 
  ip           => $::ipaddress,
  host_aliases => [ $::hostname ],
  tag          => 'nyc-site',
}

This is the same principle, only now, each node exports its $ipaddress fact value alongside its name and not a public key. The import also works the same way:

Host<<| tag == 'nyc-site' |>>

Do you remember the Cacti module that you created during the previous chapter? It makes it very simple to configure all monitored devices in the manifest of the Cacti server. However, as this is possible, wouldn't it be even better if each node in your network was registered automatically with Cacti? It's simple: make the devices export their respective cacti_device resources for the server to collect:

@@cacti_device { $::fqdn: 
  ensure => present, 
  ip     => $::ipaddress, 
  tag    => 'nyc-site', 
} 

The Cacti server, apart from including the cacti class, just needs to collect the devices now:

Cacti_device<<| tag == 'nyc-site' |>>

If one Cacti server handles all your machines, you can just omit the tag comparison:

Cacti_device<<| |>>

Once the module supports other Cacti resources, you can handle them in the same way. Let's look at an example from another popular monitoring solution.

Puppet comes with support to manage the complete configuration of Nagios (and compatible versions of Icinga). Each configuration section can be represented by a distinct Puppet resource with types such as nagios_host or nagios_service.

Each of your machines can export their individual nagios_host resources alongside their host and cacti_device resources. However, thanks to the diverse Nagios support, you can do even better.

Assuming that you have a module or class to wrap SSH handling (you are using a Forge module for the actual management, of course), you can handle monitoring from inside your own SSH server class. By adding the export to this class, you make sure that nodes that include the class (and only these nodes) will also get monitoring:

class site::ssh { 
  # ...actual SSH management... 
  @@nagios_service { "${::fqdn}-ssh": 
    use       => 'ssh_template', 
    host_name => $::fqdn, 
  } 
} 

You probably know the drill by now, but let's repeat the mantra once more:

Nagios_service<<| |>>

With this collection, the Nagios host configures itself with all services that the agent manifests create.

Speaking of useful features that are not part of the core of Puppet, you can manage the rules of your iptables firewall, of course. You need the puppetlabs-firewall module to make the appropriate types available. Then, each machine can (among other useful things) export its own required port forwarding to the firewall machines:

@@firewall { "150 forward port 443 to ${::hostname}": 
  proto       => 'tcp', 
  dport       => '443', 
  destination => $public_ip_address, 
  jump        => 'DNAT', 
  todest      => $::ipaddress, 
  tag         => 'segment03', 
}

The title of a firewall rule resource conventionally begins with a three-digit index for ordering purposes. The firewall machines collect all these rules naturally:

Firewall<<| tag == 'segment03' |>>

As you can see, the possibilities for modeling distributed systems through exported Puppet resources are manifold. The simple pattern that we've iterated for several resource types suffices for a wide range of use cases. Combined with defined resource types, it allows you to flexibly enable your manifests to work together in order to form complex cluster setups with relatively little effort. The larger your clusters, the more work Puppet lifts from you through exports and collections.

When a node manifest stops exporting any resource, that resource's record is removed from PuppetDB once the node's catalog is compiled. This usually happens when the agent checks in with the master.

However, if you take an agent out of commission permanently, this will never happen. That's why you will need to remove those exports from the DB manually. Otherwise, other nodes will keep importing the old resources.

To clean such records from PuppetDB, use the puppet node deactivate command on the master server:

puppet node deactivate vax793.example.net

The final language construct that this chapter introduces can save you quite some typing, or rather, it saves you from copying and pasting. Writing a long, repetitive manifest is not what costs you lots of time, of course. However, a briefer manifest is often more readable, and hence, more maintainable. You achieve this by defining resource defaults—attribute values that are used for resources that don't choose their own:

Mysql_grant { options => ['GRANT'], 
  privileges => ['ALL'], 
  tables     => '*.*', 
}
mysql_grant { 'root': 
  ensure => 'present', 
  user   => 'root@localhost', 
}
mysql_grant { 'apache': 
  ensure => 'present', 
  user   => 'apache@10.0.1.%', 
  tables => 'application.*', 
}
mysql_grant { 'wordpress': 
  ensure => 'present', 
  user   => 'wordpress@10.0.5.1', 
  tables => 'wordpress.*', 
}
mysql_grant { 'backup':
  ensure     => 'present',
  user       => 'backup@localhost',
  privileges => [ 'SELECT', 'LOCK TABLE' ],
}

By default, each grant should apply to all databases and comprise all privileges. This allows you to define each actual mysql_grant resource quite sparsely. Otherwise, you will have to specify the privileges property for all resources. The options attribute will be especially repetitive, because they are identical for all grants in this example.

Note that the ensure property is repetitive as well, but it was not included. It is considered good practice to exempt this attribute from resource defaults.

Despite the convenience that this approach offers, it should not be used at each apparent opportunity. It has some downsides that you should keep in mind:

These two aspects form a dangerous combination. Defaults from a composite class can affect very distant parts of a manifest:

class webserver { 
  File { owner => 'www-data' } 
  include apache, nginx, firewall, logging_client 
  file { 
    ... 
  }
}

Files declared in the webserver class should belong to a default user. However, this default takes effect recursively in the included classes as well. The owner attribute is a property: a resource that defines no value for it just ignores its current state. A value that is specified in the manifest will be enforced by the agent. Often, you do not care about the owner of a managed file:

file { '/etc/motd': content => '...' }

However, because of the default owner attribute, Puppet will now mandate that this file belongs to www-data. To avoid this, you will have to unset the default by overwriting it with undef, which is Puppet's analog to the nil value:

File { owner => undef }

This can also be done in individual resources:

file { '/etc/motd': content => '...', owner => undef }

However, doing this constantly is hardly feasible. The latter option is especially unattractive, because it leads to more complexity in the manifest code instead of simplifying it. After all, not defining a default owner attribute will be the cleaner way here.

Resource defaults should be used with consideration and care. For some properties such as file mode, owner, and group, they should usually be avoided.

The new parser was introduced in Puppet 3.5 alongside the old parser. To make use of the new language features, a special configuration item needed to be set explicitly. This allowed early adopters and people interested in the new technology to test the parser and check for code incompatibilities in an early stage.

On Puppet 3.x, the new parser was subject to change without further notice. Therefore it is recommended to upgrade to the latest 3.x release.

With directory environments, it is possible to specify environment specific settings within an environment.conf configuration file:

# /etc/puppet/environments/puppet_update/environment.conf
# environment config file for special puppet_update environment
parser = future

Next, all your puppet code needs to be put into the newly created environment path, including node classification.

On each of the different node types, it is now possible to manually run:

puppet agent --test --environment=puppet_update --noop

Check both Master and Agent output and logfiles for any errors or unwanted changes and adapt your Puppet code if necessary.

Spinning up a new Puppet Master is another approach. The following process assumes that the Puppet CA has been created using the DNS alt names setting in the puppet.conf file. In case no DNS alt names have been configured, it is required to completely recreate the Puppet CA.

Puppet CA needs to know about the CN (common name) of the Puppet Master fqdn. It is possible to provide DNS alternative names, for which the CA will also be valid.

Normally, Puppet uses the Master fqdn for the common name. But if you provide the configuration attribute dns_alt_names prior to generating the CA, this configuration option will be added to the CA.

It is highly recommended to configure dns_alt_names. Having this enabled allows you to scale up to multiple compile masters and add an additional Puppet Master for the migration process.

To find out whether DNS alt names have been added, you can use the puppet cert command:

puppet cert list –all

This command will print all certificates. Check for the certificate of your Puppet Master.

For example, consider the following:

puppet cert list --all
+ "puppetmaster.example.net" (SHA256) 7D:11:33:00:94:B3:C4:48:D2:10:B3:C7:B0:38:71:28:C5:75:2C:61:3B:3E:63:C6:95:7C:C9:DF:59:F7:C5:BE (alt names: "DNS:puppet", "DNS:puppet.example.net", "DNS:puppetmaster.example.net")

The following steps will guide you through the Puppet 4 setup. On a Debian 7 based system add the PC1 Puppet Labs repository:

curl -O http://apt.puppetlabs.com/puppetlabs-release-pc1-wheezy.deb
dpkg -i puppetlabs-release-pc1-wheezy.deb
apt-get update
apt-get install puppetserver puppet-agent

Do not start the Puppet server process yet! It is required to run the new Puppet 4 Master as CA Master, which needs the CA and certificates from the Puppet 3 Master copied over to the new Puppet 4 Master.

Within the next step, all Puppet agents need a change on the puppet.conf file. You will need to provide different settings for ca_server and server:

ini_setting { 'puppet_ca_server':
  path    => '/etc/puppet/puppet.conf',
  section => 'agent',
  setting => 'ca_server',
  value   => 'puppet4.example.net'
}

Now place all your Puppet code onto the new Puppet 4 Master into an environment (/etc/puppetlabs/code/environments/development/{manifests,modules}).

Test your code for Puppet 4 errors by running the following command on each of your nodes:

puppet agent --test --noop --server puppet4.example.net --environment development

Change your Puppet code to fix potential errors. Once no errors and no unwanted configuration changes occur on the Puppet 4 Master and agents, your code is Puppet 4 compatible.

It is important to make sure that your existing agents are prepared to operate with a master that is already at version 4. Check the following aspects:

The latter step prepares you for the agent update, because Puppet 4 will always behave like that and refrain from converting all fact values to the string type.

Do make sure that you update to Puppet Server 2.1 or later. Passenger-based masters and Puppet Server 2.0 are not compatible with Puppet 3 agents.

Another approach for verifying whether the existing Puppet code will work on Puppet 4 is unit and integration testing using rspec-puppet and beaker. This procedure is not within the scope of this book.

Whether you started fresh with Puppet 4, or used one of the preceding procedures to migrate your Puppet 3 infrastructure, it is now time to discover the benefits of the new version.

Node inheritance has been considered good practice during older Puppet releases. To avoid too much code on the node level, a generic, nonexistent host was created (basenode) and the real nodes inherited from basenode:

node basenode {
  include security
  include ldap
  include base
}
node 'www01.example.net' inherits 'basenode' {
  class { 'apache': }
  include apache::mod::php
  include webapplication
}

This node classification is no longer supported by Puppet 4.

As of 2012, the Roles and Profiles pattern became increasingly popular, bringing new methods on how to allow smart node classification. From a technical point of view, roles and profiles are Puppet classes. The profile module wraps technical modules and adapts their usage to the existing infrastructure by providing data such as ntp servers and ssh configuration options. The role module describes system business use cases and makes use of the declared profiles:

class profile::base {
  include security
  include ldap
  include base
}
class profile::webserver {
  class { 'apache': }
  include apache_mod_php
}

class role::webapplication {
  include profile::base
  include profile::webserver
  include profile::webapplication
}

node 'www01.example.net' {
  include role::webapplication
}

A minor change with a huge impact is the change of empty string comparison. Prior to Puppet 4, one could test for either an unset variable or a variable containing an empty string by checking for the variable:

class ssh (
  $server = true,
){
  if $server {...}
}

The ssh class behaved similar ($server evaluates to true) when used within the following different declarations:

include ssh
class { 'ssh': server => 'yes', }

Disabling the server section in the ssh class could be achieved by the following class declarations:

class { 'ssh': server => false, }
class { 'ssh': server => '', }

The behavior of the last example (empty string) changed in Puppet 4. The empty string now equals a true value in Boolean context just as in Ruby. If your code makes use of this way of variable checking, you need to add the check for empty string to retain the same behavior with Puppet 4:

class ssh (
  $server = true,
){
  if $server and $server != '' {...}
}

Variables sometimes look like constants and exhibit the following features:

Sometimes developers prefer to write the variables in capital letters due to the previously mentioned items, to make them look like Ruby constants.

With Puppet 4, variable names must not start with a capital letter:

class variables
{
  $Local_var = 'capital variable'
  notify { "Local capital var: ${Local_var}": }
}

Declaring this class will now produce the following error message:

root@puppetmaster:/etc/puppetlabs/code/environments/production/modules# puppet apply -e 'include variables'
Error: Illegal variable name, The given name 'Local_var' does not conform to the naming rule /^((::)?[a-z]\w*)*((::)?[a-z_]\w*)$/ at /etc/puppetlabs/code/environments/production/modules/variables/manifests/init.pp:3:3 on node puppetmaster.example.net

Due to the type system and due to the reason that Puppet 4 now takes everything as an expression, one has to name references on other declared resources properly. References now have some strict regulations:

The following will produce errors on Puppet 4:

User [Root]
User[Root]

Starting with Puppet 4, references have to be written in the following pattern:

Type['title']

Our example needs to be changed to:

User['root']

Many modules (even on the module Forge) have used the hyphen in the module name. The hyphen is now no longer a string character, but a mathematical operator (subtraction). In consequence, hyphens are now strictly forbidden in the following descriptors:

When using modules with hyphens, the hyphen needs to be removed or replaced with a string character (for example, the underscore).

This is possible with older versions as follows:

class syslog-ng {...}

include syslog-ng

Now, the new style is as follows:

class syslog_ng {
  ...
}

include syslog_ng

Some people used the possibility to put .rb files as manifests inside modules. These .rb files contained Ruby code and were mostly needed for working with data. Puppet 4 now has data types that make this obsolete.

The support for these Ruby manifests has been removed in Puppet 4.

With Puppet 3 and older, it was required to specify absolute class names in case that a local class name was identical to another module:

# in module "mysql"
class mysql {
  ...
}
# in module "application"
class application::mysql {
  include mysql
}

Within the scope of the application:: namespace, Puppet 3 would search this namespace for a mysql class to be included. Effectively, the application::mysql class would include itself. But this was not what we intended to do. We were looking for the mysql module instead. As a workaround, everybody was encouraged to specify the absolute path to the mysql module class:

class application::mysql {
  include ::mysql
}

This relative name resolution no longer applies in Puppet 4. The original example works now.

Due to the reason that Puppet 3 was not aware of the different data types (mostly everything was dealt with as being of type string) it was possible to combine several different data types.

Puppet 4 now is very strict when it comes to combining different data types. The easiest example is dealing with float and integer; when adding a float and an integer, the result will be of type float.

Combining actions on different data types such as string and bool will now result in an error.

The following code will work:

case $::operatingsystemmajrelease {
  '8': {
    include base::debian::jessie
  }
}

On the other hand, the following code will not work:

if $::operatingsystemmajrelease > 7 {
  include base::debian::jessie
}

You will receive the following error message:

Error: Evaluation Error: Comparison of: String > Integer, is not possible. Caused by 'A String is not comparable to a non String'

Review comparison of different Facter variables carefully. Some Facter variables such as operatingsystemmajrelease return data of the type string, whereas processorcount returns an integer value.

There are a number of ways in which a Puppet manifest can approach this problem of micromanagement. The most direct way is to define whole sets of classes—one for each individual node:

class site::mysql_server01 {
  class { 'mysql': server_id => '1', … }
}
class site::mysql_server02 {
  class { 'mysql': server_id => '2', … }
}
… 
class site::mysql_aux01 {
  class { 'mysql': server_id => '101', … }
}
# and so forth ...

This is a very high-maintenance solution for the following reasons:

In short, this is the brute-force approach that introduces its own share of cost. A more economic approach would be to pass the values that are different among nodes (and only those!) to a wrapper class:

node 'xndp12-sql09.example.net' {
  class { 'site::mysql_server':
    mysql_server_id => '103',
  }
}

This wrapper can declare the mysql class in a generic fashion, thanks to the individual parameter value per node:

class site::mysql_server(
  String $mysql_server_id
) {
  class { 'mysql': 
    server_id => $mysql_server_id, 
    ...
  }
}

This is much better, because it eliminates the redundancy and its impact on maintainability. The wrinkle is that the node blocks can become quite messy with parameter assignments for many different subsystems. Explanatory comments contribute to the wall of text that each node block can become.

You can take this a step further by defining lookup tables in hash variables, outside of any node or class, on the global scope:

$mysql_config_table = {
  'xndp12-sql01.example.net' => {
    server_id   => '1',
    buffer_pool => '12G',
  },
  …
}

This alleviates the need to declare any variables in node blocks. The classes look up the values directly from the hash:

class site::mysql_server(
  $config = $mysql_config_table[$::certname]
) {
  class { 'mysql':
    server_id => $config['server_id'], 
    ...
  }
}

This is pretty sophisticated and is actually close to the even better way that you will learn about in this chapter. Note that this approach still retains a leftover possibility of redundancy. Some configuration values are likely to be identical among all nodes that belong to one group, but are unique to each group (for example, preshared keys of any variety).

This requires that all servers in the hypothetical xndp12 cluster contain some key/value pairs that are identical for all members:

$crypt_key_xndp12 = 'xneFGl%23ndfAWLN34a0t9w30.zges4'
$config = {
'xndp12-stor01.example.net' => { $crypt_key =>$crypt_key_xndp12, … },
'xndp12-stor02.example.net' => { $crypt_key =>$crypt_key_xndp12, … },
'xndp12-sql01.example.net'  => { $crypt_key =>$crypt_key_xndp12, … },
...
}

This is not ideal but let's stop here. There is no point in worrying about even more elaborate ways to sort configuration data into recursive hash structures. Such solutions will quickly grow very difficult to understand and maintain anyway. The silver bullet is an external database that holds all individual and shared values. Before I go into the details of using Hiera for just this purpose, let's discuss the general ideas of hierarchical data storage.

The support for retrieving data values from Hiera has been built into Puppet since version 3. All you need in order to get started is a hiera.yaml file in the configuration directory.

As the filename extension suggests, the configuration is in the YAML format and contains a hash with keys for the backends, the hierarchy, and backend-specific settings. The keys are noted as Ruby symbols with a leading colon:

# /etc/puppetlabs/code/hiera.yaml
:backends:
  - yaml
:hierarchy: 
  - node/%{::clientcert}
  - role/%{::role}
  - location/%{::datacenter}
  - common
:yaml: 
  :datadir: /etc/puppetlabs/code/environments/%{::environment}/hieradata

Note that the value of :backends is actually a single element array. You can pick multiple backends. The significance will be explained later. The :hierarchy value contains a list of the actual layers that were described earlier. Each entry is the name of a data source. When Hiera retrieves a value, it searches each data source in turn. The %{} expression allows you to access the values of Puppet variables. Use only facts or global scope variables here—anything else will make Hiera's behavior quite confusing.

Finally, you will need to include configurations for each of your backends. The configuration above uses the YAML backend only, so there is only a hash for :yaml with the one supported :datadir key. This is where Hiera will expect to find YAML files with data. For each data source, the datadir can contain one .yaml file. As the names of the sources are dynamic, you will typically create more than four or five data source files. Let's create some examples before we have a short discussion on the combination of multiple backends.

The backend of your Hiera setup determines how you have to store your configuration values. For the YAML backend, you fill datadir with files that each holds a hash of values. Let's put some elements of the reporting engine configuration into the example hierarchy:

# /etc/puppetlabs/code/environments/production/hieradata/common.yaml
reporting::server: stats01.example.net
reporting::server_port: 9033

The values in common.yaml are defaults that are used for all agents. They are at the broad base of the hierarchy. Values that are specific to a location or role apply to smaller groups of your agents. For example, the database servers of the postgres role should run some special reporting plugins:

# /etc/puppetlabs/code/environments/production/hieradata/role/postgres.yaml
reporting::plugins:
  - iops
  - cpuload

On such a higher layer, you can also override the values from the lower layers. For example, a role-specific data source such as role/postgres.yaml can set a value for reporting::server_port as well. The layers are searched from the most to the least specific, and the first value is used. This is why it is a good idea to have a node-specific data source at the top of the hierarchy. On this layer, you can override any value for each agent. In this example, the reporting node can use the loopback interface to reach itself:

#/etc/puppetlabs/.../hieradata/node/stats01.example.net.yaml
reporting::server: localhost

Each agent receives a patchwork of configuration values according to the concrete YAML files that make up its specific hierarchy.

Don't worry if all this feels a bit overwhelming. There are more examples in this chapter. Hiera also has the charming characteristic of seeming rather complicated on paper, but it feels very natural and intuitive once you try using it yourself.

There are two built-in backends: YAML and JSON. This chapter will focus on YAML, because it's a very convenient and efficient form of data notation. The JSON backend is very similar to YAML. It looks for data in .json files instead of .yaml for each data source; these files use a different data notation format.

The use of multiple backends should never be truly necessary. In most cases, a well-thought-out hierarchy will suffice for your needs. With a second backend, data lookup will traverse your hierarchy once per backend. This means that the lowest level of your primary backend will rank higher than any layer from additional backends.

In some cases, it might be worthwhile to add another backend just to get the ability to define even more basic defaults in an alternative location—perhaps a distributed filesystem or a source control repository with different commit privileges.

Also, note that you can add custom backends to Hiera, so these might also be sensible choices for secondary or even tertiary backends. A Hiera backend is written in Ruby, like the Puppet plugins. The details of creating such a backend are beyond the scope of this book.

You have studied the theory of storing data in Hiera at length, so it's finally time to see how to make use of this in Puppet.

You have seen how to invoke the hiera function for value retrieval. There is really not more to it than what you have seen in the previous section, except for an optional parameter. It allows you to include an additional layer at the top of your hierarchy. If the key is found in the named data source, it will override the result from the regular hierarchy:

$plugins = hiera('reporting::plugins', [], 'global-overrides')

If the reporting::plugins key is found in the global-overrides data source, the value is taken from there. Otherwise, the normal hierarchy is searched.

Generally, assigning the retrieved value to a manifest variable is quite common. However, you can also invoke the hiera function in other useful contexts, such as the following:

@@cacti_device { $::fqdn:
  ip => hiera('snmp_address', $::ipaddress),
}

The lookup result can be handed to a resource directly as a parameter value. This is an example of how to allow Hiera to define a specific IP address per machine that should be used for a specific service. It acts as a simple way to manually override Facter's assumptions.

$max_threads = hiera('max_threads')
if $max_threads !~ Integer {
    fail "The max_threads value must be an integer number"
}

Another frequent occurrence is a parameter default that is made dynamic through a Hiera lookup:

define logrotate::config(
  Integer $rotations = hiera('logrotate::rotations', 7)
) {
  # regular define code here
}

For logrotate::config resources that are declared with an explicit parameter value, the Hiera value is ignored:

logrotate::config { '/var/log/cacti.log': rotations => 12 }

This can be a little confusing. Still, the pattern adds some convenience. Most agents can rely on the default. The hierarchy allows you to tune this default on multiple levels of granularity.

The concept of parameterized classes might have gotten a somewhat tarnished reputation, judging from our coverage of it so far. It allegedly makes it difficult to include classes from multiple places in the manifest, or silently allows it under shifting circumstances. While that is true, you can avoid these issues by relying on Hiera for your class parameterization needs.

Since Puppet's Version 3.2, it has been possible to choose the values for any class's parameters right in the Hiera data. Whenever you include a class that has any parameters, Puppet will query Hiera to find a value for each of them. The keys must be named after the class and parameter names, joined by a double colon. Remember the cacti class from Chapter 5, Extending Your Puppet Infrastructure with Modules? It had a $redirect parameter. To define its value in Hiera, add the cacti::redirect key:

# node/cacti01.example.net.yaml
cacti::redirect: false

Some classes have very elaborate interfaces—the apache class from the Puppet Labs Apache module accepts 49 parameters at the time of writing this. If you need many of those, you can put them into the target machine's dedicated YAML file as one coherent block of keys with values. It will be quite readable, because the apache:: prefixes line up.

You don't save any lines compared to specifying the parameters right in the manifest, but at least the wall of options will not get in your way while you're programming in your manifests—you separated data from code.

The point that is perhaps the most redeeming for class parameterization is that each key is independent in your hierarchy. Many parameters can most likely be defined for many or all of your machines. Clusters of application servers can share some settings (if your hierarchy includes a layer on which they are grouped together), and you can override parameters for single machines as you see fit:

# common.yaml
apache::default_ssl_cert: /etc/puppetlabs/puppet/ssl/certs/%{::clientcert}.pem
apache::default_ssl_key: /etc/puppetlabs/puppet/ssl/private_keys/%{::clientcert}.pem
apache::purge_configs: false

The preceding example prepares your site to use the Puppet certificates for HTTPS. This is a good choice for internal services, because trust to the Puppet CA can be easily established, and the certificates are available on all agent machines. The third parameter, purge_configs, prevents the module from obliterating any existing Apache configuration that is not under Puppet's management.

Let's see an example of a more specific hierarchy layer which overrides this setting:

# role/httpsec.yaml
apache::purge_configs: true
apache::server_tokens: Minimal
apache::server_signature: off
apache::trace_enable: off

On machines that have the httpsec role, the Apache configuration should be purged so that it matches the managed configuration completely. The hierarchy of such machines also defines some additional values that are not defined in the common layer. The SSL settings from common remain untouched.

A specific machine's YAML can override keys from either layer if need be:

# node/sec02-sxf12.yaml
apache::default_ssl_cert: /opt/ssl/custom.pem
apache::default_ssl_key: /opt/ssl/custom.key
apache::trace_enable: extended

All these settings require no additional work. They take effect automatically, provided that the apache class from the puppetlabs-apache module is included.

For some users, this might be the only way in which Hiera is employed on their master, which is perfectly valid. You can even design your manifests specifically to expose all configurable items as class parameters. However, keep in mind that another advantage of Hiera is that any value can be retrieved from many different places in your manifest.

For example, if your firewalled servers are reachable through dedicated NAT ports, you will want to add this port to each machine's Hiera data. The manifest can export this value not only to the firewall server itself, but also to external servers that use it in scripts and configurations to reach the exporting machine:

$nat_port = hiera('site::net::nat_port')
@@firewall { "650 forward port ${nat_port} to ${::fqdn}":
  proto       => 'tcp',
  dport       => $nat_port, 
  destination => hiera('site::net::nat_ip'),
  jump   => 'DNAT',
  todest => $::ipaddress,
  tag    => hiera('site::net::firewall_segment'),
}

The values will most likely be defined on different hierarchical layers. nat_port is agent-specific and can only be defined in the %{::fqdn} (or %{::clientcert} for better security) derived data source. nat_ip is probably identical for all servers in the same cluster. They might share a server role. firewall_segment could well be identical for all servers that share the same location:

# stor03.example.net.yaml
site::net::nat_port: 12020
...
# role/storage.yaml
site::net::nat_ip: 198.58.119.126
...
# location/portland.yaml
site::net::firewall_segment: segment04
...

As previously mentioned, some of this data will be helpful in other contexts as well. Assume that you deploy a script through a defined type. The script sends messages to remote machines. The destination address and port are passed to the defined type as parameters. Each node that should be targeted can export this script resource:

@@site::maintenance_script {"/usr/local/bin/maint-${::fqdn}":
  address => hiera('site::net::nat_ip'),
  port    => hiera('site::net::nat_port'),
}

It would be impractical to do all this in one class that takes the port and address as parameters. You would want to retrieve the same value from within different classes or even modules, each taking care of the respective exports.

Some examples in this chapter defined array values in Hiera. The good news is that retrieving arrays and hashes from Hiera is not at all different from simple strings, numbers, or Boolean values. The hiera function will return all these values, which are ready for use in the manifest.

There are two more functions that offer special handling for such values: the hiera_array and hiera_hash functions.

When the hiera_array function is invoked, it gathers all named values from the whole hierarchy and merges them into one long array that comprises all elements that were found. Take the distributed firewall configuration once more, for example. Each node should be able to export a list of rules that open ports for public access. The manifest for this would be completely driven by Hiera:

if hiera('site::net::nat_ip', false) {
  @@firewall { "200 NAT ports for ${::fqdn}":
    port        => hiera_array('site::net::nat_ports'),
    proto       => 'tcp',
    destination => hiera('site::net::nat_ip'),
    jump        => 'DNAT',
    todest      => $::ipaddress,
  }
}

Please note that the title "200 NAT ports" does not allude to the number of ports, but just adheres to the naming conventions for firewall resources. The numeric prefix makes it easy to maintain order. Also, note the seemingly nonsensical default value of false for the site::net::nat_ip key in the if clause. This forms a useful pattern, though—the resource should only be exported if public_ip is defined for the respective node.

if hiera('feature_flag_A', undef) != undef { … }

The hierarchy can then hold ports on several layers:

# common.yaml
nat_ports: 22

The SSH port should be available for all nodes that get a public address. Note that this value is not an array itself. This is fine; Hiera will include scalar values in the resulting list without any complaint.

# role-webserver.yaml
nat_ports: [ 80, 443 ]

Standalone web application servers present their HTTP and HTTPS ports to the public:

# tbt-backend-test.example.net.yaml
nat_ports: 
  - 5973
  - 5974
  - 5975
  - 6630

The testing instance for your new cloud service should expose a range of ports for custom services. If it has the webserver role (somehow), it will lead to an export of ports 22, 80, and 443 as well as its individually chosen list.

A similar alternative lookup function exists for hashes. The hiera_hash function also traverses the whole hierarchy and constructs a hash by merging all hashes it finds under the given Hiera key from all hierarchy layers. Hash keys in higher layers overwrite those from lower layers. All values must be hashes. Strings, arrays, or other data types are not allowed in this case:

# common.yaml
haproxy_settings:
  log_socket: /dev/log
  log_level: info
  user: haproxy
  group: haproxy
  daemon: true

These are the default settings for haproxy at the lowest hierarchy level. On web servers, the daemon should run as the general web service user:

# role/webserver.yaml
haproxy_settings:
  user: www-data
  group: www-data

When retrieved using hiera('haproxy_settings'), this will just evaluate to the hash, {'user'=>'www-data','group'=>'www-data'}. The hash at the role-specific layer completely overrides the default settings.

To get all values, create a merger using hiera_hash('haproxy_settings') instead. The result is likely to be more useful:

{ 'log_socket' =>'/dev/log', 'log_level' => 'info',
'user' => 'www-data', 'group' => 'www-data', 'daemon' => true }

The limitations are similar to those of hiera_array. Keys from any hierarchy level cannot be removed, they can only be overwritten with different values. The end result is quite similar to what you would get from replacing the hash with a group of keys:

# role/webserver.yaml
haproxy::user: www-data
haproxy::group: www-data

If you opt to do this, the data can also be easily fit to a class that can bind these values to parameters automatically. Preferring flat structures can, therefore, be beneficial. Defining hashes in Hiera is still generally worthwhile, as the next section explains.

You can now move most of the concrete configuration to the data storage. Classes can be included from the manifest or through Hiera. Puppet looks up parameter values in the hierarchy, and you can flexibly distribute the configuration values there in order to achieve just the desired result for each node with minimal effort and redundancy.

This does not mean that you don't write actual manifest code anymore. The manifest is still the central pillar of your design. You will often need logic that uses the configuration data as input. For example, there might be classes that should only be included if a certain value is retrieved from Hiera:

if hiera('use_caching_proxy', false) {
    include nginx
}

If you try and rely on Hiera exclusively, you will have to add nginx to the classes array at all places in the hierarchy that set the use_caching_proxy flag to true. This is prone to mistakes. What's worse is that the flag can be overridden from true to false at a more specific layer, but the nginx element cannot be removed from an array that is retrieved by hiera_include.

It is important to keep in mind that the manifest and data should complement each other. Build manifests primarily and add lookup function calls at opportune places. Defining flags and values in Hiera should allow you (or the user of your modules) to alter the behavior of the manifest. The data should not be the driver of the catalog composition, except for places in which you replace large numbers of static resources with large data structures.

To round things off, let's build a complete example of a module that is enhanced with Hiera. Create a demo module in the environment of your choice. We will go with production:

# /etc/puppetlabs/code/environments/production/modules/demo/manifests/init.pp
class demo(
  Boolean $auto = false,
  Boolean $syslog = true,
  String $user = 'nobody'
) {
  file { '/usr/local/bin/demo': … }
  if $auto {
    cron { 'auto-demo':
      user    => $user,
      command => '/usr/local/bin/demo'
      ...
    }
    $atoms = hiera('demo::atoms', {})
    if $atoms !~ Hash[String, Hash] {
      fail "The demo::atoms value must be a hash of resource descriptions"
    }
    $atoms.each |$title, $attributes| {
      demo::atom { $title:
        address => $attributes['address'],
        port    => $attributes['port'],
    }
  }
}

This class implicitly looks up three Hiera keys for its parameters:

There is also an explicit lookup of an optional demo::atoms hash that creates configuration items for the module. The data is converted to resources of the following defined type:

# /etc/puppetlabs/code/.../modules/demo/manifests/atom.pp
define demo::atom(
  String $address,
  Integer[1,65535] $port=14193
) {
  file { "/etc/demo.d/${name}":
    ensure  => 'file',
    content => "---\nhost: ${address}\nport: ${port}\n",
    mode    => '0644',
    owner   => 'root',
    group   => 'root',
  }
}

The module uses a default of nobody for user. Let's assume that your site does not run scripts with this account, so you set your preference in common.yaml. Furthermore, assume that you also don't commonly use syslog:

demo::user: automation
demo::syslog: false

If this user account is restricted on your guest workstations, Hiera should set an alternative value in role/public_desktop.yaml:

demo::user: maintenance

Now, let's assume that your cron jobs are usually managed in site modules, but not for web servers. So, you should let the demo module itself take care of this on web servers through the $auto parameter:

# role/webserver.yaml
demo::auto: true

Finally, the web server, int01-web01.example.net should be an exception. Perhaps it's supposed to run no background jobs whatsoever:

# int01-web01.example.net.yaml
demo::auto: false

Concerning configuration resources, this is how to make each machine add itself as a peer:

# common.yaml
demo::atoms:
  self:
    address: localhost

Let's assume that the Kerberos servers should not try this. Just make them override the definition with an empty hash:

# role/kerberos.yaml
demo::atoms: {}

Here is how to make the database servers also contact the custom server running on the Puppet master machine on a nonstandard port:

# role-dbms.yaml
demo::atoms:
  self:
    address: localhost
  master:
    address: master.example.net
    port: 60119

With all your Hiera data in place, you can now include the demo class from your site.pp (or an equivalent) file indiscriminately. It is often a good idea to be able to allow certain agent machines to opt out of this behavior in the future. Just add an optional Hiera flag for this:

# site.pp
if hiera('enable_demo', true) {
    include demo
}

Agents that must not include the module can be given a false value for the enable_demo key in their data now.