Create a site.pp file and place the following code in it:

  node default {
    package { 'httpd':
      ensure => 'installed'
    }
  }

This manifest will ensure that any node, on which this manifest is applied, will install a package called 'httpd'. The default keyword is a wildcard to Puppet; it applies anything within the node default definition to any node. When Puppet applies the manifest to a node, it uses a Resource Abstraction Layer (RAL) to translate the package type into the package management system of the target node. What this means is that we can use the same manifest to install the httpd package on any system for which Puppet has a Provider for the package type. Providers are the pieces of code that do the real work of applying a manifest. When the previous code is applied to a node running on a YUM-based distribution, the YUM provider will be used to install the httpd RPM packages. When the same code is applied to a node running on an APT-based distribution, the APT provider will be used to install the httpd DEB package (which may not exist, most debian-based systems call this package apache2; we'll deal with this sort of naming problem later).

When facter is installed (as a dependency for puppet), several fact definitions are installed by default. You can reference each of these facts by name from the command line.

Running facter without any arguments causes facter to print all the facts known about the system. We will see in later chapters that facter can be extended with your own custom facts. All facts are available for you to use as variables; variables are discussed in the next section.

In this example, the package will be installed before the service is started. Using require within the definition of the httpd service ensures that the package is installed first, regardless of the order within the manifest file.

Package['httpd']

Class['something::somewhere']

example::thing {'one':} 

Example::Thing['one']

Trifecta

The relationship between package and service previously mentioned is an important and powerful paradigm of Puppet. Adding one more resource-type file into the fold, creates what puppeteers refer to as the trifecta. Almost all system administration tasks revolve around these three resource types. As a system administrator, you install a package, configure the package with files, and then start the service.

Diagram of Trifecta (Files require package for directory, service requires files and package)

We will need the same definitions as our last example; we need the package and service installed. We now need two more things. We need the configuration file and index page (index.html) created. For this, we follow these steps:

The require attribute to the file resources tell Puppet that we need the /var/www/cookbook directory created before we can create the index.html file. The important concept to remember is that we cannot assume anything about the target system (node). We need to define everything on which the target depends. Anytime you create a file in a manifest, you have to ensure that the directory containing that file exists. Anytime you specify that a service should be running, you have to ensure that the package providing that service is installed.

In this example, using metaparameters, we can be confident that no matter what state the node is in before running Puppet, after Puppet runs, the following will be true:

In this section, I'll show you a few of the more important examples and how to make sure that your code is style compliant.

service {'httpd':
  ensure  => running,
}

package { 'exim4':

package { exim4:

file { '/etc/motd':
  content => "Welcome to ${::fqdn}\n"
}

name => 'Nucky Thompson',
mode => '0700',
owner => 'deploy',

ensure => installed,
enable => true,
ensure => running,

if "false" {
  notify { 'True': }
}
if 'false' {
  notify { 'Also true': }
}
if false {
  notify { 'Not true': }
}

source => "puppet:///modules/webserver/${brand}.conf",

service { 'memcached':
  ensure => running,
  enable => true,
}

package { 'puppet': ensure => installed }

package { 'rake':
  ensure   => installed,
  provider => gem,
  require  => Package['rubygems'],
}

file { "/var/www/${app}/shared/config/rvmrc":
  owner   => 'deploy',
  group   => 'deploy',
  content => template('rails/rvmrc.erb'),
  require => File["/var/www/${app}/shared/config"],
}

file { '/etc/php5/cli/php.ini':
  ensure => link,
  target => '/etc/php.ini',
}

To create and apply a simple manifest, follow these steps:

When several people are working on a code base, it's easy for style inconsistencies to creep in. Fortunately, there's a tool available which can automatically check your code for compliance with the style guide: puppet-lint. We'll see how to use this in the next section.

Here's what you need to do to install Puppet-lint:

Follow these steps to use Puppet-lint:

You can find out more about Puppet-lint at https://github.com/rodjek/puppet-lint.

Should you follow Puppet style guide and, by extension, keep your code lint-clean? It's up to you, but here are a couple of things to think about:

Having said that, it's possible to tell Puppet-lint to ignore certain checks if you've chosen not to adopt them in your codebase. For example, if you don't want Puppet-lint to warn you about code lines exceeding 80 characters, you can run Puppet-lint with the following option:

t@cookbook ~$ puppet-lint --no-80chars-check

Run puppet-lint --help to see the complete list of check configuration commands.

Following are the steps to create an example module:

When we created the module using Puppet's module generate command, we used the name thomas-memcached. The name before the hyphen is your username or your username on Puppet forge (an online repository of modules). Since we want Puppet to be able to find the module by the name memcached, we make a symbolic link between thomas-memcached and memcached.

Modules have a specific directory structure. Not all of these directories need to be present, but if they are, this is how they should be organized:

modules/
  └MODULE_NAME/  never use a dash (-) in a module name
     └examples/ example usage of the module
     └files/ flat files used by the module
     └lib/
        └facter/ define new facts for facter
        └puppet/
           └parser/
              └functions/ define a new puppet function, like sort() 
           └provider/ define a provider for a new or existing type
           └util/ define helper functions (in ruby)
           └type/ define a new type in puppet
     └manifests/
        └init.pp  class MODULE_NAME { }
     └spec/ rSpec tests
     └templates/ erb template files used by the module

All manifest files (those containing Puppet code) live in the manifests directory. In our example, the memcached class is defined in the manifests/init.pp file, which will be imported automatically.

Inside the memcached class, we refer to the memcached.conf file:

file { '/etc/memcached.conf':
  source => 'puppet:///modules/memcached/memcached.conf',
}

The preceding source parameter tells Puppet to look for the file in:

MODULEPATH/ (/home/thomas/.puppet/modules)

└memcached/

└files/

└memcached.conf

Learn to love modules because they'll make your Puppet life a lot easier. They're not complicated, however, practice and experience will help you judge when things should be grouped into modules, and how best to arrange your module structure. Modules can hold more than manifests and files as we'll see in the next two sections.

file { '/etc/memcached.conf':
  content => template('memcached/memcached.conf.erb'),
}

MODULEPATH/memcached/templates/memcached.conf.erb

You can download modules provided by other people and use them in your own manifests just like the modules you create. For more on this, see Using Public Modules recipe in Chapter 7, Managing Applications.

Here are some tips on how to name things in your manifests:

Puppet community maintains a set of best practice guidelines for your Puppet infrastructure, which includes some hints on naming conventions:

http://docs.puppetlabs.com/guides/best_practices.html

Some people prefer to include multiple classes on a node by using a comma-separated list, rather than separate include statements, for example:

  node 'server014' inherits 'server' {
    include mail::server, repo::gem, repo::apt, zabbix
  }

This is a matter of style, but I prefer to use separate include statements, one on a line, because it makes it easier to copy and move around class inclusions between nodes without having to tidy up the commas and indentation every time.

I mentioned inheritance in a couple of the preceding examples; if you're not sure what this is, don't worry, I'll explain this in detail in the next chapter.

Here's an example of how to use inline_template:

Pass your Ruby code to inline_template within Puppet manifest, as follows:

cron { 'chkrootkit':
  command => '/usr/sbin/chkrootkit >
    /var/log/chkrootkit.log 2>&1',
  hour    => inline_template('<%= @hostname.sum % 24 %>'),
  minute  => '00',
}

Anything inside the string passed to inline_template is executed as if it were an ERB template. That is, anything inside the <%= and %> delimiters will be executed as Ruby code, and the rest will be treated as a string.

In this example, we use inline_template to compute a different hour for this cron resource (a scheduled job) for each machine, so that the same job does not run at the same time on all machines. For more on this technique, see the Distributing cron jobs efficiently recipe in Chapter 6, Managing Resources and Files.

In ERB code, whether inside a template file or an inline_template string, you can access your Puppet variables directly by name using an @ prefix, if they are in the current scope or the top scope (facts):

<%= @fqdn %>

To reference variables in another scope, use scope.lookupvar, as follows:

<%= "The value of something from otherclass is " + scope.lookupvar('otherclass::something') %>

You should use inline templates sparingly. If you really need to use some complicated logic in your manifest, consider using a custom function instead (see the Creating custom functions recipe in Chapter 9, External Tools and the Puppet Ecosystem).

Here's a common example of how arrays are used:

Where Puppet encounters an array as the name of a resource, it creates a resource for each element in the array. In the example, a new package resource is created for each of the packages in the $packages array, with the same parameters (ensure => installed). This is a very compact way to instantiate many similar resources.

Although arrays will take you a long way with Puppet, it's also useful to know about an even more flexible data structure: the hash.

$interface = {
  'name' => 'eth0',
  'ip'   => '192.168.0.1',
  'mac'  => '52:54:00:4a:60:07' 
}
notify { "(${interface['ip']}) at ${interface['mac']} on ${interface['name']}": }

t@cookbook:~/.puppet/manifests$ puppet apply hash.pp
Notice: (192.168.0.1) at 52:54:00:4a:60:07 on etho

define lunchprint() {
  notify { "Lunch included ${name}":}": }
}

$lunch = ['egg', 'beans', 'chips']
lunchprint { $lunch: }

t@mylaptop ~ $ puppet apply lunchprint.pp 
...
Notice: Lunch included chips
Notice: Lunch included beans
Notice: Lunch included egg

$menu = 'egg beans chips'
$items = split($menu, ' ')
lunchprint { $items: }

t@mylaptop ~ $ puppet apply lunchprint2.pp 
...
Notice: Lunch included chips
Notice: Lunch included beans
Notice: Lunch included egg.

$menu = 'egg and beans and chips'
$items = split($menu, ' and ')

$lunch = 'egg:beans,chips'
$items = split($lunch, ':|,')

Here's an example of a useful conditional statement. Add the following code to your manifest:

  if $::timezone == 'UTC' {
    notify { 'Universal Time Coordinated':}
  } else {
    notify { "$::timezone is not UTC": }
  }

Puppet treats whatever follows an if keyword as an expression and evaluates it. If the expression evaluates to true, Puppet will execute the code within the curly braces.

Optionally, you can add an else branch, which will be executed if the expression evaluates to false.

Here are some more tips on using if statements.

if $::timezone == 'UTC' {
  notify { 'Universal Time Coordinated': }
} elseif $::timezone == 'GMT' {
  notify { 'Greenwich Mean Time': }
} else {
  notify { "$::timezone is not UTC": }
}

if $::timezone == 'UTC' {
  
}

if $::timezone != 'UTC' {
  …
}

if $::uptime_days > 365 {
  notify { 'Time to upgrade your kernel!': }
}

if $::mtu_eth0 <= 1500 {
  notify {"Not Jumbo Frames": }
}

if ($::uptime_days > 365) and ($::kernel == 'Linux') {
  …
}

if ($role == 'webserver') and ( ($datacenter == 'A') or ($datacenter == 'B') ) {
  …
}

This is one example of using a regular expression in a conditional statement. Add the following to your manifest:

if $::architecture =~ /64/ {
  notify { '64Bit OS Installed': }
} else {
  notify { 'Upgrade to 64Bit': }
  fail('Not 64 Bit')
}

Puppet treats the text supplied between the forward slashes as a regular expression, specifying the text to be matched. If the match succeeds, the if expression will be true and so the code between the first set of curly braces will be executed. In this example, we used a regular expression because different distributions have different ideas on what to call 64bit; some use amd64, while others use x86_64. The only thing we can count on is the presence of the number 64 within the fact. Some facts that have version numbers in them are treated as strings to Puppet. For instance, $::facterversion. On my test system, this is 2.0.1, but when I try to compare that with 2, Puppet fails to make the comparison:

Error: comparison of String with 2 failed at /home/thomas/.puppet/manifests/version.pp:1 on node cookbook.example.com

If you wanted instead to do something if the text does not match, use !~ rather than =~:

if $::kernel !~ /Linux/ {
  notify { 'Not Linux, could be Windows, MacOS X, AIX, or ?': }
}

Regular expressions are very powerful, but can be difficult to understand and debug. If you find yourself using a regular expression so complex that you can't see at a glance what it does, think about simplifying your design to make it easier. However, one particularly useful feature of regular expressions is the ability to capture patterns.

$input = 'Puppet is better than manual configuration'
if $input =~ /(.*) is better than (.*)/ {
  notify { "You said '${0}'. Looks like you're comparing ${1}
    to ${2}!": }
}

Here are some examples of selector and case statements:

Our example demonstrates both the selector and the case statement, so let's see in detail how each of them works.

Once you've got a grip of the basic use of selectors and case statements, you may find the following tips useful.

case $::lsbdistdescription {
  /Ubuntu (.+)/: {
    notify { "You have Ubuntu version ${1}": }
  }
  /CentOS (.+)/: {
    notify { "You have CentOS version ${1}": }
  }
  default: {}
}

$lunch = 'Filet mignon.'
$lunchtype =  $lunch ? {
  /fries/ => 'unhealthy',
  /salad/ => 'healthy',
  default => 'unknown',
}

notify { "Your lunch was ${lunchtype}": }

t@mylaptop ~ $ puppet apply lunchtype.pp
Notice: Your lunch was unknown
Notice: /Stage[main]/Main/Notify[Your lunch was unknown]/message: defined 'message' as 'Your lunch was unknown'

The following steps will show you how to use the in operator:

The value of an in expression is Boolean (true or false) so you can assign it to a variable:

$debianlike = $::operatingsystem in [ 'Debian', 'Ubuntu' ]

if $debianlike {
  notify { 'You are in a maze of twisty little packages, all alike': }
}

Follow these steps to build the example:

The regsubst function takes at least three parameters: source, pattern, and replacement. In our example, we specified the source string as $::ipaddress, which, on this machine, is as follows:

192.168.122.148

We specify the pattern function as follows:

(.*)\..*

We specify the replacement function as follows:

\1.0

The pattern captures all of the string up to the last period (\.) in the \1 variable. We then match on .*, which matches everything to the end of the string, so when we replace the string at the end with \1.0, we end up with only the network portion of the IP address, which evaluates to the following:

192.168.122.0

We could have got the same result in other ways, of course, including the following:

$class_c = regsubst($::ipaddress, '\.\d+$', '.0')

Here, we only match the last octet and replace it with .0, which achieves the same result without capturing.

The pattern function can be any regular expression, using the same (Ruby) syntax as regular expressions in if statements.

Many of the experimental features deal with how code is evaluated, for example, in an earlier example we compared the value of the $::facterversion fact with a number, but the value is treated as a string so the code fails to compile. Using the future parser, the value is converted and no error is reported as shown in the following command line output:

t@cookbook:~/.puppet/manifests$ puppet apply --parser=future version.pp 
Notice: Compiled catalog for cookbook.example.com in environment production in 0.36 seconds
Notice: Finished catalog run in 0.03 seconds

$apache = $::osfamily ? {
  'Debian' => 'apache2',
  'RedHat' => 'httpd'
} 
$packages = ['memcached'] << $apache
package {$packages: ensure => installed}

$sysadmins = [ 'thomas','john','josko' ]
$appowners = [ 'mike', 'patty', 'erin' ]
$users = $sysadmins + $appowners
notice ($users)

t@cookbook:~/.puppet/manifests$ puppet apply --parser=future concat.pp Notice: [thomas, john, josko, mike, patty, erin]
Notice: Compiled catalog for cookbook.example.com in environment production in 0.36 seconds
Notice: Finished catalog run in 0.03 seconds
Merging Hashes

$iface = {
  'name' => 'eth0',
  'ip'   => '192.168.0.1',
  'mac'  => '52:54:00:4a:60:07' 
}  + {'route' => '192.168.0.254'}
notice ($iface)

t@cookbook:~/.puppet/manifests$ puppet apply --parser=future hash2.pp
Notice: {route => 192.168.0.254, name => eth0, ip => 192.168.0.1, mac => 52:54:00:4a:60:07}
Notice: Compiled catalog for cookbook.example.com in environment production in 0.36 seconds
Notice: Finished catalog run in 0.03 seconds

$count = [1,2,3,4,5]
$sum = reduce($count) | $total, $i | { $total + $i }
notice("Sum is $sum")

t@cookbook:~/.puppet/manifests$ puppet apply --parser future lambda.pp
Notice: Sum is 15
Notice: Compiled catalog for cookbook.example.com in environment production in 0.36 seconds
Notice: Finished catalog run in 0.03 seconds

$filter = filter ($count) | $i | { $i > 3 } notice("Filtered array is $filter")

Notice: Filtered array is [4, 5]

$map = map ($count) | $i | { $i * $i } notice("Square of array is $map")

Notice: Square of array is [1, 4, 9, 16, 25]

$firewall_rules = ['192.168.0.1','80','192.168.0.10','443']
slice ($firewall_rules,2) |$ip, $port| { notice("Allow $ip on $port") }

Notice: Allow 192.168.0.1 on 80
Notice: Allow 192.168.0.10 on 443

slice ($firewall_rules,2) |$ip, $port| { firewall {"$port from $ip": dport  => $port, source => "$ip", action => 'accept', }
}

each ($count) |$c| { notice($c) }

Notice: 1
Notice: 2
Notice: 3
Notice: 4
Notice: 5

If your Linux distribution uses APT for package management, go to http://apt.puppetlabs.com/ and download the appropriate Puppet labs release package for your distribution. For our wheezy cookbook node, we will use http://apt.puppetlabs.com/puppetlabs-release-wheezy.deb.

If you are using a Linux distribution that uses YUM for package management, go to http://yum.puppetlabs.com/ and download the appropriate Puppet labs release package for your distribution.

Now that we have a method to install Puppet on our nodes, we need to turn our attention to keeping our Puppet manifests organized. In the next section, we will see how to use Git to keep our code organized and consistent.

In this section, we'll import your existing manifest files into Git. If you have created a Puppet directory in a previous section use that, otherwise, use your existing manifest directory.

In this example, we'll create a new Git repository on a server accessible from all our nodes. There are several steps we need to take to have our code held in a Git repository:

Follow these steps:

Git tracks changes to files, and stores a complete history of all changes. The history of the repo is made up of commits. A commit represents the state of the repo at a particular point in time, which you create with the git commit command and annotate with a message.

You've now added your Puppet manifest files to the repo and created your first commit. This updates the history of the repo, but only in your local working copy. To synchronize the changes with the git.example.com copy, the git push command pushes all changes made since the last sync.

Now that you have a central Git repository for your Puppet manifests, you can check out multiple copies of it in different places and work on them before committing your changes. For example, if you're working in a team, each member can have their own local copy of the repo and synchronize changes with the others via the central server. You may also choose to use GitHub as your central Git repository server. GitHub offers free Git repository hosting for public repositories, and you can pay for GitHub's premium service if you don't want your Puppet code to be publicly available.

In the next section, we will use our Git repository for both centralized and decentralized Puppet configurations.

Create a new test node, call this new node whatever you wish, I'll use testnode for mine. Install Puppet on the machine as we have previously done.

Create a bootstrap.pp manifest that will perform the following configuration steps on our new node:

First, our bootstrap.pp manifest ensures that Git is installed. The manifest then goes on to ensure that the ssh key for the Git user on git.example.com is installed into the Puppet user's home directory (/var/lib/puppet by default). The manifest then ensures that the host key for git.example.com is trusted by the Puppet user. With ssh configured, the bootstrap ensures that /etc/puppet/cookbook exists and is a directory.

We then use an exec to have Git clone the repository into /etc/puppet/cookbook. With all the code in place, we then call puppet apply a final time to deploy the code from the repository. In a production setting, you would distribute the bootstrap.pp manifest to all your nodes, possibly via an internal web server, using a method similar to curl http://puppet/bootstrap.pp >bootstrap.pp && puppet apply bootstrap.pp

Follow these steps:

Now, whenever you need to run Puppet, you can simply run papply. In future, when we apply Puppet changes, I'll ask you to run papply instead of the full puppet apply command.

As you've seen, to run Puppet on a machine and apply a specified manifest file, we use the puppet apply command:

puppet apply manifests/site.pp

When you're using modules (such as the Puppet module we just created), you also need to tell Puppet where to search for modules, using the modulepath argument:

puppet apply manifests/nodes.pp \--modulepath=/home/ubuntu/puppet/modules

In order to run Puppet with the root privileges it needs, we have to put sudo before everything:

sudo puppet apply manifests/nodes.pp \--modulepath=/home/ubuntu/puppet/modules

Finally, any additional arguments passed to papply will be passed through to Puppet itself, by adding the $* parameter:

sudo puppet apply manifests/nodes.pp \--modulepath=/home/ubuntu/puppet/modules $*

That's a lot of typing, so putting this in a script makes sense. We've added a Puppet file resource that will deploy the script to /usr/local/bin and make it executable:

file { '/usr/local/bin/papply': source => 'puppet:///modules/puppet/papply.sh', mode   => '0755',}

Finally, we include the Puppet module in our default node declaration:

node default {
  include base
  include puppet
}

You can do the same for any other nodes managed by Puppet.

You'll need the Git repo we set up in the Managing your manifests with Git and Creating a decentralized Puppet architecture recipes, and the papply script from the Writing a papply script recipe. You'll need to apply the bootstrap.pp manifest we created to install ssh keys to download the latest repository.

Follow these steps:

When we created the bootstrap.pp manifest, we made sure that the Puppet user can checkout the Git repository using an ssh key. This enables the Puppet user to run the Git pull in the cookbook directory unattended. We've also added the pull-updates script, which does this and runs Puppet if any changes are pulled:

#!/bin/sh
cd /etc/puppet/cookbook
sudo –u puppet git pull && papply

We deploy this script to the node with Puppet:

file { '/usr/local/bin/pull-updates':
  source => 'puppet:///modules/puppet/pull-updates.sh',
  mode   => '0755',
}

Finally, we've created a cron job that runs pull-updates at regular intervals (every 10 minutes, but feel free to change this if you need to):

cron { 'run-puppet':
  ensure  => 'present',
  command => '/usr/local/bin/pull-updates',
  minute  => '*/10',
  hour    => '*',
}

Congratulations, you now have a fully-automated Puppet infrastructure! Once you have applied the bootstrap.pp manifest, run Puppet on the repository; the machine will be set up to pull any new changes and apply them automatically.

So, for example, if you wanted to add a new user account to all your machines, all you have to do is add the account in your working copy of the manifest, and commit and push the changes to the central Git repository. Within 10 minutes, it will automatically be applied to every machine that's running Puppet.

Install httpd on a centrally accessible server and create a password protected area to store the bootstrap script. In my example, I'll use the Git server I set up previously, git.example.com. Start by creating a directory in the root of your web server:

# cd /var/www/html
# mkdir bootstrap

Now perform the following steps:

Now that you have a safe location to store the bootstrap script, create a bootstrap script for each OS you support in the bootstrap directory. In this example, I'll show you how to do this for a Red Hat Enterprise Linux 6-based distribution.

Create a script named el6.sh in the bootstrap directory with the following contents:

#!/bin/bash

# bootstrap for EL6 distributions
SERVER=git.example.com
LOCATION=/bootstrap
BOOTSTRAP=bootstrap.pp
USER=bootstrap
PASS=cookbook

# install puppet
curl http://yum.puppetlabs.com/RPM-GPG-KEY-puppetlabs >/etc/pki/rpm-gpg/RPM-GPG-KEY-puppetlabs
rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-puppetlabs
yum -y install http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm
yum -y install puppet
# download bootstrap
curl --user $USER:$PASS http://$SERVER/$LOCATION/$BOOTSTRAP >/tmp/$BOOTSTRAP
# apply bootstrap
cd /tmp
puppet apply /tmp/$BOOTSTRAP
# apply puppet
puppet apply --modulepath /etc/puppet/cookbook/modules /etc/puppet/cookbook/manifests/site.pp

The apache configuration only permits access to the bootstrap directory with a username and password combination. We supply these with the --user argument to curl, thereby getting access to the file. We use a pipe (|) to redirect the output of curl into bash. This causes bash to execute the script. We write our bash script like we would any other bash script. The bash script downloads our bootstrap.pp manifest and applies it. Finally, we apply the Puppet manifest from the Git repository and the machine is configured as a member of our decentralized infrastructure.

To support another operating system, we only need to create a new bash script. All Linux distributions will support bash scripting, Mac OS X does as well. Since we placed much of our logic into the bootstrap.pp manifest, the bootstrap script is quite minimal and easy to port to new operating systems.

The Puppet master process is started by running puppet master; most Linux distributions have start and stop scripts for the Puppet master in a separate package. To get started, we'll create a new debian server named puppet.example.com.

The Puppet master package includes the start and stop scripts for the Puppet master service. We use Puppet to install the package and start the service. Once the service is started, we can point another node at the Puppet master (you might need to disable the host-based firewall on your machine).

When we ran puppet agent, Puppet looked for a host named puppet.example.com (since our test node is in the example.com domain); if it couldn't find that host, it would then look for a host named Puppet. We can specify the server to contact with the --server option to puppet agent. When we installed the Puppet master package and started the Puppet master service, Puppet created default SSL certificates based on our hostname. In the next section, we'll see how to create an SSL certificate that has multiple DNS names for our Puppet server.

Install the Puppet master package if you haven't already done so. You will then need to start the Puppet master service at least once to create a certificate authority (CA).

The steps are as follows:

When your puppet agents connect to the Puppet server, they look for a host called Puppet, they then look for a host called Puppet.[your domain]. If your clients are in different domains, then you need your Puppet master to reply to all the names correctly. By removing the existing certificate and generating a new one, you can have your Puppet master reply to multiple DNS names.

Install the puppetmaster-passenger package:

# puppet resource package puppetmaster-passenger ensure=installed
Notice: /Package[puppetmaster-passenger]/ensure: ensure changed 'purged'
 to 'present'
package { 'puppetmaster-passenger':
  ensure => '3.7.0-1puppetlabs1',
}

The steps are as follows:

The passenger configuration file uses the existing Puppet master certificates to listen on port 8140 and handles all the SSL communication between the server and the client. Once the certificate information has been dealt with, the connection is handed off to a ruby process started from passenger using the command line arguments from the config.ru file.

In this case, the $0 variable is set to master and the arguments variable is set to --rack --confdir /etc/puppet --vardir /var/lib/puppet; this is equivalent to running the following from the command line:

puppet master --rack --confdir /etc/puppet --vardir /var/lib/puppet

You can add additional configuration parameters to the config.ru file to further alter how Puppet runs when it's running through passenger. For instance, to enable debugging on the passenger Puppet master, add the following line to config.ru before the run Puppet::Util::CommandLine.new.execute line:

ARGV << "--debug"

Set the environmentpath function of your installation by adding a line to the [main] section of /etc/puppet/puppet.conf as follows:

[main]
...
environmentpath=/etc/puppet/environments

The steps are as follows:

Each environment can have its own modulepath if you create an environment.conf file within the environment directory. More information on environments can be found on the Puppet labs website at https://docs.puppetlabs.com/puppet/latest/reference/environments.html.

Install the PuppetDB module in the production environment you created in the previous recipe. If you didn't create directory environments, don't worry, using puppet module install will install the module to the correct location for your installation with the following command:

root@puppet:~# puppet module install puppetlabs-puppetdb
Notice: Preparing to install into /etc/puppet/environments/production/modules ...
Notice: Downloading from https://forgeapi.puppetlabs.com ...
Notice: Installing -- do not interrupt ...
/etc/puppet/environments/production/modules
└─┬ puppetlabs-puppetdb (v3.0.1)
  ├── puppetlabs-firewall (v1.1.3)
  ├── puppetlabs-inifile (v1.1.3)
  └─┬ puppetlabs-postgresql (v3.4.2)
    ├─┬ puppetlabs-apt (v1.6.0)
    │ └── puppetlabs-stdlib (v4.3.2)
    └── puppetlabs-concat (v1.1.0)

Now that our Puppet master has the PuppetDB module installed, we need to apply the PuppetDB module to our Puppet master, we can do this in the site manifest. Add the following to your (production) site.pp:

node puppet {
  class { 'puppetdb': }
  class { 'puppetdb::master::config': 
    puppet_service_name => 'apache2',
  }
}

Run puppet agent to apply the puppetdb class and the puppetdb::master::config class:

root@puppet:~# puppet agent -t
Info: Caching catalog for puppet
Info: Applying configuration version '1410416952'
...
Info: Class[Puppetdb::Server::Jetty_ini]: Scheduling refresh of Service[puppetdb]
Notice: Finished catalog run in 160.78 seconds

The PuppetDB module is a great example of how a complex configuration task can be puppetized. Simply by adding the puppetdb class to our Puppet master node, Puppet installed and configured postgresql and puppetdb.

When we called the puppetdb::master::config class, we set the puppet_service_name variable to apache2, this is because we are running Puppet through passenger. Without this line our agent would try to start the puppetmaster process instead of apache2.

The agent then set up the configuration files for PuppetDB and configured Puppet to use PuppetDB. If you look at /etc/puppet/puppet.conf, you'll see the following two new lines:

storeconfigs = true
storeconfigs_backend = puppetdb

Now that PuppetDB is configured and we've had a successful agent run, PuppetDB will have data we can query:

root@puppet:~# puppet node status puppet
puppet
Currently active
Last catalog: 2014-09-11T06:45:25.267Z
Last facts: 2014-09-11T06:45:22.351Z

Hiera should have already been installed as a dependency on your Puppet master. If it has not already, install it using Puppet:

root@puppet:~# puppet resource package hiera ensure=installed
package { 'hiera':
  ensure => '1.3.4-1puppetlabs1',
}

Hiera uses a hierarchy to search through a set of yaml files to find the appropriate values. We defined this hierarchy in hiera.yaml with the single entry for common.yaml. We used the hiera function in site.pp to lookup the value for message and store that value in the variable $message. The values used for the definition of the hierarchy can be any facter facts defined about the system. A common hierarchy is shown as:

:hierarchy:
  - hosts/%{hostname}
  - os/%{operatingsystem}
  - network/%{network_eth0}
  - common

Hiera can be used for automatic parameter lookup with parameterized classes. For example, if you have a class named cookbook::example with a parameter named publisher, you can include the following in a Hiera yaml file to automatically set this parameter:

cookbook::example::publisher: 'PacktPub'

Another often used fact is environment you may reference the environment of the client node using %{environment} as shown in the following hierarchy:

:hierarchy:
hosts/%{hostname}
os/%{operatingsystem}
environment/%{environment}
common

The default Hiera function returns the first match to the search key, you can also use hiera_array and hiera_hash to search and return all values stored in Hiera.

Hiera can also be searched from the command line as shown in the following command line (note that currently the command line Hiera utility uses /etc/hiera.yaml as its configuration file whereas the Puppet master uses /etc/puppet/hiera.yaml):

root@puppet:/etc/puppet# rm /etc/hiera.yaml 
root@puppet:/etc/puppet# ln -s /etc/puppet/hiera.yaml /etc/
root@puppet:/etc/puppet# hiera message
Default Message

Install and configure Hiera as in the last section and use the hierarchy defined in the previous recipe that includes a hosts/%{hostname} entry.

The following are the steps:

Hiera searches the hierarchy for files that match the values returned by facter. In this case, the cookbook-test.yaml file is found by substituting the hostname of the node into the search path /etc/puppet/hieradata/hosts/%{hostname}.yaml.

Using Hiera, it is possible to greatly reduce the complexity of your Puppet code. We will use yaml files for separate values, where previously you had large case statements or nested if statements.

To set up hiera-gpg, follow these steps:

In this example, we'll create a piece of encrypted data and retrieve it using hiera-gpg as follows:

When you install hiera-gpg, it adds to Hiera, the ability to decrypt .gpg files. So you can put any secret data into a .yaml file that you then encrypt to the appropriate key with GnuPG. Only machines that have the right secret key will be able to access this data.

For example, you might encrypt the MySQL root password using hiera-gpg and install the corresponding key only on your database servers. Although other machines may also have a copy of the secret.gpg file, it's not readable to them unless they have the decryption key.

You might also like to know about hiera-eyaml, another secret-data backend for Hiera that supports encryption of individual values within a Hiera data file. This could be handy if you need to mix encrypted and unencrypted facts within a single file. Find out more about hiera-eyaml at https://github.com/TomPoulton/hiera-eyaml.

Install the msgpack gem onto your Puppet master and your nodes. Use Puppet to do the work for you with Puppet resource. You may need to install the ruby-dev or ruby-devel package on your nodes/server at this point:

t@ckbk:~$ sudo puppet resource package msgpack ensure=installedprovider=gem
Notice: /Package[msgpack]/ensure: created
package { 'msgpack':
  ensure => ['0.5.8'],
}

Set the preferred_serialization_format to msgpack in the [agent] section of your nodes puppet.conf file:

[agent]
preferred_serialization_format=msgpack

The master will be sent this option when the node begins communicating with the master. Any classes that support serialization with msgpack will be transmitted with msgpack.Serialization of the data between nodes and the master will in theory increase the speed at which nodes communicate by optimizing the data that is travelling between them. This feature is still experimental.

Follow these steps:

The check_syntax.sh script will prevent you from committing any files with syntax errors when it is used as the pre-commit hook for Git:

t@mylaptop:~/puppet$ git commit -m "test commit"
Error: Could not parse for environment production: Syntax error at
  '}' at line 3
Error: Try 'puppet help parser validate' for usage
manifests/nodes.pp: Error: 1 syntax errors found, aborting commit.

If you add the hooks directory to your Git repo, anyone who has a checkout can copy the script into their local hooks directory to get this syntax checking behavior.

Follow these steps to get started:

Perform the following steps:

We created a bare repository on the Puppet master that we then use as a remote for the repository on git.example.com (remote repositories must be bare). We then clone that bare repository into the production directory. We add the bare repository on puppet.example.com as a remote to the bare repository on git.example.com. We then create a post-receive hook in the repository on git.example.com.

The hook issues a Git push to the Puppet master bare repository. We then update the production directory from the updated bare repository on the Puppet master. In the next section, we'll modify the hook to use branches.

In the previous section, we created a production directory that was based on the master branch; we'll remove that directory now:

puppet@puppet:/etc/puppet/environments$ mv production production.master

Modify the post-receive hook to accept a branch variable. The hook will use this variable to create a directory on the Puppet master as follows:

#!/bin/sh

read oldrev newrev refname
branch=${refname#*\/*\/}

git push puppetmaster $branch
ssh puppet@puppet.example.com "if [ ! -d 
/etc/puppet/environments/$branch ]; then git clone
 /etc/puppet/environments/puppet.git
 /etc/puppet/environments/$branch; fi; cd
 /etc/puppet/environments/$branch; git checkout $branch; git pull"

Modify your README file again and push to the repository on git.example.com:

t@mylaptop puppet$ git add README
t@mylaptop puppet$ git commit -m "Adding README"
[master 539d9f8] Adding README
 1 file changed, 1 insertion(+)
t@mylaptop puppet$ git push
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 374 bytes | 0 bytes/s, done.
Total 3 (delta 1), reused 0 (delta 0)
remote: To puppet@puppet.example.com:/etc/puppet/environments/puppet.git
remote:    0d6b49f..539d9f8  master -> master
remote: Cloning into '/etc/puppet/environments/master'...
remote: done.
remote: Already on 'master'
remote: Already up-to-date.
To git@git.example.com:repos/puppet.git
   0d6b49f..539d9f8  master -> master

The hook now reads in the refname and parses out the branch that is being updated. We use that branch variable to clone the repository into a new directory and check out the branch.

Now when we want to create a new environment, we can create a new branch in the Git repository. The branch will create a directory on the Puppet master. Each branch of the Git repository represents an environment on the Puppet master:

Now whenever we create a new branch, a corresponding directory is created in our environment's directory. A one-to-one mapping is established between environments and branches.

Here are the steps to refactor using arrays of resources:

Most of Puppet's resource types can accept an array instead of a single name, and will create one instance for each of the elements in the array. All the parameters you provide for the resource (for example, ensure => installed) will be assigned to each of the new resource instances. This shorthand will only work when all the resources have the same attributes.

To show you how to use resource defaults, we'll create an apache module. Within this module we will specify that the default owner and group are the apache user as follows:

The resource default we defined specifies the owner, group, and mode for all file resources within this class (also known as within this scope). Unless you specifically override a resource default, the value for an attribute will be taken from the default.

You can specify resource defaults for any resource type. You can also specify resource defaults in site.pp. I find it useful to specify the default action for Package and Service resources as follows:

  Package { ensure => 'installed' }
  Service {
    hasrestart => true,
    enable     => true,
    ensure     => true,
  }

With these defaults, whenever you specify a package, the package will be installed. Whenever you specify a service, the service will be started and enabled to run at boot. These are the usual reasons you specify packages and services, most of the time these defaults will do what you prefer and your code will be cleaner. When you need to disable a service, simply override the defaults.

The following steps will show you how to create a definition:

You can think of a defined type (introduced with the define keyword) as a cookie-cutter. It describes a pattern that Puppet can use to create lots of similar resources. Any time you declare a tmpfile instance in your manifest, Puppet will insert all the resources contained in the tmpfile definition.

In our example, the definition of tmpfile contains a single file resource whose content is Hello, world\n and whose path is /tmp/${name}. If you declared an instance of tmpfile with the name foo:

tmpfile { 'foo': }

Puppet will create a file with the path /tmp/foo. In other words, ${name} in the definition will be replaced by the name of any actual instance that Puppet is asked to create. It's almost as though we created a new kind of resource: tmpfile, which has one parameter—its name.

Just like with regular resources, we don't have to pass just one title; as in the preceding example, we can provide an array of titles and Puppet will create as many resources as required.

In the example, we created a definition where the only parameter that varies between instances is the name parameter. But we can add whatever parameters we want, so long as we declare them in the definition in parentheses after the name parameter, as follows:

  define tmpfile($greeting) {
    file { "/tmp/${name}": content => $greeting,
    }
  }

Next, pass values to them when we declare an instance of the resource:

  tmpfile{ 'foo':
    greeting => "Good Morning\n",
  }

You can declare multiple parameters as a comma-separated list:

  define webapp($domain,$path,$platform) {
    ...
  }
  webapp { 'mywizzoapp':
    domain   => 'mywizzoapp.com',
    path     => '/var/www/apps/mywizzoapp',
    platform => 'Rails',
  }

You can also declare default values for any parameters that aren't supplied, thus making them optional:

  define tmpfile($greeting,$mode='0644') {
    ...
  }

This is a powerful technique for abstracting out everything that's common to certain resources, and keeping it in one place so that you don't repeat yourself. In the preceding example, there might be many individual resources contained within webapp: packages, config files, source code checkouts, virtual hosts, and so on. But all of them are the same for every instance of webapp except the parameters we provide. These might be referenced in a template, for example, to set the domain for a virtual host.

To help you find out if you're running on a particular node or class of nodes all nodes are automatically tagged with the node name and the names of any classes they include. Here's an example that shows you how to use tagged to get this information:

You can use tags to create a collection of resources, and then make the collection a dependency for some other resource. For example, say some service depends on a config file that is built from a number of file snippets, as in the following example:

  class firewall::service {
    service { 'firewall': ... 
    }
    File <| tag == 'firewall-snippet' |> ~> Service['firewall'] 
  }
  class myapp { 
    file { '/etc/firewall.d/myapp.conf': tag => 'firewall-snippet', ... 
    } 
  }

Here, we've specified that the firewall service should be notified if any file resource tagged firewall-snippet is updated. All we need to do to add a firewall config snippet for any particular application or service is to tag it firewall-snippet, and Puppet will do the rest.

Although we could add a notify => Service["firewall"] function to each snippet resource if our definition of the firewall service were ever to change, we would have to hunt down and update all the snippets accordingly. The tag lets us encapsulate the logic in one place, making future maintenance and refactoring much easier.

Here are the steps to create an example of using run stages:

Let's examine this code in detail to see what's happening. First, we declare the run stages first and last, as follows:

  stage { 'first': before => Stage['main'] }
  stage { 'last': require => Stage['main'] }

For the first stage, we've specified that it should come before main. That is, every resource marked as being in the first stage will be applied before any resource in the main stage (the default stage).

The last stage requires the main stage, so no resource in the last stage can be applied until after every resource in the main stage.

We then declare some classes that we'll later assign to these run stages:

  class me_first {
    notify { 'This will be done first': }
  }
  class me_last {
    notify { 'This will be done last': }
  }

We can now put it all together and include these classes on the node, specifying the run stages for each as we do so:

  class { 'me_first': stage => 'first',
  }
  class { 'me_last': stage => 'last',
  }

Note that in the class declarations for me_first and me_last, we didn't have to specify that they take a stage parameter. The stage parameter is another metaparameter, which means it can be applied to any class or resource without having to be explicitly declared. When we ran puppet agent on our Puppet node, the notify from the me_first class was applied before the notifies from first_class and second_class. The notify from me_last was applied after the main stage, so it comes after the two notifies from first_class and second_class. If you run puppet agent multiple times, you will see that the notifies from first_class and second_class may not always appear in the same order but the me_first class will always come first and the me_last class will always come last.

You can define as many run stages as you like, and set up any ordering for them. This can greatly simplify a complicated manifest that would otherwise require lots of explicit dependencies between resources. Beware of accidentally introducing dependency cycles, though; when you assign something to a run stage you're automatically making it dependent on everything in prior stages.

You may like to define your stages in the site.pp file instead, so that at the top level of the manifest, it's easy to see what stages are available.

Gary Larizza has written a helpful introduction to using run stages, with some real-world examples, on his website:

http://garylarizza.com/blog/2011/03/11/using-run-stages-with-puppet/

A caveat: many people don't like to use run stages, feeling that Puppet already provides sufficient resource ordering control, and that using run stages indiscriminately can make your code very hard to follow. The use of run stages should be kept to a minimum wherever possible. There are a few key examples where the use of stages creates less complexity. The most notable is when a resource modifies the system used to install packages on the system. It helps to have a package management stage that comes before the main stage. When packages are defined in the main (default) stage, your manifests can count on the updated package management configuration information being present. For instance, for a Yum-based system, you would create a yumrepos stage that comes before main. You can specify this dependency using chaining arrows as shown in the following code snippet:

  stage {'yumrepos': }
  Stage['yumrepos'] -> Stage['main']

We can then create a class that creates a Yum repository (yumrepo) resource and assign it to the yumrepos stage as follows:

  class {'yums': stage => 'yumrepos',
  }
  class yums {
    notify {'always before the rest': }
    yumrepo {'testrepo': baseurl => 'file:///var/yum', ensure  => 'present',
    }
  }

For Apt-based systems, the same example would be a stage where Apt sources are defined. The key with stages is to keep their definitions in your site.pp file where they are highly visible and to only use them sparingly where you can guarantee that you will not introduce dependency cycles.

We'll create two modules to store our roles and profiles. Roles will contain one or more profiles. Each role or profile will be defined as a subclass, such as profile::base

Breaking down the parts of the web server configuration into different profiles allows us to apply those parts independently. We created a base profile that we can expand to include all the resources we would like applied to all nodes. Our roles::webserver class simply includes the base and apache classes.

As we'll see in the next section, we can pass parameters to classes to alter how they work. In our roles::webserver class, we can use the class instantiation syntax instead of include, and override it with parameters in the classes. For instance, to pass a parameter to the base class, we would use:

  class {'profiles::base':
    parameter => 'newvalue'
  }

where we previously used:

include profiles::base

In this example, we'll create a definition that accepts parameters:

The class definition class eventmachine($version) { is just like a normal class definition except it specifies that the class takes one parameter: $version. Inside the class, we've defined a package resource:

  package { 'eventmachine':
    provider => gem,
    ensure   => $version,
  }

This is a gem package, and we're requesting to install version $version.

Include the class on a node, instead of the usual include syntax:

include eventmachine

On doing so, there will be a class statement:

  class { 'eventmachine':
    version => '1.0.3',
  }

This has the same effect but also sets a value for the parameter as version.

You can specify multiple parameters for a class as:

  class mysql($package, $socket, $port) {

Then supply them in the same way:

  class { 'mysql':
    package => 'percona-server-server-5.5',
    socket  => '/var/run/mysqld/mysqld.sock',
    port    => '3306',
  }

class mysql($package, $socket, $port='3306') {

  class mysql(
    package = percona-server-server-5.5",
    socket  = '/var/run/mysqld/mysqld.sock',
    port    = '3306') {

Install and configure hiera as we did in Chapter 2, Puppet Infrastructure. Create a global or common yaml file; this will serve as the default for all values.

When we instantiate the mysql class in our manifest, we provided no values for any of the attributes. Puppet knows to look for a value in Hiera that matches class_name::parameter_name: or ::class_name::parameter_name:.

When Puppet finds a value, it uses it as the parameter for the class. If Puppet fails to find a value in Hiera and no default is defined, a catalog failure will result in the following command line:

Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Must pass package to Class[Mysql] at /etc/puppet/environments/production/manifests/site.pp:6 on node hiera-test.example.com

This error indicates that Puppet would like a value for the parameter package.

You can define a Hiera hierarchy and supply different values for parameters based on facts. You could, for instance, have %{::osfamily} in your hierarchy and have different yaml files based on the osfamily parameter (RedHat, Suse, and Debian).

Here are some examples of how to make your manifests more portable:

The advantage of this approach (which could be called top-down) is that you only need to make your choices once. The alternative, bottom-up approach would be to have a selector or case statement everywhere a setting is used:

  service { $::operatingsystem? {
    /Ubuntu|Debian/ => 'ssh', default         => 'sshd' }: ensure => running,
  }

This not only results in lots of duplication, but makes the code harder to read. And when a new operating system is added to the mix, you'll need to make changes throughout the whole manifest, instead of just in one place.

If you are writing a module for public distribution (for example, on Puppet Forge), making your module as cross-platform as possible will make it more valuable to the community. As far as you can, test it on many different distributions, platforms, and architectures, and add the appropriate variables so that it works everywhere.

If you use a public module and adapt it to your own environment, consider updating the public version with your changes if you think they might be helpful to other people.

Even if you are not thinking of publishing a module, bear in mind that it may be in production use for a long time and may have to adapt to many changes in the environment. If it's designed to cope with this from the start, it'll make life easier for you or whoever ends up maintaining your code.

Here's an example of using Facter facts in a manifest:

Facter provides a standard way for manifests to get information about the nodes to which they are applied. When you refer to a fact in a manifest, Puppet will query Facter to get the current value and insert it into the manifest. Facter facts are top scope variables.

You can also use facts in ERB templates. For example, you might want to insert the node's hostname into a file, or change a configuration setting for an application based on the memory size of the node. When you use fact names in templates, remember that they don't need a dollar sign because this is Ruby, not Puppet:

$KLogPath <%= case @kernelversion when '2.6.31' then
'/var/run/rsyslog/kmsg' else '/proc/kmsg' end %>

When referring to facts, use the @ syntax. Variables that are defined at the same scope as the function call to template can also be referenced with the @ syntax. Out of scope variables should use the scope function. For example, to reference the mysql::port variable we defined earlier in the mysql modules, use the following:

MySQL Port = <%= scope['::mysql::port'] %>

Applying this template results in the following file:

[root@hiera-test ~]# puppet agent -t
...
Info: Caching catalog for hiera-test.example.com
Notice: /Stage[main]/Erb/File[/tmp/template-test]/ensure: defined content as '{md5}96edacaf9747093f73084252c7ca7e67'
Notice: Finished catalog run in 0.41 seconds [root@hiera-test ~]# cat /tmp/template-test
MySQL Port = 3306

Follow these steps to prepare to run the example:

This example calls the external script we created previously and gets its output:

The generate function runs the specified script or program and returns the result, in this case, a cheerful message from Ruby.

This isn't terribly useful as it stands but you get the idea. Anything a script can do, print, fetch, or calculate, for example, the results of a database query, can be brought into your manifest using generate. You can also, of course, run standard UNIX utilities such as cat and grep.

If you need to pass arguments to the executable called by generate, add them as extra arguments to the function call:

$message = generate('/bin/cat', '/etc/motd')

Puppet will try to protect you from malicious shell calls by restricting the characters you can use in a call to generate, so shell pipes and redirection aren't allowed, for example. The simplest and safest thing to do is to put all your logic into a script and then call that script.

Here's an example of using the shellquote function:

First we define the $source and $target variables, which are the two filenames we want to use in the command line:

$source = 'Hello Jerry'
$target = 'Hello... Newman'

Then we call shellquote to concatenate these variables into a quoted, space-separated string as follows:

$argstring = shellquote($source, $target)

Then we put together the final command line:

$command = "/bin/mv ${argstring}"

The result will be:

/bin/mv "Hello Jerry" "Hello... Newman"

This command line can now be run with an exec resource. What would happen if we didn't use shellquote?

$source = 'Hello Jerry'
$target = 'Hello... Newman'
$command = "/bin/mv ${source} ${target}"
notify { $command: }

Notice: /bin/mv Hello Jerry Hello... Newman

This won't work because mv expects space-separated arguments, so it will interpret this as a request to move three files Hello, Jerry, and Hello... into a directory named Newman, which probably isn't what we want.

Install the puppetlabs-stdlib module using puppet:

t@mylaptop ~ $ puppet module install puppetlabs-stdlib
Notice: Preparing to install into /home/thomas/.puppet/modules ...
Notice: Downloading from https://forgeapi.puppetlabs.com ...
Notice: Installing -- do not interrupt ...
/home/thomas/.puppet/modules
└── puppetlabs-stdlib (v4.5.1)

This installs the module from the forge into my user's puppet directory; to install into the system directory, run the command as root or use sudo. For the purpose of this example, we'll continue working as our own user.

Using the file_line resource type, we can ensure that a line exists or is absent in a config file. Using file_line we can quickly make edits to files without controlling the entire file.

We installed the puppetlabs-stdlib module into the default module path for Puppet, so when we ran puppet apply, Puppet knew where to find the file_line type definition. Puppet then created the /tmp/cookbook file if it didn't exist. The line Hello World! was not found in the file, so Puppet added the line to the file.

We can define more instances of file_line and add more lines to the file; we can have multiple resources modifying a single file.

Modify the oneline.pp file and add another file_line resource:

  file {'/tmp/cookbook':
    ensure => 'file',
  }
  file_line {'cookbook-hello':
    path    => '/tmp/cookbook',
    line    => 'Hello World!',
    require => File['/tmp/cookbook'],
  }
  file_line {'cookbook-goodbye':
    path    => '/tmp/cookbook',
    line    => 'So long, and thanks for all the fish.',
    require => File['/tmp/cookbook'],
  }

Now apply the manifest again and verify whether the new line is appended to the file:

t@mylaptop ~/.puppet/manifests $ puppet apply oneline.pp 
Notice: Compiled catalog for mylaptop in environment production in 0.36 seconds
Notice: /Stage[main]/Main/File_line[cookbook-goodbye]/ensure: created
Notice: Finished catalog run in 0.02 seconds
t@mylaptop ~/.puppet/manifests $ cat /tmp/cookbook 
Hello World!
So long, and thanks for all the fish.

The file_line type also supports pattern matching and line removal as we'll show you in the following example:

  file {'/tmp/cookbook':
    ensure => 'file',
  }
  file_line {'cookbook-remove':
    ensure  => 'absent',
    path    => '/tmp/cookbook',
    line    => 'Hello World!',
    require => File['/tmp/cookbook'],
  }
  file_line {'cookbook-match':
    path    => '/tmp/cookbook',
    line    => 'Oh freddled gruntbuggly, thanks for all the fish.',
    match   => 'fish.$',
    require => File['/tmp/cookbook'],
  }

Verify the contents of /tmp/cookbook before your Puppet run:

t@mylaptop ~/.puppet/manifests $ cat /tmp/cookbook 
Hello World!
So long, and thanks for all the fish.

Apply the updated manifest:

t@mylaptop ~/.puppet/manifests $ puppet apply oneline.pp 
Notice: Compiled catalog for mylaptop in environment production in 0.30 seconds
Notice: /Stage[main]/Main/File_line[cookbook-match]/ensure: created
Notice: /Stage[main]/Main/File_line[cookbook-remove]/ensure: removed
Notice: Finished catalog run in 0.02 seconds

Verify that the line has been removed and the goodbye line has been replaced:

t@mylaptop ~/.puppet/manifests $ cat /tmp/cookbook 
Oh freddled gruntbuggly, thanks for all the fish.

Editing files with file_line works well if the file is unstructured. Structured files may have similar lines in different sections that have different meanings. In the next section, we'll show you how to deal with one particular type of structured file, a file using INI syntax.

Install the module from the forge as follows:

t@mylaptop ~ $ puppet module install puppetlabs-inifile
Notice: Preparing to install into /home/tuphill/.puppet/modules ...
Notice: Downloading from https://forgeapi.puppetlabs.com ...
Notice: Installing -- do not interrupt ...
/home/tuphill/.puppet/modules
└── puppetlabs-inifile (v1.1.3)

In this example, we will create a /tmp/server.conf file and ensure that the server_true setting is set in that file:

The inifile module defines two types, ini_setting and ini_subsetting. Our manifest defines an ini_setting resource that creates a server = true setting within the main section of the ini file. In our case, the file didn't exist, so Puppet created the file, then created the main section, and finally added the setting to the main section.

Using ini_subsetting, you can have several resources added to a setting. For instance, our server.conf file has a server's line, we could have each node append its own hostname to a server's line. Add the following to the end of the initest.pp file:

  ini_subsetting {'server_name':
    path    => '/tmp/server.conf',
    section => 'main',
    setting => 'server_host',
    subsetting => "$hostname",
  }

Apply the manifest:

t@mylaptop ~/.puppet/manifests $ puppet apply initest.pp 
Notice: Compiled catalog for mylaptop in environment production in 0.34 seconds
Notice: /Stage[main]/Main/Ini_subsetting[server_name]/ensure: created
Notice: Finished catalog run in 0.02 seconds
t@mylaptop ~/.puppet/manifests $ cat /tmp/server.conf 
[main]
server = true
server_host = mylaptop

Now temporarily change your hostname and rerun Puppet:

t@mylaptop ~/.puppet/manifests $ sudo hostname inihost
t@mylaptop ~/.puppet/manifests $ puppet apply initest.pp 
Notice: Compiled catalog for inihost in environment production in 0.43 seconds
Notice: /Stage[main]/Main/Ini_subsetting[server_name]/ensure: created
Notice: Finished catalog run in 0.02 seconds
t@mylaptop ~/.puppet/manifests $ cat /tmp/server.conf 
[main]
server = true
server_host = mylaptop inihost

If your configuration files are not in INI syntax, another tool, Augeas, can be used. In the following section, we will use augeas to modify files.

Follow these steps to create an example augeas resource:

We declare an augeas resource named enable-ip-forwarding:

augeas { 'enable-ip-forwarding':

We specify that we want to make changes in the file /etc/sysctl.conf:

incl => '/etc/sysctl.conf',

Next we specify the lens to use on this file. Augeas uses files called lenses to translate a configuration file into an object representation. Augeas ships with several lenses, they are located in /usr/share/augeas/lenses by default. When specifying the lens in an augeas resource, the name of the lens is capitalized and has the .lns suffix. In this case, we will specify the Sysctl lens as follows:

lens => 'Sysctl.lns',

The changes parameter specifies the changes we want to make. Its value is an array, because we can supply several changes at once. In this example, there is only change, so the value is an array of one element:

changes => ['set net.ipv4.ip_forward 1'],

In general, Augeas changes take the following form:

set <parameter> <value>

In this case, the setting will be translated into a line like this in /etc/sysctl.conf:

net.ipv4.ip_forward=1

I've chosen /etc/sysctl.conf as the example because it can contain a wide variety of kernel settings and you may want to change these settings for all sorts of different purposes and in different Puppet classes. You might want to enable IP forwarding, as in the example, for a router class but you might also want to tune the value of net.core.somaxconn for a load-balancer class.

This means that simply puppetizing the /etc/sysctl.conf file and distributing it as a text file won't work because you might have several different and conflicting versions depending on the setting you want to modify. Augeas is the right solution here because you can define augeas resources in different places, which modify the same file and they won't conflict.

For more information about using Puppet and Augeas, see the page on the Puppet Labs website http://projects.puppetlabs.com/projects/1/wiki/Puppet_Augeas.

Another project that uses Augeas is Augeasproviders. Augeasproviders uses Augeas to define several types. One of these types is sysctl, using this type you can make sysctl changes without knowing how to write the changes in Augeas. More information is available on the forge at https://forge.puppetlabs.com/domcleal/augeasproviders.

Learning how to use Augeas can be a little confusing at first. Augeas provides a command line tool, augtool, which can be used to get acquainted with making changes in Augeas.

Although it's possible to create our own system to build files from pieces, we'll use the puppetlabs supported concat module. We will start by installing the concat module, in a previous example we installed the module to our local machine. In this example, we'll modify the Puppet server configuration and download the module to the Puppet server.

In your Git repository create an environment.conf file with the following contents:

modulepath = public:modules
manifest = manifests/site.pp

Create the public directory and download the module into that directory as follows:

t@mylaptop ~/puppet $ mkdir public && cd public
t@mylaptop ~/puppet/public $ puppet module install puppetlabs-concat --modulepath=.
Notice: Preparing to install into /home/thomas/puppet/public ...
Notice: Downloading from https://forgeapi.puppetlabs.com ...
Notice: Installing -- do not interrupt ...
/home/thomas/puppet/public
└─┬ puppetlabs-concat (v1.1.1)
  └── puppetlabs-stdlib (v4.3.2)

Now add the new modules to our Git repository:

t@mylaptop ~/puppet/public $ git add .
t@mylaptop ~/puppet/public $ git commit -m "adding concat"
[production 50c6fca] adding concat
 407 files changed, 20089 insertions(+)

Then push to our Git server:

t@mylaptop ~/puppet/public $ git push origin production

Now that we have the concat module available on our server, we can create a concat container resource in our base module:

  concat {'hosts.allow':
    path => '/etc/hosts.allow',
    mode => 0644
  }

Create a concat::fragment module for the header of the new file:

  concat::fragment {'hosts.allow header':
    target  => 'hosts.allow',
    content => "# File managed by puppet\n",
    order   => '01'
  }

Create a concat::fragment that includes a local file:

  concat::fragment {'hosts.allow local':
    target => 'hosts.allow',
    source => '/etc/hosts.allow.local',
    order  => '10',
  }

Create a concat::fragment module that will go at the end of the file:

  concat::fragment {'hosts.allow tftp':
    target  => 'hosts.allow',
    content => "in.ftpd: .example.com\n",
    order   => '50',
  }

On the node, create /etc/hosts.allow.local with the following contents:

  in.tftpd: .example.com

Run Puppet to have the file created:

[root@cookbook ~]# puppet agent -t
Info: Caching catalog for cookbook.example.com
Info: Applying configuration version '1412138600'
Notice: /Stage[main]/Base/Concat[hosts.allow]/File[hosts.allow]/ensure: defined content as '{md5}b151c8bbc32c505f1c4a98b487f7d249'
Notice: Finished catalog run in 0.29 seconds

Verify the contents of the new file as: