Chapter 4. Writing Manifests

The very first concept we want to introduce you to is the Puppet manifest. A manifest is a file containing Puppet configuration language that describes how resources should be configured. The manifest is the closest thing to what one might consider a Puppet program. It declares resources that define state to be enforced on a node. It is therefore the base component for Puppet configuration policy, and a building block for complex Puppet modules.

This chapter will focus on how to write configuration policies for Puppet 4 manifests. Writing manifests well is the single most important part of building Puppet policies.

Let’s get started with the smallest component within a manifest.

Implementing Resources

Resources are the smallest building block of the Puppet configuration language. They represent a singular element that you wish to evaluate, create, or remove. Puppet comes with many built-in resource types. The stock resource types manipulate system components that you are already familiar with, including:

  • Users
  • Groups
  • Files
  • Host file entries
  • Packages
  • Services

Furthermore, you can create your own resources. However, we’ll begin with one of the simplest resources—the notify resource. Let’s start with the standard first program written in every language:

notify { 'greeting':
  message => 'Hello, world!'
}

This code declares a notify resource with a title of greeting. It has a single attribute, message, which is assigned the value we’d expect in our first program. Attributes are separated from their values by a hash rocket (also called a fat comma), which is a very common way to identify key/value pairs in Perl, Ruby, and PHP scripting languages.

This tiny bit of code is a fully functional and valid manifest. This manifest (with one single resource) does only one thing, which is to output that greeting message every time it is called. Let’s go ahead and use Puppet to evaluate this manifest:

[vagrant@client ~]$ cat /vagrant/manifests/helloworld.pp

notify { 'greeting':
  message => 'Hello, world!'
}
Note
As part of the definition of your virtual system, we have preinstalled some Puppet manifests in the /vagrant/manifests/ directory for use in this class. We’ll refer to these throughout the book.

As you can see from this example, manifests are text files named with a .pp file extension—they describe resources using the Puppet configuration language. You can create or modify a Puppet manifest using any text editor.

Applying a Manifest

One of Puppet’s best features is the ease of testing your code. Puppet does not require you to set up complicated testing environments to evaluate Puppet manifests. It is easy—nay, downright trivial—to test a Puppet manifest.

Let’s go ahead and apply this manifest. We will do this using the puppet apply command, which tells Puppet to apply a single Puppet manifest:

[vagrant@client ~]$ puppet apply /vagrant/manifests/helloworld.pp
Notice: Compiled catalog for client.example.com in environment production
Notice: Hello, world!
Notice: /Stage[main]/Main/Notify[greeting]/message:
  defined 'message' as 'Hello, world!'
Notice: Finished catalog run in 0.01 seconds

As you can see, Puppet has applied the manifest. It does this in several steps:

  1. Builds (compiles) a Puppet catalog from the manifest.
  2. Uses dependency and ordering information to determine evaluation order.
  3. Evaluates the target resource to determine if changes should be applied.
  4. Creates, modifies, or removes the resource—a notification message is created.
  5. Provides verbose feedback about the catalog application.

Don’t worry about memorizing these steps at this point in the learning process. For now, it’s just important that you have a general idea of the process—we’ll discuss these concepts in increasing depth throughout this book. We’ll cover the catalog build and evaluation process in great detail at the end of Part I.

For now, just remember that you’ll use puppet apply to apply Puppet manifests. It will provide you verbose feedback on actions Puppet took to bring the target resources into alignment with the declared policy.

Declaring Resources

There are only a few rules to remember when declaring resources. The format is always the same:

resource_type { 'resource_title':
  ensure     => present,         # usually 'present' or 'absent'
  attribute1 => 1234,            # numbers are unquoted
  attribute2 => 'value',         # strings should be quoted
  attribute3 => ['red','blue'],  # arrays contain other data types
  noop       => false,           # boolean is unquoted true/false
}
Tip
Don’t get hung up analyzing the data types shown in this example. We will cover the different data types of Puppet 4 exhaustively in Chapter 5.

The most important rule for resources is: there can be only one. Within a manifest or set of manifests being applied together (the catalog for a node), a resource of a given type can only be declared once with a given title. Every resource of that type must have a unique title.

For example, the following manifest will fail because the same title is used for both file resources:

[vagrant@client ~]$ cat myfile.pp
file { 'my_file':
  ensure => present,
  path   => 'my_file.txt',
}

file { 'my_file':
  ensure => present,
  path   => 'my_file.csv',
}

notify { 'my_file':
  message => "My file is present",
}

[vagrant@client ~]$ puppet apply myfile.pp
Error: Evaluation Error: Error while evaluating a Resource Statement,
  Duplicate declaration: File[my_file] is already declared in file
  /home/vagrant/myfile.pp:1; cannot redeclare at /home/vagrant/myfile.pp:6

You’ll notice that no complaint was given for the notify resource with the same title. This is not a conflict. Only resources of the same type cannot utilize the same title. Naming the preceding files with their full paths ensures no conflicts:

file { '/home/vagrant/my_file.txt':
  ensure => present,
  path   => '/home/vagrant/my_file.txt',
}

file { '/home/vagrant/my_file.csv':
  ensure => present,
  path   => '/home/vagrant/my_file.csv',
}

Viewing Resources

Puppet can show you an existing resource written out in the Puppet configuration language. This makes it easy to generate code based on existing configurations. Let’s demonstrate how to view a resource with an email alias:

[vagrant@client ~]$ puppet resource mailalias postmaster
mailalias { 'postmaster':
  ensure    => 'present',
  recipient => ['root'],
  target    => '/etc/aliases',
}

The puppet resource command queries the resource and shows its current state as Puppet sees it. The output gives you the exact structure, syntax, and attributes to declare this mailalias resource in a Puppet manifest. You can add this to a manifest file, change the recipient, and then use puppet apply to change the postmaster alias on this node.

Let’s examine another resource—the user you are logged in as:

[vagrant@client ~]$ puppet resource user vagrant
Error: Could not run: undefined method 'exists?' for nil:NilClass

This somewhat confusing error message means that you don’t have the privileges to view that resource—so let’s escalate our privileges to complete this command with sudo:

[vagrant@client ~]$ sudo puppet resource user vagrant
user { 'vagrant':
  ensure           => 'present',
  gid              => '1000',
  groups           => ['vagrant'],
  home             => '/home/vagrant',
  password         => '$1$sC3NqLSG$FsXVyW7azpoh76edOfAWm1',
  password_max_age => '99999',
  password_min_age => '0',
  shell            => '/bin/bash',
  uid              => '1000',
}

If you look at the resource, you’ll see why root access was necessary. The user resource contains the user’s password hash, which required root privilege to read from the shadow file. As with the previous mailalias resource, you could write this user resource to a manifest file, replace the password hash, and use sudo puppet apply to change the root password.

Warning

Some resources attributes output by puppet resource are read-only attributes. You’ll receive a clear is read-only error message if you attempt to declare them. Refer to the resource’s documentation to learn which attributes are read-only.

Executing Programs

Let’s examine a resource type that executes commands. Use the exec resource to execute programs as part of your manifest:

exec { 'echo-holy-cow':
  path      => ['/bin'],
  cwd       => '/tmp',
  command   => 'echo "holy cow!" > testfile.txt',
  creates   => '/tmp/testfile.txt',
  returns   => [0],
  logoutput => on_failure,
}

Now when you apply this manifest, it will create the testfile.txt file. Notice that we use single quotes to encapsulate the values given to the attributes.

Best Practice

Use single quotes for any value that does not contain a variable. This protects against accidental interpolation of a variable. Use double quotes with strings containing variables.

The exec resource declared above uses the creates attribute. This attribute defines a file that will be created by the command execution. When the named file exists, the command is not executed. This means the manifest can be run repeatedly and nothing will change after the file is initially created. Let’s test this out here:

[vagrant@client ~]$ puppet apply /vagrant/manifests/tmp-testfile.pp
Notice: Compiled catalog for client.example.com in environment production
Notice: /Stage[main]/Main/Exec[echo-holy-cow]/returns: executed successfully
Notice: Finished catalog run in 0.07 seconds

[vagrant@client ~]$ puppet apply /vagrant/manifests/tmp-testfile.pp
Notice: Compiled catalog for client.example.com in environment production
Notice: Finished catalog run in 0.01 seconds

There are a wide variety of attributes you can use to control whether or not an exec resource will be executed, and which exit codes indicate success or failure. This is a complex and feature-rich resource. Whenever you create an exec resource, test carefully and refer to “Puppet Type Reference: exec” on the Puppet docs site.

Was That Idempotent?

I must beg your forgiveness: I have deliberately led you astray to teach you a common mistake for newcomers to declarative programming. While exec is an essential resource type, it is best to avoid using it whenever possible. We’ll apply the exact same result with a more appropriate file resource next.

If you examine the preceding exec resource, you’ll note that we had to declare how to make the change, and also whether or not to make the change. This is very similar to imperative programming, and can be very difficult to maintain.

This resource protects itself against running again with the creates attribute. However, if the contents of the file were changed, this resource would not repair the contents. We would need to add more tests to validate the file contents.

It is generally difficult to write declarative code using an exec resource. There is a tendency to revert to an imperative programming style. Except in circumstances where no other method is possible, use of an exec is generally an indication of a poorly written manifest. In fact, at several companies where I have worked, the presence of an exec resource within a commit caused it to be flagged for a mandatory code review.

Best Practice

Use an idempotent Puppet resource type to apply the change.

Managing Files

How else could we create this file? We could have used the file resource. Let’s examine one now:

file { '/tmp/testfile.txt':
  ensure  => present,
  mode    => '0644',
  replace => true,
  content => 'holy cow!',
}

This is a proper declarative policy. We declare that the file should exist, and what the contents of the file should be. We do not need to concern ourselves with how, or when to make changes to the file. Furthermore, we were able to ensure the contents of the file remained consistent, which is not possible within an echo command.

Tip
Notice that the declarative manifest used fewer lines of text, and guaranteed a more consistent output.

Let’s apply this policy now:

[vagrant@client ~]$ puppet apply /vagrant/manifests/file-testfile.pp
Notice: Compiled catalog for client.example.com in environment production
Notice: /Stage[main]/Main/File[/tmp/testfile.txt]/content: content changed
  '{md5}0eb429526e5e170cd9ed4f84c24e4' to '{md5}3d508c8566858d8a168a290dd709c'
Notice: /Stage[main]/Main/File[/tmp/testfile.txt]/mode:
  mode changed '0664' to '0644'
Notice: Finished catalog run in 0.03 seconds

[vagrant@client ~]$ puppet apply /vagrant/manifests/file-testfile.pp
Notice: Compiled catalog for client.example.com in environment production
Notice: Finished catalog run in 0.02 seconds

Unlike with the previous exec resource, Puppet observed that the contents were different and changed the file to match. Now, you’re probably thinking to yourself, “Aren’t the file contents the same in both?” Nope. It’s not obvious in the exec declaration, but echo appends a trailing newline to the text. As you can see here, the file contents don’t include a newline:

[vagrant@client ~]$ cat /tmp/testfile.txt
holy cow![vagrant@client ~]$

You can easily adjust the file contents to include as many newline characters as you want. Because the newline character is interpolated, you’ll need to use double quotes around the contents.

You could also change the replace attribute from true to false if you want to initially create a missing file, but not replace one that has been changed.

file { '/tmp/testfile.txt':
  ensure  => present,
  mode    => '0644',
  replace => false,
  content => "holy cow!\n",
}

You can find complete details of the many attributes available for the file resource at “Puppet Type Resource: file” on the Puppet docs site.

Finding File Backups

Every file changed by a Puppet file resource is backed up on the node in a directory specified by the $clientbucketdir configuration setting. Unfortunately, the file backups are stored in directories indexed by a hash of the file contents, which makes them quite tricky to find.

You can back up a file to this storage any time you want, like so:

$ sudo puppet filebucket --local backup /tmp/testfile.txt
/tmp/testfile.txt: 3d508c856685853ed8a168a290dd709c

Use this command to get a list of every file backed up. Here you can see the backup performed by puppet apply when it changed the file, as well as your manual backup a few minutes later:

$ sudo puppet filebucket --local list
0eb429526e5e170cd9ed4f84c24e442b 2015-11-19 08:18:06 /tmp/testfile.txt
3d508c856685853ed8a168a290dd709c 2015-11-19 08:23:42 /tmp/testfile.txt

Restoring Files

Use the hash associated with a specific file version to view the contents of the file at that point in time:

$ sudo puppet filebucket --local get 0eb429526e5e170cd9ed4f84c24e442b
holy cow!

You can compare an installed file to a backup version, or compare two backup versions, using the filebucket diff command:

$ sudo puppet filebucket --local diff \
    0eb429526e5e170cd9ed4f84c24e442b /tmp/testfile.txt
--- /tmp/diff20151119-4940-1h3dwn9	2015-11-19 08:20:59.994974403 +0000
+++ /tmp/testfile.txt	2015-11-19 08:18:06.162104809 +0000
@@ -1 +1 @@
-holy cow!
+holy cow!
\ No newline at end of file

You can restore a stored file to any location:

$ sudo puppet filebucket --local restore \
    /tmp/testfile.old 0eb429526e5e170cd9ed4f84c24e442b

This command gives no output unless there is an error, so you should find the file has been restored as requested.

Avoiding Imperative Manifests

In the previous section, we established that it is best practice to avoid using exec resources. To understand the reason for this, let’s return to our previous discussion of the word declarative.

If the code tells the interpreter what to do, when to do it, and how to do it, then it is functioning in an imperative manner. If the code tells the interpreter what it wants the result to be, then the code is declarative.

Tip
It is common to see newcomers write imperative Puppet manifests. They struggle to instruct Puppet exactly how to do something. It can take months before they grasp the simplicity of declaring final state. Learn this properly now, and get a head start on becoming a Puppet expert.

Yes, the exec resource will let you write imperative manifests in Puppet. However, as we discussed in Chapter 1, you’ll find that it takes more effort, is harder to maintain, and is less likely to produce consistent results.

Rob Nelson provided us with the perfect metaphor:1

If you try hard enough, you can make it act like an imperative language. That would be akin to ripping the bottom out of your car, dressing in a leopard-print toga and a ratty blue tie, and driving to work with your feet. Is that something you really want to do, or just watch on Boomerang?

The reason I advocate so strongly to not use exec is fairly simple. The exec resource executes the command, but it doesn’t really know what the command does. That command could modify the node state in any form, and Puppet wouldn’t be aware of it. This creates a fire and forget situation, where the only piece of information returned to Puppet is the exit code from the command:

Notice: /Stage[main]/Main/Exec[echo-holy-cow]/returns: executed successfully

All you know is that the command here returned an exit code you expected (0 by default, which is the Unix convention for success). You don’t really know what the command did. If someone wrote a Puppet manifest last week, and then removed it from the execution this week, you have to compare backups (if you have any) to determine what the command did.

When you declare resources that specify desired state, Puppet logs both the previous and the changed values. Files that are changed are backed up, and can be restored if necessary. For example, let’s examine what happened when we applied the file resource:

Notice: /Stage[main]/Main/File[/tmp/testfile.txt]/content: content changed
  '{md5}0eb429526e5e170cd9ed4f84c24e44' to '{md5}3d508c856685853ed8a168a290dd70'
Notice: /Stage[main]/Main/File[/tmp/testfile.txt]/mode:
  mode changed '0664' to '0644'

Without having access to read the original Puppet manifest, you know that the file mode and the file contents were changed. You can validate the file contents later to match the md5 hash, or restore a copy of the file as it was before this change was made.

Using the file resource is more concise, easier to read, and more consistent in application. Best of all, it creates a detailed log of which changes were made. I have found this is true of every situation where you can replace an exec resource with a native resource. Use an exec resource only when no other choice is available.

Testing Yourself

Let’s pause for a second and utilize what you have learned to build a Puppet manifest:

  1. Use puppet resource to create a new manifest from the file /tmp/testfile.txt.
  2. Remove the type attribute, which is redundant with the resource name.
  3. Change the file mode to be group writable (either 0664 or ug=rw,o=r).
  4. Change the content to say something that amuses you.
  5. Run puppet apply yourmanifest.pp and observe the changes.
  6. Confirm the changes with the following commands:
    • ls -la /tmp
    • cat /tmp/testfile.txt
  7. Run puppet apply again and see what happens.

You won’t need to use sudo to make any of these changes, as you are the owner of this file.

Reviewing Writing Manifests

In this chapter, you learned about manifests, single files that contain Puppet resources to be evaluated and, if necessary, applied on the node. Manifests are the closest thing that could be called a program in the conventional sense.

You learned how to create and apply resources, the smallest building blocks of a Puppet manifest. You learned the common syntax of a resource, and how to retrieve the details of an existing resource on a system in the Puppet configuration language.

In addition, you learned how to instruct the Puppet agent to output messages during application of the manifest for debugging purposes.

Finally, you learned how to execute programs with an exec resource, as well as why this can be bad design. You learned how a more declarative style requires less code, and works more consistently in all situations.

Get Learning Puppet 4 now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial