Ansible: Up and Running, 3rd Edition

Chapter 1. Introduction

It’s an interesting time to be working in the IT industry. We no longer deliver software to our customers by installing a program on a single machine and calling it a day. Instead, we are all gradually turning into cloud engineers.

We now deploy software applications by stringing together services that run on a distributed set of computing resources and communicate over different networking protocols. A typical application can include web servers, application servers, memory-based caching systems, task queues, message queues, SQL databases, NoSQL datastores, and load balancers.

IT professionals also need to make sure to have the proper redundancies in place, so that when failures happen (and they will), our software systems will handle them gracefully. Then there are the secondary services that we also need to deploy and maintain, such as logging, monitoring, and analytics, as well as third-party services we need to interact with, such as infrastructure-as-a-service (IaaS) endpoints for managing virtual machine instances.¹

You can wire up these services by hand: spinning up the servers you need, logging into each one, installing packages, editing config files, and so forth, but it’s a pain. It’s time-consuming, error-prone, and just plain dull to do this kind of work manually, especially around the third or fourth time. And for more complex tasks, like standing up an OpenStack cloud, doing it by hand is madness. There must a better way.

If you’re reading this, you’re probably already sold on the idea of configuration management and considering adopting Ansible as your configuration management tool. Whether you’re a developer deploying your code to production, or you’re a systems administrator looking for a better way to automate, I think you’ll find Ansible to be an excellent solution to your problem.

A Note About Versions

The example code in this book was tested against several versions of Ansible. Ansible 5.9.0 is the latest version as of this writing; Ansible Tower includes version 2.9.27 in the most recent release. Ansible 2.8 went End of Life with the release of 2.8.20 on April 13, 2021. Expect Ansible to evolve further.

For years the Ansible community has been highly active in creating roles and modules—so active that there are thousands of modules and more than 20,000 roles. The difficulties of managing a project of this scale led creators to reorganize the Ansible content into three parts:

• Core components, created by the Ansible team

• Certified content, created by Red Hat’s business partners

• Community content, created by thousands of enthusiasts worldwide

Ansible 2.9 has lots of built-in features, and later versions are more composable. This new setup makes it more easily maintainable as a whole.

The examples provided in this book should work in various versions of Ansible, but version changes in general call for testing, which we will address in Chapter 14.

What’s with the Name Ansible?

It’s a science-fiction reference. An ansible is a fictional communication device that can transfer information faster than the speed of light. Ursula K. Le Guin invented the concept in her book Rocannon’s World (Ace Books, 1966), and other sci-fi authors have since borrowed the idea, including Orson Scott Card. Ansible cofounder Michael DeHaan took the name Ansible from Card’s book Ender’s Game (Tor, 1985). In that book, the ansible was used to control many remote ships at once, over vast distances. Think of it as a metaphor for controlling remote servers.

Ansible: What Is It Good For?

Ansible is often described as a configuration management tool and is typically mentioned in the same breath as Puppet, Chef, and Salt. When IT professionals talk about configuration management, we typically mean writing some kind of state description for our servers, then using a tool to enforce that the servers are, indeed, in that state: the right packages are installed, configuration files have the expected values and have the expected permissions, the right services are running, and so on. Like other configuration management tools, Ansible exposes a domain-specific language (DSL) that you use to describe the state of your servers.

You can use these tools for software deployment as well. When people talk about deployment, they are usually referring to the process of generating binaries or static assets (if necessary) from software written by in-house developers, copying the required files to servers, adding configuration properties and environment variables, and starting services in a particular order. Capistrano and Fabric are two examples of open source deployment tools. Ansible is a great tool for deployment as well as configuration management. Using a single tool for both makes life simpler for the folks responsible for system integration.

Some people talk about the need to orchestrate deployments. Orchestration is the process of coordinating deployment when multiple remote servers are involved and things must happen in a specific order. For example, you might need to bring up the database before bringing up the web servers, or take web servers out of the load balancer one at a time to upgrade them without downtime. DeHaan designed Ansible from the ground up to be good at this, and to perform actions on multiple servers. It has a refreshingly simple model for controlling the order in which actions happen.

Finally, you’ll hear people talk about provisioning new servers. In the context of public clouds such as Amazon EC2, provisioning refers to spinning up new virtual machine instances or cloud-native software as a service (SaaS). Ansible has got you covered here, with modules for talking to clouds including EC2, Azure,² Digital Ocean, Google Compute Engine, Linode, and Rackspace,³ as well as any clouds that support the OpenStack APIs.

Note

Confusingly, the Vagrant tool, covered later in this chapter, uses the term provisioner to refer to a tool that does configuration management. It thus refers to Ansible as a kind of provisioner. Vagrant calls tools that create machines, such as VirtualBox and VMWare, providers. Vagrant uses the term machine to refer to a virtual machine and box to refer to a virtual machine image.

How Ansible Works

Figure 1-1 shows a sample use case of Ansible in action. A user we’ll call Alice is using Ansible to configure three Ubuntu-based web servers to run NGINX. She has written a script called webservers.yml. In Ansible, the equivalent of a script is called a playbook. A playbook describes which hosts (what Ansible calls remote servers) to configure, and an ordered list of tasks to perform on those hosts. In this example, the hosts are web1, web2, and web3, and the tasks are things such as these:

• Install NGINX

• Generate a NGINX configuration file

• Copy over the security certificate

• Start the NGINX service

Figure 1-1. Running an Ansible playbook to configure three web servers

In the next chapter, we’ll elaborate what’s in this playbook; for now, we’ll focus on its role in the overall process. Alice executes the playbook by using the ansible-playbook command. Alice starts her Ansible playbook by typing first the command and then the name of the playbook on a terminal line:

$ ansible-playbook webservers.yml

Ansible will make SSH connections in parallel to web1, web2, and web3. It will then execute the first task on the list on all three hosts simultaneously. In this example, the first task is installing the NGINX package, so the task in the playbook would look something like this:

- name: Install nginx
  package:
    name: nginx

Ansible will do the following:

1. Generate a Python script that installs the NGINX package

2. Copy the script to web1, web2, and web3

3. Execute the script on web1, web2, and web3

4. Wait for the script to complete execution on all hosts

Ansible will then move to the next task in the list and go through these same four steps.

It’s important to note the following:

1. Ansible runs each task in parallel across all hosts.

2. Ansible waits until all hosts have completed a task before moving to the next task.

3. Ansible runs the tasks in the order that you specify them.

What’s So Great About Ansible?

There are several open source configuration management tools out there to choose from, so why choose Ansible? Here are 21 reasons that drew us to it. In short: Ansible is simple, powerful, and secure.

Simple

Ansible was designed to have a dead simple setup process and a minimal learning curve. 

Easy-to-read syntax

Ansible uses the YAML file format and Jinja2 templating, both of which are easy to pick up. Recall that Ansible configuration management scripts are called playbooks. Ansible actually builds the playbook syntax on top of YAML, which is a data format language that was designed to be easy for humans to read and write. In a way, YAML is to JSON what Markdown is to HTML.

Easy to audit

You can inspect Ansible playbooks in several ways, like listing all actions and hosts involved. For dry runs, we often use ansible-playbook --check. With built-in logging it is easy to see who did what and where. The logging is pluggable and log collectors can easily ingest the logs.

Little to nothing to install on the remote hosts

To manage servers with Ansible, Linux servers need to have SSH and Python installed, while Windows servers need WinRM enabled. On Windows, Ansible uses PowerShell instead of Python, so there is no need to preinstall an agent or any other software on the host.

On the control machine (that is, the machine that you use to control remote machines), it is best to install Python 3.8 or later. Depending on the resources you manage with Ansible, you might have external library prerequisites. Check the documentation to see whether a module has specific requirements.

Ansible scales down

The authors of this book use Ansible to manage hundreds of nodes. But what got us hooked is how it scales down. You can use Ansible on very modest hardware, like a Raspberry Pi or an old PC. Using it to configure a single node is easy: simply write a single playbook. Ansible obeys Alan Kay’s maxim: “Simple things should be simple; complex things should be possible.”

Easy to share

We do not expect you to reuse Ansible playbooks across different contexts. In Chapter 7, we will discuss roles, which are a way of organizing your playbooks, and Ansible Galaxy, an online repository of these roles.

The primary unit of reuse in the Ansible community nowadays is the collection. You can organize your modules, plug-ins, libraries, roles, and even playbooks into a collection and share it on Ansible Galaxy. You can also share internally using Automation Hub, a part of Ansible Tower. Roles can be shared as individual repositories.

In practice, though, every organization sets up its servers a little bit differently, and you are best off writing playbooks for your organization rather than trying to reuse generic ones. We believe the primary value of looking at other people’s playbooks is to see how things work, unless you work with a particular product for which the vendor is a certified partner or involved in the Ansible community.

System abstraction

Ansible works with simple abstractions of system resources like files, directories, users, groups, services, packages, and web services.

By way of comparison, let’s look at how to configure a directory in the shell. You would use these three commands:

mkdir -p /etc/skel/.ssh
chown root:root /etc/skel/.ssh
chmod go-wrx /etc/skel/.ssh

By contrast, Ansible offers the file module as an abstraction, where you define the parameters of the desired state. This one action has the same effect as the three shell commands combined:

- name: Ensure .ssh directory in user skeleton
  file:
    path: /etc/skel/.ssh
    mode: '0700'
    owner: root
    group: root
    state: directory

With this layer of abstraction, you can use the same configuration management scripts to manage servers running Linux distributions. For example, instead of having to deal with a specific package manager like dnf, yum, or apt, Ansible has a “package” abstraction that you can use instead (just be aware that package names might differ). But you can also use the system-specific abstractions if you prefer.

If you really want to, you can write your Ansible playbooks to take different actions, depending on the variety of operating systems of the remote servers. But Bas, one of the authors of this book, tries to avoid that where he can; he instead focuses on writing playbooks for the systems that are in actual use.

Top to bottom tasks

Books on configuration management often mention the concept of convergence, or eventual consistent state. Convergence in configuration management is strongly associated with the configuration management system CFEngine by Mark Burgess. If a configuration management system is convergent, the system may run multiple times to put a server into its desired state, with each run bringing the server closer to that state.

Eventual consistent state does not really apply to Ansible, since it does not run multiple times to configure servers. Instead, Ansible modules work in such a way that running a playbook a single time should put each server into the desired state.

Powerful

Having Ansible at your disposal can bring huge productivity gains in several areas of systems management. The high-level abstractions Ansible provides (like roles) make it so that you can set up and configure things faster and potentially more securely.

Batteries included

You can use Ansible to execute arbitrary shell commands on your remote servers, but its real power comes from the wide variety of modules available. You use modules to perform tasks such as installing a package, restarting a service, or copying a configuration file.

As you will see later, Ansible modules are declarative; you use them to describe the state you want the server to be in. For example, you would invoke the user module like this to ensure there is an account named “deploy” in the web group:

- name: Ensure deploy user exists
  user:
    name: deploy
    group: web

Push-based

Chef and Puppet are configuration management systems that use agents. They are pull-based by default. Agents installed on the servers periodically check in with a central service and download configuration information from the service. Making configuration management changes to servers goes something like this:

1. You: make a change to a configuration management script.

2. You: push the change up to a configuration management central service.

3. Agent on server: wakes up after periodic timer fires.

4. Agent on server: connects to configuration management central service.

5. Agent on server: downloads new configuration management scripts.

6. Agent on server: executes configuration management scripts locally that change server state.

In contrast, Ansible is push-based by default. Making a change looks like this:

1. You: make a change to a playbook.

2. You: run the new playbook.

3. Ansible: connects to servers and executes modules that change the state of the servers.

As soon as you run the ansible-playbook command, Ansible connects to the remote servers and does its thing; this lowers the risk of random servers potentially breaking whenever their scheduled tasks fail to change things successfully. The push-based approach has a significant advantage: you control when the changes happen to the servers. You do not need to wait around for a timer to expire. Each step in a playbook can target one or a group of servers. You get more work done instead of logging into the servers by hand.

Multitier orchestration

Push mode also allows you to use Ansible for multitier orchestration, managing distinct groups of machines for an operation like an update. You can orchestrate the monitoring system, the load balancers, the databases, and the web servers with specific instructions so they work in concert. That’s very hard to do with a pull-based system.

Masterless

Advocates of the pull-based approach claim that it is superior for scaling to large numbers of servers and for dealing with new servers that can come online anytime. A central configuration management system, however, slowly stops working when thousands of agents pull their configuration at the same time, especially when they need multiple runs to converge. In comparison, Ansible comes with the ansible-pull command, which can pull playbooks from a VCS repository like GitHub. Ansible does not need a master, but you can use a central system to run playbooks if you want to.

Pluggable and embeddable

A sizable part of Ansible’s functionality comes from the Ansible Plugin System, of which the Lookup and Filter plug-ins are most used. Plug-ins augment Ansible’s core functionality with logic and features that are accessible to all modules. Modules introduce new “verbs” to the Ansible language. You can write your own plug-ins (see Chapter 10) and modules (Chapter 12) in Python.

You can integrate Ansible into other products: Kubernetes and Ansible Tower are examples of successful integrations. Ansible Runner “is a tool and python library that helps when interfacing with Ansible directly or as part of another system whether that be through a container image interface, as a standalone tool, or as a Python module that can be imported.”⁴

Using the ansible-runner library, you can run an Ansible playbook from within a Python script:

#!/usr/bin/env python3
import ansible_runner

r = ansible_runner.run(private_data_dir='./playbooks', playbook='playbook.yml')

print("{}: {}".format(r.status, r.rc))
print("Final status:")
print(r.stats)

Works with lots of stuff

Ansible modules cater to a wide range of system administration tasks. This list has the categories of the kinds of modules that you can use. These link to the module index in the documentation:

• Cloud
• Files
• Monitoring
• Source Control
• Clustering
• Identity
• Net Tools
• Storage
• Commands
• Infrastructure
• Network
• System
• Crypto
• Inventory
• Notification
• Utilities
• Database
• Messaging
• Packaging
• Windows

Really scalable

Large enterprises use Ansible successfully in production with tens of thousands of nodes and have excellent support for environments where servers are dynamically added and removed. Organizations with hundreds of software teams typically use AWX or a combination of Ansible Tower and Automation Hub for auditability, and for security with role-based access controls.

Worried about the scalability of SSH? Ansible uses SSH multiplexing to optimize performance, and there are folks out there who are managing thousands of nodes with Ansible (see Chapter 12 of this book).

Secure

Automation with Ansible helps us to improve system security to security baselines and compliance standards. 

Codified knowledge

Your authors like to think of Ansible playbooks as executable documentation. Playbooks are like the README files that used to describe the commands you had to type out to deploy your software, except that these instructions will never go out of date because they are also the code that executes. Product experts can create playbooks that take best practices into account. When novices use such a playbook to install the product, they can be sure they’ll get a good result.

Reproducible systems

If you set up your entire system with Ansible, it will pass what Steve Traugott calls the “tenth-floor test”: “Can I grab a random machine that’s never been backed up and throw it out the tenth-floor window without losing sysadmin work?”

Equivalent environments

Ansible has a clever way to organize content that helps define configuration at the proper level. It is easy to create a setup for distinct development, testing, staging, and production environments. A staging environment is designed to be as similar as possible to the production environment so that developers can detect any problems before changes go live. 

Encrypted variables

If you need to store sensitive data such as passwords or tokens, then ansible-vault is an effective tool to use. We use it to store encrypted variables in Git. We’ll discuss it in detail in Chapter 8.

Secure transport

Ansible simply uses Secure Shell (SSH) for Linux and WinRM for Windows. We typically secure and harden these widely used systems-management protocols with strong configuration and firewall settings.

If you prefer using a pull-based model, Ansible has official support for pull mode, using a tool it ships with called ansible-pull. This book won’t cover pull mode, but you can read more about it in the official Ansible documentation.

Idempotency

Modules are also idempotent. Idempotence is a nice property because it means that it is safe to run an Ansible playbook multiple times against a server. Let’s see what that means when we need a user named deploy:

- name: Ensure deploy user exists
  user:
    name: deploy
    group: web

If the deploy user does not exist, Ansible will create it. If it does exist, Ansible will not do anything. This is a vast improvement over the homegrown shell script approach, where running the shell script a second time might have a different (and unintended) effect.⁵

No daemons

There is no Ansible agent listening on a port. Therefore, when you use Ansible, there is no extra attack surface. (There is still an attack surface with software supply chain elements like Python libraries and other imported content.)

What Is Ansible, Inc.’s Relationship to Ansible?

The name Ansible refers to both the software and the company that runs the open source project. Michael DeHaan, the creator of Ansible the software, is the former CTO of Ansible the company. To avoid confusion, we refer to the software as Ansible and to the company as Ansible, Inc.

Ansible, Inc. sells training and consulting services for Ansible, as well as a web-based management tool called Ansible Tower, which we cover in Chapter 19. In October 2015, Red Hat bought Ansible, Inc.; IBM bought Red Hat in 2019.

Is Ansible Too Simple?

When Lorin was working on an earlier edition of this book, the editor mentioned that “some folks who use the XYZ configuration management tool call Ansible a for-loop over SSH scripts.” If you are considering switching over from another configuration management tool, you might be concerned at this point about whether Ansible is powerful enough to meet your needs. 

As you will soon learn, Ansible supplies a lot more functionality than shell scripts. In addition to idempotence, Ansible has excellent support for templating, as well as defining variables at different scopes. Anybody who thinks Ansible is equivalent to working with shell scripts has never had to support a nontrivial program written in shell. We will always choose Ansible over shell scripts for configuration management tasks if given a choice.

What Do I Need to Know?

To be productive with Ansible, you need to be familiar with basic Unix/Linux system administration tasks. Ansible makes it easy to automate your tasks, but it is not the kind of tool that “automagically” does things that you otherwise would not know how to do.

For this book, we have assumed that you are familiar with at least one Linux distribution (such as Ubuntu, RHEL/CentOS, or SUSE), and that you know how to:

• Connect to a remote machine using SSH

• Interact with the Bash command-line shell (pipes and redirection)

• Install packages

• Use the sudo command

• Check and set file permissions

• Start and stop services

• Set environment variables

• Write scripts (any language)

If these concepts are all familiar to you, you are good to go with Ansible.

We will not assume you have knowledge of any particular programming language. For instance, you do not need to know Python to use Ansible unless you want to publish your own module.

What Isn’t Covered

This book is not an exhaustive treatment of Ansible. The first part is designed to get you working productively in Ansible as quickly as possible. Then it describes how to perform certain tasks that are not obvious from the official documentation.

We don’t cover all of Ansible’s modules in detail: there are more than 3,500 of them. You can use the ansible-doc command-line tool with what you have installed to view the reference documentation and the module index mentioned previously.

Chapter 8 covers only the basic features of Jinja2, the template engine that Ansible uses, primarily because your authors memorize only basic features when we use Jinja2 with Ansible. If you need to use more advanced Jinja2 features in templates, check out the official Jinja2 documentation.

Nor do we go into detail about some features of Ansible that are mainly useful when you are running it on an older version of Linux.

Finally, there are features of Ansible we don’t cover simply to keep the book a manageable length. We encourage you to check out the official documentation to find out more about these features.

Moving Forward

This introductory chapter covered the basic concepts of Ansible at a general level, including how it communicates with remote servers and how it differs from other configuration management tools. The next chapters discuss how to practice using Ansible.

¹ For more on building and maintaining these types of distributed systems, check out Thomas A. Limoncelli, Strata R. Chalup, and Christina J. Hogan’s The Practice of Cloud System Administration, volumes 1 and 2 (Addison-Wesley), and Designing Data-Intensive Applications by Martin Kleppman (O’Reilly).

² Yes, Azure supports Linux servers.

³ For example, see “Using Ansible at Scale to Manage a Public Cloud”, a slide presentation by Jesse Keating, formerly of Rackspace.

⁴ Ansible Runner documentation, last accessed June 2, 2022.

⁵ If you are interested in what Ansible’s original author thinks of the idea of convergence, see Michael DeHaan’s Ansible Project newsgroup post “Idempotence, Convergence, and Other Silly Fancy Words We Use Too Often”.