Infrastructure as Code, 2nd Edition

Infrastructure as Code, 2nd Edition

by Kief Morris
Released December 2020
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781098114671

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

Six years ago, Infrastructure as Code was a new concept. Today, as even banks and other conservative organizations plan moves to the cloud, development teams for companies worldwide are attempting to build large infrastructure codebases. With this practical book, Kief Morris of ThoughtWorks shows you how to effectively use principles, practices, and patterns pioneered by DevOps teams to manage cloud-age infrastructure.

Ideal for system administrators, infrastructure engineers, software developers, team leads, and architects, this updated edition demonstrates how you can exploit cloud and automation technology to make changes easily, safely, quickly, and responsibly. You'll learn how to define everything as code and apply software design and engineering practices to build your system from small, loosely coupled pieces.

This book covers:

  • Foundations: Use Infrastructure as Code to drive continuous change and raise the bar of operational quality, using tools and technologies to build cloud-based platforms
  • Working with infrastructure stacks: Learn how to define, provision, test, and continuously deliver changes to infrastructure resources
  • Working with servers and other platforms: Use patterns to design provisioning and configuration of servers and clusters
  • Working with large systems and teams: Learn workflows, governance, and architectural patterns to create and manage infrastructure elements

Publisher resources

View/Submit Errata

Table of contents

  1. Preface
    1. Why I Wrote This Book
    2. What’s New and Different in This Edition
    3. What’s Next
    4. What This Book Is and Isn’t
    5. Some History of Infrastructure as Code
    6. Who This Book Is For
    7. Principles, Practices, and Patterns
    8. The ShopSpinner Examples
    9. Conventions Used in This Book
    10. O’Reilly Online Learning
    11. How to Contact Us
    12. Acknowledgments
  2. I. Foundations
  3. 1. What Is Infrastructure as Code?
    1. From the Iron Age to the Cloud Age
    2. Infrastructure as Code
    3. Benefits of Infrastructure as Code
    4. Use Infrastructure as Code to Optimize for Change
      1. Objection: “We don’t make changes often enough to justify automating them”
      2. Objection: “We should build first and automate later”
      3. Objection: “We must choose between speed and quality”
    5. The Four Key Metrics
    6. Three Core Practices for Infrastructure as Code
      1. Core Practice: Define Everything as Code
      2. Core Practice: Continuously Test and Deliver All Work in Progress
      3. Core Practice: Build Small, Simple Pieces That You Can Change Independently
    7. Conclusion
  4. 2. Principles of Cloud Age Infrastructure
    1. Principle: Assume Systems Are Unreliable
    2. Principle: Make Everything Reproducible
    3. Pitfall: Snowflake Systems
    4. Principle: Create Disposable Things
    5. Principle: Minimize Variation
      1. Configuration Drift
    6. Principle: Ensure That You Can Repeat Any Process
    7. Conclusion
  5. 3. Infrastructure Platforms
    1. The Parts of an Infrastructure System
    2. Infrastructure Platforms
    3. Infrastructure Resources
      1. Compute Resources
      2. Storage Resources
      3. Network Resources
    4. Conclusion
  6. 4. Core Practice: Define Everything as Code
    1. Why You Should Define Your Infrastructure as Code
    2. What You Can Define as Code
      1. Choose Tools with Externalized Configuration
      2. Manage Your Code in a Version Control System
    3. Infrastructure Coding Languages
      1. Infrastructure Scripting
      2. Declarative Infrastructure Languages
      3. Programmable, Imperative Infrastructure Languages
      4. Declarative Versus Imperative Languages for Infrastructure
      5. Domain-Specific Infrastructure Languages
      6. General-Purpose Languages Versus DSLs for Infrastructure
    4. Implementation Principles for Defining Infrastructure as Code
      1. Separate Declarative and Imperative Code
      2. Treat Infrastructure Code Like Real Code
    5. Conclusion
  7. II. Working with Infrastructure Stacks
  8. 5. Building Infrastructure Stacks as Code
    1. What Is an Infrastructure Stack?
      1. Stack Code
      2. Stack Instance
      3. Configuring Servers in a Stack
      4. Low-Level Infrastructure Languages
      5. High-Level Infrastructure Languages
    2. Patterns and Antipatterns for Structuring Stacks
      1. Antipattern: Monolithic Stack
      2. Pattern: Application Group Stack
      3. Pattern: Service Stack
      4. Pattern: Micro Stack
    3. Conclusion
  9. 6. Building Environments with Stacks
    1. What Environments Are All About
      1. Delivery Environments
      2. Multiple Production Environments
      3. Environments, Consistency, and Configuration
    2. Patterns for Building Environments
      1. Antipattern: Multiple-Environment Stack
      2. Antipattern: Copy-Paste Environments
      3. Pattern: Reusable Stack
    3. Building Environments with Multiple Stacks
    4. Conclusion
  10. 7. Configuring Stack Instances
    1. Using Stack Parameters to Create Unique Identifiers
    2. Example Stack Parameters
    3. Patterns for Configuring Stacks
      1. Antipattern: Manual Stack Parameters
      2. Pattern: Stack Environment Variables
      3. Pattern: Scripted Parameters
      4. Pattern: Stack Configuration Files
      5. Pattern: Wrapper Stack
      6. Pattern: Pipeline Stack Parameters
      7. Pattern: Stack Parameter Registry
    4. Configuration Registry
      1. Implementing a Configuration Registry
      2. Single or Multiple Configuration Registries
    5. Handling Secrets as Parameters
      1. Encrypting Secrets
      2. Secretless Authorization
      3. Injecting Secrets at Runtime
      4. Disposable Secrets
    6. Conclusion
  11. 8. Core Practice: Continuously Test and Deliver
    1. Why Continuously Test Infrastructure Code?
      1. What Continuous Testing Means
      2. What Should We Test with Infrastructure?
    2. Challenges with Testing Infrastructure Code
      1. Challenge: Tests for Declarative Code Often Have Low Value
      2. Challenge: Testing Infrastructure Code Is Slow
      3. Challenge: Dependencies Complicate Testing Infrastructure
    3. Progressive Testing
      1. Test Pyramid
      2. Swiss Cheese Testing Model
    4. Infrastructure Delivery Pipelines
      1. Pipeline Stages
      2. Scope of Components Tested in a Stage
      3. Scope of Dependencies Used for a Stage
      4. Platform Elements Needed for a Stage
      5. Delivery Pipeline Software and Services
    5. Testing in Production
      1. What You Can’t Replicate Outside Production
      2. Managing the Risks of Testing in Production
    6. Conclusion
  12. 9. Testing Infrastructure Stacks
    1. Example Infrastructure
      1. The Example Stack
      2. Pipeline for the Example Stack
    2. Offline Testing Stages for Stacks
      1. Syntax Checking
      2. Offline Static Code Analysis
      3. Static Code Analysis with API
      4. Testing with a Mock API
    3. Online Testing Stages for Stacks
      1. Preview: Seeing What Changes Will Be Made
      2. Verification: Making Assertions About Infrastructure Resources
      3. Outcomes: Proving Infrastructure Works Correctly
    4. Using Test Fixtures to Handle Dependencies
      1. Test Doubles for Upstream Dependencies
      2. Test Fixtures for Downstream Dependencies
      3. Refactor Components So They Can Be Isolated
    5. Life Cycle Patterns for Test Instances of Stacks
      1. Pattern: Persistent Test Stack
      2. Pattern: Ephemeral Test Stack
      3. Antipattern: Dual Persistent and Ephemeral Stack Stages
      4. Pattern: Periodic Stack Rebuild
      5. Pattern: Continuous Stack Reset
    6. Test Orchestration
      1. Support Local Testing
      2. Avoid Tight Coupling with Pipeline Tools
      3. Test Orchestration Tools
    7. Conclusion
  13. III. Working with Servers and Other Application Runtime Platforms
  14. 10. Application Runtimes
    1. Cloud Native and Application-Driven Infrastructure
    2. Application Runtime Targets
      1. Deployable Parts of an Application
      2. Deployment Packages
    3. Deploying Applications to Servers
      1. Packaging Applications in Containers
      2. Deploying Applications to Server Clusters
    4. Deploying Applications to Application Clusters
    5. Packages for Deploying Applications to Clusters
    6. Deploying FaaS Serverless Applications
    7. Application Data
      1. Data Schemas and Structures
      2. Cloud Native Application Storage Infrastructure
    8. Application Connectivity
    9. Service Discovery
    10. Conclusion
  15. 11. Building Servers as Code
    1. What’s on a Server
    2. Where Things Come From
    3. Server Configuration Code
      1. Server Configuration Code Modules
      2. Designing Server Configuration Code Modules
      3. Versioning and Promoting Server Code
      4. Server Roles
    4. Testing Server Code
      1. Progressively Testing Server Code
      2. What to Test with Server Code
      3. How to Test Server Code
    5. Creating a New Server Instance
      1. Hand-Building a New Server Instance
      2. Using a Script to Create a Server
      3. Using a Stack Management Tool to Create a Server
      4. Configuring the Platform to Automatically Create Servers
      5. Using a Networked Provisioning Tool to Build a Server
    6. Prebuilding Servers
      1. Hot-Cloning a Server
      2. Using a Server Snapshot
      3. Creating a Clean Server Image
    7. Configuring a New Server Instance
      1. Frying a Server Instance
      2. Baking Server Images
      3. Combining Baking and Frying
      4. Applying Server Configuration When Creating a Server
    8. Conclusion
  16. 12. Managing Changes to Servers
    1. Change Management Patterns: When to Apply Changes
      1. Antipattern: Apply On Change
      2. Pattern: Continuous Configuration Synchronization
      3. Pattern: Immutable Server
    2. How to Apply Server Configuration Code
      1. Pattern: Push Server Configuration
      2. Pattern: Pull Server Configuration
    3. Other Server Life Cycle Events
      1. Stopping and Restarting a Server Instance
      2. Replacing a Server Instance
      3. Recovering a Failed Server
    4. Conclusion
  17. 13. Server Images as Code
    1. Building a Server Image
      1. Why Build a Server Image?
      2. How to Build a Server Image
      3. Tools for Building Server Images
      4. Online Image Building Process
      5. Offline Image Building Process
    2. Origin Content for a Server Image
      1. Building from a Stock Server Image
      2. Building a Server Image from Scratch
      3. Provenance of a Server Image and its Content
    3. Changing a Server Image
      1. Reheating or Baking a Fresh Image
      2. Versioning a Server Image
      3. Updating Server Instances When an Image Changes
      4. Providing and Using a Server Image Across Teams
      5. Handling Major Changes to an Image
    4. Using a Pipeline to Test and Deliver a Server Image
      1. Build Stage for a Server Image
      2. Test Stage for a Server Image
      3. Delivery Stages for a Server Image
    5. Using Multiple Server Images
      1. Server Images for Different Infrastructure Platforms
      2. Server Images for Different Operating Systems
      3. Server Images for Different Hardware Architectures
      4. Server Images for Different Roles
      5. Layering Server Images
      6. Sharing Code Across Server Images
    6. Conclusion
  18. 14. Building Clusters as Code
    1. Application Cluster Solutions
      1. Cluster as a Service
      2. Packaged Cluster Distribution
    2. Stack Topologies for Application Clusters
      1. Monolithic Stack Using Cluster as a Service
      2. Monolithic Stack for a Packaged Cluster Solution
      3. Pipeline for a Monolithic Application Cluster Stack
      4. Example of Multiple Stacks for a Cluster
    3. Sharing Strategies for Application Clusters
      1. One Big Cluster for Everything
      2. Separate Clusters for Delivery Stages
      3. Clusters for Governance
      4. Clusters for Teams
      5. Service Mesh
    4. Infrastructure for FaaS Serverless
    5. Conclusion
  19. IV. Designing Infrastructure
  20. 15. Core Practice: Small, Simple Pieces
    1. Designing for Modularity
      1. Characteristics of Well-Designed Components
      2. Rules for Designing Components
      3. Use Testing to Drive Design Decisions
    2. Modularizing Infrastructure
      1. Stack Components Versus Stacks as Components
      2. Using a Server in a Stack
    3. Drawing Boundaries Between Components
      1. Align Boundaries with Natural Change Patterns
      2. Align Boundaries with Component Life Cycles
      3. Align Boundaries with Organizational Structures
      4. Create Boundaries That Support Resilience
      5. Create Boundaries That Support Scaling
      6. Align Boundaries to Security and Governance Concerns
      7. Conclusion
  21. 16. Building Stacks from Components
    1. Infrastructure Languages for Stack Components
      1. Reuse Declarative Code with Modules
      2. Dynamically Create Stack Elements with Libraries
    2. Patterns for Stack Components
      1. Pattern: Facade Module
      2. Antipattern: Obfuscation Module
      3. Antipattern: Unshared Module
      4. Pattern: Bundle Module
      5. Antipattern: Spaghetti Module
      6. Pattern: Infrastructure Domain Entity
    3. Building an Abstraction Layer
    4. Conclusion
  22. 17. Using Stacks as Components
    1. Discovering Dependencies Across Stacks
      1. Pattern: Resource Matching
      2. Pattern: Stack Data Lookup
      3. Pattern: Integration Registry Lookup
      4. Dependency Injection
    2. Conclusion
  23. V. Delivering Infrastructure
  24. 18. Organizing Infrastructure Code
    1. Organizing Projects and Repositories
      1. One Repository, or Many?
      2. One Repository for Everything
      3. A Separate Repository for Each Project (Microrepo)
      4. Multiple Repositories with Multiple Projects
    2. Organizing Different Types of Code
      1. Project Support Files
      2. Cross-Project Tests
      3. Dedicated Integration Test Projects
      4. Organize Code by Domain Concept
      5. Organizing Configuration Value Files
    3. Managing Infrastructure and Application Code
      1. Delivering Infrastructure and Applications
      2. Testing Applications with Infrastructure
      3. Testing Infrastructure Before Integrating
      4. Using Infrastructure Code to Deploy Applications
    4. Conclusion
  25. 19. Delivering Infrastructure Code
    1. Delivering Infrastructure Code
      1. Building an Infrastructure Project
      2. Packaging Infrastructure Code as an Artifact
      3. Using a Repository to Deliver Infrastructure Code
    2. Integrating Projects
      1. Pattern: Build-Time Project Integration
      2. Pattern: Delivery-Time Project Integration
      3. Pattern: Apply-Time Project Integration
    3. Using Scripts to Wrap Infrastructure Tools
      1. Assembling Configuration Values
      2. Simplifying Wrapper Scripts
    4. Conclusion
  26. 20. Team Workflows
    1. The People
    2. Who Writes Infrastructure Code?
    3. Applying Code to Infrastructure
      1. Applying Code from Your Local Workstation
      2. Applying Code from a Centralized Service
      3. Personal Infrastructure Instances
      4. Source Code Branches in Workflows
    4. Preventing Configuration Drift
      1. Minimize Automation Lag
      2. Avoid Ad Hoc Apply
      3. Apply Code Continuously
      4. Immutable Infrastructure
    5. Governance in a Pipeline-based Workflow
      1. Reshuffling Responsibilities
      2. Shift Left
      3. An Example Process for Infrastructure as Code with Governance
    6. Conclusion
  27. 21. Safely Changing Infrastructure
    1. Reduce the Scope of Change
      1. Small Changes
      2. Example of Refactoring
    2. Pushing Incomplete Changes to Production
      1. Parallel Instances
      2. Backward Compatible Transformations
      3. Feature Toggles
    3. Changing Live Infrastructure
      1. Infrastructure Surgery
      2. Expand and Contract
      3. Zero Downtime Changes
    4. Continuity
      1. Continuity by Preventing Errors
      2. Continuity by Fast Recovery
      3. Continuous Disaster Recovery
      4. Chaos Engineering
      5. Planning for Failure
    5. Data Continuity in a Changing System
      1. Lock
      2. Segregate
      3. Replicate
      4. Reload
      5. Mixing Data Continuity Approaches
    6. Conclusion
  28. Index

Product information

  • Title: Infrastructure as Code, 2nd Edition
  • Author(s): Kief Morris
  • Release date: December 2020
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781098114671