Infrastructure as Code

Infrastructure as Code

by Kief Morris
Released June 2016
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781491924358

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon
Start your free trial

Book description

Virtualization, cloud, containers, server automation, and software-defined networking are meant to simplify IT operations. But many organizations adopting these technologies have found that it only leads to a faster-growing sprawl of unmanageable systems. This is where infrastructure as code can help. With this practical guide, author Kief Morris of ThoughtWorks shows you how to effectively use principles, practices, and patterns pioneered through the DevOps movement to manage cloud age infrastructure.

Ideal for system administrators, infrastructure engineers, team leads, and architects, this book demonstrates various tools, techniques, and patterns you can use to implement infrastructure as code. In three parts, you’ll learn about the platforms and tooling involved in creating and configuring infrastructure elements, patterns for using these tools, and practices for making infrastructure as code work in your environment.

  • Examine the pitfalls that organizations fall into when adopting the new generation of infrastructure technologies
  • Understand the capabilities and service models of dynamic infrastructure platforms
  • Learn about tools that provide, provision, and configure core infrastructure resources
  • Explore services and tools for managing a dynamic infrastructure
  • Learn specific patterns and practices for provisioning servers, building server templates, and updating running servers

Publisher resources

View/Submit Errata

Table of contents

  1. Preface
    1. How I Learned to Stop Worrying and to Love the Cloud
      1. My First Virtual Server Farm
      2. The Sorcerer’s Apprentice
      3. Cloud from Scratch
    2. Why I’m Writing This Book
    3. Who This Book Is For
    4. What Tools Are Covered
    5. How to Read This Book
    6. Conventions Used in This Book
    7. Safari® Books Online
    8. How to Contact Us
    9. Acknowledgments
  2. I. Foundations
  3. 1. Challenges and Principles
    1. Why Infrastructure as Code?
    2. What Is Infrastructure as Code?
      1. Goals of Infrastructure as Code
    3. Challenges with Dynamic Infrastructure
      1. Server Sprawl
      2. Configuration Drift
      3. Snowflake Servers
      4. Fragile Infrastructure
      5. Automation Fear
      6. Erosion
    4. Principles of Infrastructure as Code
      1. Systems Can Be Easily Reproduced
      2. Systems Are Disposable
      3. Systems Are Consistent
      4. Processes Are Repeatable
      5. Design Is Always Changing
    5. Practices
      1. Use Definition Files
      2. Self-Documented Systems and Processes
      3. Version All the Things
      4. Continuously Test Systems and Processes
      5. Small Changes Rather Than Batches
      6. Keep Services Available Continuously
    6. Antifragility: Beyond “Robust”
      1. The Secret Ingredient of Antifragile IT Systems
    7. Conclusion
    8. What’s Next?
  4. 2. Dynamic Infrastructure Platforms
    1. What Is a Dynamic Infrastructure Platform?
    2. Requirements for a Dynamic Infrastructure Platform
      1. Programmable
      2. On-Demand
      3. Self-Service
    3. Infrastructure Resources Provided by the Platform
      1. Compute Resources
      2. Storage Resources
      3. Network Resources
    4. Types of Dynamic Infrastructure Platforms
      1. Public IaaS Cloud
      2. Community IaaS Cloud
      3. Private IaaS Cloud
      4. Antipattern: Hand-Cranked Cloud
      5. Hybrid and Mixed Cloud Options
      6. Bare-Metal Clouds
    5. Deciding on a Dynamic Infrastructure Platform
      1. Public or Private?
      2. Cloud Portability
    6. Mechanical Sympathy with the Cloud and Virtualization
    7. Conclusion
  5. 3. Infrastructure Definition Tools
    1. Choosing Tools for Infrastructure as Code
      1. Requirement: Scriptable Interface
      2. Requirement: Unattended Mode for Command-Line Tools
      3. Requirement: Support for Unattended Execution
      4. Requirement: Externalized Configuration
    2. Configuration Definition Files
      1. Reusability with Configuration Definitions
    3. Working with Infrastructure Definition Tools
      1. Provisioning Infrastructure with Procedural Scripts
      2. Defining Infrastructure Declaratively
      3. Using Infrastructure Definition Tools
      4. Configuring Servers
    4. Configuration Registries
      1. Lightweight Configuration Registries
      2. Is a Configuration Registry a CMDB?
      3. The CMDB Audit and Fix Antipattern
      4. The Infrastructure-as-Code Approach to CMDB
    5. Conclusion
  6. 4. Server Configuration Tools
    1. Goals for Automated Server Management
    2. Tools for Different Server Management Functions
      1. Tools for Creating Servers
      2. Tools for Configuring Servers
      3. Tools for Packaging Server Templates
      4. Tools for Running Commands on Servers
      5. Using Configuration from a Central Registry
    3. Server Change Management Models
      1. Ad Hoc Change Management
      2. Configuration Synchronization
      3. Immutable Infrastructure
      4. Containerized Services
    4. Containers
      1. Managing Ruby Applications with and without Containers
      2. Are Containers Virtual Machines?
      3. Using Containers Rather than Virtual Machines
      4. Running Containers
      5. Security and Containers
    5. Conclusion
  7. 5. General Infrastructure Services
    1. Considerations for Infrastructure Services and Tools
      1. Prefer Tools with Externalized Configuration
      2. Prefer Tools That Assume Infrastructure Is Dynamic
      3. Prefer Products with Cloud-Compatible Licensing
      4. Prefer Products That Support Loose Coupling
    2. Sharing a Service Between Teams
      1. Service Instance Templates
    3. Monitoring: Alerting, Metrics, and Logging
      1. Alerting: Tell Me When Something Is Wrong
      2. Metrics: Collect and Analyze Data
      3. Log Aggregation and Analysis
    4. Service Discovery
      1. Server-Side Service Discovery Pattern
      2. Client-Side Service Discovery Pattern
    5. Distributed Process Management
      1. Orchestrating Processes with Server Roles
      2. Orchestrating Processes with Containers
      3. Scheduling Short Jobs
      4. Container Orchestration Tools
    6. Software Deployment
      1. Deployment Pipeline Software
      2. Packaging Software
    7. Conclusion
  8. II. Patterns
  9. 6. Patterns for Provisioning Servers
    1. Server Provisioning
      1. A Server’s Life
      2. What Goes onto a Server
      3. Types of Things on a Server
      4. Server Roles
    2. Patterns for Creating Servers
      1. Antipattern: Handcrafted Server
      2. Practice: Wrap Server Creation Options in a Script
      3. Antipattern: Hot Cloned Server
      4. Pattern: Server Template
      5. Antipattern: Snowflake Factory
    3. Patterns for Bootstrapping New Servers
      1. Pushing to Bootstrap
      2. Pulling to Bootstrap
      3. Practice: Smoke Test Every New Server Instance
    4. Conclusion
  10. 7. Patterns for Managing Server Templates
    1. Stock Templates: Can’t Someone Else Do It?
    2. Provisioning Servers Using Templates
      1. Provisioning at Creation Time
      2. Provisioning in the Template
      3. Balancing Provisioning Across Template and Creation
    3. The Process for Building a Server Template
      1. Creating Templates for Multiple Platforms
    4. Origin Images
      1. Antipattern: Hot Cloned Server Template
      2. Baking a Template from an OS Installation Image
      3. Baking a Template from a Stock Image
      4. Building a Template from a Unikernel
      5. Customizing a Server Template without Booting It
    5. Updating Server Templates
      1. Reheating a Template
      2. Baking a Fresh Template
      3. Versioning Server Templates
    6. Building Templates for Roles
      1. Pattern: Layered Template
      2. Sharing Base Scripts for Templates
    7. Automating Server Template Management
      1. Customizing Servers Before Baking
      2. Practice: Automatically Test Server Templates
    8. Conclusion
  11. 8. Patterns for Updating and Changing Servers
    1. Models for Server Change Management
      1. Ad Hoc Change Management
      2. Continuous Configuration Synchronization
      3. Immutable Servers
      4. Containerized Servers
    2. General Patterns and Practices
      1. Practice: Minimize Server Templates
      2. Practice: Replace Servers When the Server Template Changes
      3. Pattern: Phoenix Servers
    3. Patterns and Practices for Continuous Deployment
      1. Pattern: Masterless Configuration Management
      2. Practice: Apply Cron
      3. Continuous Synchronization Flow
      4. The Unconfigured Country
    4. Patterns and Practices for Immutable Servers
      1. Server Image as Artifact
      2. Simplifying Confirmation Management Tooling with Immutable Servers
      3. Immutable Server Flow
      4. Bootstrap Configuration with Immutable Servers
      5. Transactional Server Updates
    5. Practices for Managing Configuration Definitions
      1. Practice: Keep Configuration Definitions Minimal
      2. Organizing Definitions
      3. Practice: Use Test-Driven Development (TDD) to Drive Good Design
    6. Conclusion
  12. 9. Patterns for Defining Infrastructure
    1. Environments
      1. Antipattern: Handcrafted Infrastructure
      2. Defining Infrastructure Stacks as Code
      3. Antipattern: Per-Environment Definition Files
      4. Pattern: Reusable Definition Files
      5. Practice: Test and Promote Stack Definitions
      6. Self-Service Environments
    2. Organizing Infrastructure
      1. Antipattern: Monolithic Stack
      2. Avoid “Lift and Shift” When Migrating Infrastructure
      3. Dividing an Application Environment into Multiple Stacks
      4. Managing Configuration Parameters Between Stacks
      5. Sharing Infrastructure Elements
      6. Practice: Manage Application Code and Infrastructure Code Together
      7. Approaches to Sharing Definitions
      8. Practice: Align Infrastructure Design with the Scope of Change
      9. Example: An Infrastructure Design for Microservices
    3. Running Definition Tools
    4. Conclusion
  13. III. Practices
  14. 10. Software Engineering Practices for Infrastructure
    1. System Quality
      1. Poor-Quality Systems Are Difficult to Change
      2. High-Quality Systems Are Easier and Safer to Change
      3. Infrastructure Quality Through Code
      4. Fast Feedback
    2. VCS for Infrastructure Management
      1. What to Manage in a VCS
    3. Continuous Integration (CI)
      1. Continuously Testing Branches Is Not Continuous Integration
      2. Who Broke the Build?
      3. Ignoring Tests That Fail
      4. CI for Infrastructure
    4. Continuous Delivery (CD)
      1. The Problem with the Integration Phase
      2. Deployment Pipelines and Change Pipelines
      3. Continuous Delivery Is Not Continuous Deployment
    5. Code Quality
      1. Clean Code
      2. Practice: Manage Technical Debt
    6. Managing Major Infrastructure Changes
      1. Feature Toggles
    7. Conclusion
  15. 11. Testing Infrastructure Changes
    1. The Agile Approach to Testing
      1. Automating Tests for Fast Feedback
      2. Organically Building a Test Suite
    2. Structuring the Test Suite: The Test Pyramid
      1. Avoiding an Unbalanced Test Suite
      2. Practice: Test at the Lowest Level Possible
      3. Practice: Only Implement the Layers You Need
      4. Practice: Prune the Test Suite Often
      5. Practice: Continuously Review Testing Effectiveness
    3. Implementing a Balanced Test Suite
      1. Low-Level Testing
      2. Mid-Level Testing
      3. Higher-Level Tests
      4. Testing Operational Quality
    4. Managing Test Code
      1. Practice: Keep Test Code with the Code It Tests
      2. Anti-Pattern: Reflective Tests
      3. Techniques to Isolate Components for Testing
      4. Refactoring Components so They Can Be Isolated
      5. Managing External Dependencies
      6. Test Setup
    5. Roles and Workflow for Testing
      1. Principle: People Should Write Tests for What They Build
      2. The Test-Writing Habit
      3. Principle: Everyone Should Have Access to the Testing Tools
      4. The Value of a Quality Analyst
      5. Test-Driven Development (TDD)
    6. Conclusion
  16. 12. Change Management Pipelines for Infrastructure
    1. Benefits of a Change Management Pipeline
    2. Guidelines for Designing Pipelines
      1. Ensure Consistency Across Stages
      2. Get Immediate Feedback for Every Change
      3. Run Automated Stages Before Manual Stages
      4. Get Production-Like Sooner Rather Than Later
    3. Basic Pipeline Designs
      1. The Local Development Stage
      2. The Build Stage
      3. Publishing a Configuration Artifact
      4. Automated Testing Stages
      5. Manual Validation Stages
      6. Apply to Live
      7. The Rhythm of the Pipeline
    4. Practices for Using a Pipeline
      1. Practice: Prove Production Readiness for Every Change
      2. Practice: Start Every Change from the Beginning of the Pipeline
      3. Practice: Stop the Line on Any Failure
    5. Scaling Pipelines to More Complex Systems
      1. Pattern: Fan-In Pipelines
      2. Practice: Keep Pipelines Short
      3. Practice: Decouple Pipelines
      4. Integration Models
    6. Techniques for Handling Dependencies Between Components
      1. Pattern: Library Dependency
      2. Pattern: Self-Provisioned Service Instance
      3. Providing Pre-Release Library Builds
      4. Providing Test Instances of a Service to Consumers
      5. Using Test Instances of a Service as a Consumer
    7. Practices for Managing Interfaces Between Components
      1. Practice: Ensure Backward Compatibility of Interfaces
      2. Practice: Decouple Deploying from Releasing
      3. Practice: Use Version Tolerance
      4. Practice: Provide Test Doubles
      5. Practice: Test the Provider with Contract Tests
      6. Practice: Test with a Reference Consumer
      7. Practice: Smoke Test the Provider Interface
      8. Practice: Run Consumer-Driven Contract (CDC) Tests
    8. Conclusion
  17. 13. Workflow for the Infrastructure Team
    1. Automate Anything That Moves
      1. Make the Change Manually
      2. Ad Hoc Automation
      3. Autonomic Automation
      4. Autonomic Automation Workflow
    2. Using a Local Sandbox
      1. Using Local Virtualization for a Sandbox
      2. Example Workflow with Local Testing
      3. Using the Virtualization Platform for Sandboxes
    3. Codebase Organization Patterns
      1. Antipattern: Branch-Based Codebases
      2. Pattern: One Trunk per Component
      3. Pattern: Single Trunk
    4. Workflow Effectiveness
      1. Expediting Changes
      2. Code Reviews
      3. Fitting Governance into the Workflow
    5. Conclusion
  18. 14. Continuity with Dynamic Infrastructure
    1. Service Continuity
      1. True Availability
      2. Using Dynamic Server Pools for Recovery
      3. Software Design for Dynamic Infrastructure
      4. Compartmentalizing for Continuity
    2. Zero-Downtime Changes
      1. Pattern: Blue-Green Replacement
      2. Pattern: Phoenix Replacement
      3. Practice: Reduce the Scope of Replacement
      4. Pattern: Canary Replacement
      5. Routing Traffic for Zero-Downtime Replacements
      6. Zero-Downtime Changes with Data
    3. Data Continuity
      1. Replicating Data Redundantly
      2. Regenerating Data
      3. Delegating Data Persistence
      4. Backing Up to Persistent Storage
    4. Disaster Recovery
      1. Continuous Disaster Recovery
      2. The DR Plan: Planning for Disaster
      3. Practice: Prefer Rebuilding Things over Cold Standby
      4. Continuous Monitoring through the Pipeline
    5. Security
      1. Automatically Papering over Compromises
      2. Reliable Updates as a Defense
      3. Provenance of Packages
      4. Automated Hardening
      5. Automating Security Validation in the Pipeline
      6. The Change Pipeline as a Vulnerability
      7. Managing Security Risks with Cloud Accounts
    6. Conclusion
  19. 15. Organizing for Infrastructure as Code
    1. Evolutionary Architecture
      1. Learning under Fire
      2. Start with a Trailblazer Pipeline
    2. Measuring Effectiveness
      1. Agree on Desired Outcomes First
      2. Choose Metrics that Help the Team
      3. Track and Improve Cycle Time
      4. Use Kanban to Make Work Visible
      5. Retrospectives and Post-Mortems
    3. Organize to Empower Users
      1. Pitfalls of the Divided Function Model
      2. Adopt a Self-Service Model
      3. Take Full Responsibility: You Build It, You Run It
      4. Organizing into Cross-Functional Teams
    4. Governance through Continuous Change Management
      1. Provide Solid Building Blocks
      2. Prove Operational Readiness in the Pipeline
      3. Sharing Ownership of Operational Quality
      4. Review and Audit Automated Processes
      5. Optimize for Time to Detect and Fix Problems
    5. Conclusion: It’s Never Finished
  20. Index

Product information

  • Title: Infrastructure as Code
  • Author(s): Kief Morris
  • Release date: June 2016
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781491924358