Designing for Scalability with Erlang/OTP

Designing for Scalability with Erlang/OTP

by Francesco Cesarini, Steve Vinoski
Released May 2016
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781449361563

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

If you need to build a scalable, fault tolerant system with requirements for high availability, discover why the Erlang/OTP platform stands out for the breadth, depth, and consistency of its features. This hands-on guide demonstrates how to use the Erlang programming language and its OTP framework of reusable libraries, tools, and design principles to develop complex commercial-grade systems that simply cannot fail.

In the first part of the book, you’ll learn how to design and implement process behaviors and supervision trees with Erlang/OTP, and bundle them into standalone nodes. The second part addresses reliability, scalability, and high availability in your overall system design. If you’re familiar with Erlang, this book will help you understand the design choices and trade-offs necessary to keep your system running.

  • Explore OTP’s building blocks: the Erlang language, tools and libraries collection, and its abstract principles and design rules
  • Dive into the fundamentals of OTP reusable frameworks: the Erlang process structures OTP uses for behaviors
  • Understand how OTP behaviors support client-server structures, finite state machine patterns, event handling, and runtime/code integration
  • Write your own behaviors and special processes
  • Use OTP’s tools, techniques, and architectures to handle deployment, monitoring, and operations

Publisher resources

View/Submit Errata

Table of contents

  1. Preface
    1. Francesco: Why This Book?
    2. Steve: Why This Book?
    3. Who Should Read This Book
    4. How To Read This Book
    5. Acknowledgments
    6. Conventions Used in This Book
    7. Using Code Examples
    8. Safari® Books Online
    9. How to Contact Us
  2. 1. Introduction
    1. Defining the Problem
    2. OTP
      1. Erlang
      2. Tools and Libraries
      3. System Design Principles
      4. Erlang Nodes
    3. Distribution, Infrastructure, and Multicore
    4. Summing Up
    5. What You’ll Learn in This Book
  3. 2. Introducing Erlang
    1. Recursion and Pattern Matching
    2. Functional Influence
      1. Fun with Anonymous Functions
      2. List Comprehensions: Generate and Test
    3. Processes and Message Passing
    4. Fail Safe!
    5. Links and Monitors for Supervision
      1. Links
      2. Monitors
    6. Records
    7. Maps
    8. Macros
    9. Upgrading Modules
    10. ETS: Erlang Term Storage
    11. Distributed Erlang
      1. Naming and Communication
      2. Node Connections and Visibility
    12. Summing Up
    13. What’s Next?
  4. 3. Behaviors
    1. Process Skeletons
    2. Design Patterns
      1. Callback Modules
    3. Extracting Generic Behaviors
      1. Starting the Server
      2. The Client Functions
      3. The Server Loop
      4. Functions Internal to the Server
    4. The Generic Server
    5. Message Passing: Under the Hood
    6. Summing Up
    7. What’s Next?
  5. 4. Generic Servers
    1. Generic Servers
    2. Behavior Directives
    3. Starting a Server
    4. Message Passing
      1. Synchronous Message Passing
      2. Asynchronous Message Passing
      3. Other Messages
      4. Unhandled Messages
      5. Synchronizing Clients
    5. Termination
    6. Call Timeouts
      1. Deadlocks
    7. Generic Server Timeouts
      1. Hibernating Behaviors
    8. Going Global
    9. Linking Behaviors
    10. Summing Up
    11. What’s Next?
  6. 5. Controlling OTP Behaviors
    1. The sys Module
      1. Tracing and Logging
      2. System Messages
      3. Your Own Trace Functions
      4. Statistics, Status, and State
      5. The sys Module Recap
    2. Spawn Options
      1. Memory Management and Garbage Collection
      2. Spawn Options to Avoid
      3. Timeouts
    3. Summing Up
    4. What’s Next?
  7. 6. Finite State Machines
    1. Finite State Machines the Erlang Way
    2. Coffee FSM
      1. The Hardware Stub
      2. The Erlang Coffee Machine
    3. Generic FSMs
    4. A Behavior Example
      1. Starting the FSM
      2. Sending Events
      3. Termination
    5. Summing Up
    6. Get Your Hands Dirty
      1. The Phone Controllers
      2. Let’s Test It
    7. What’s Next?
  8. 7. Event Handlers
    1. Events
    2. Generic Event Managers and Handlers
      1. Starting and Stopping Event Managers
      2. Adding Event Handlers
      3. Deleting an Event Handler
      4. Sending Synchronous and Asynchronous Events
      5. Retrieving Data
      6. Handling Errors and Invalid Return Values
      7. Swapping Event Handlers
      8. Wrapping It All Up
    3. The SASL Alarm Handler
    4. Summing Up
    5. What’s Next?
  9. 8. Supervisors
    1. Supervision Trees
    2. OTP Supervisors
      1. The Supervisor Behavior
      2. Starting the Supervisor
      3. The Supervisor Specification
      4. Dynamic Children
      5. Non-OTP-Compliant Processes
      6. Scalability and Short-Lived Processes
      7. Synchronous Starts for Determinism
      8. Testing Your Supervision Strategy
    3. How Does This Compare?
    4. Summing Up
    5. What’s Next?
  10. 9. Applications
    1. How Applications Run
    2. The Application Structure
    3. The Callback Module
      1. Starting and Stopping Applications
    4. Application Resource Files
      1. The Base Station Controller Application File
    5. Starting an Application
    6. Environment Variables
    7. Application Types and Termination Strategies
    8. Distributed Applications
    9. Start Phases
    10. Included Applications
      1. Start Phases in Included Applications
    11. Combining Supervisors and Applications
    12. The SASL Application
      1. Progress Reports
      2. Error Reports
      3. Crash Reports
      4. Supervisor Reports
    13. Summing Up
    14. What’s Next?
  11. 10. Special Processes and Your Own Behaviors
    1. Special Processes
      1. The Mutex
      2. Starting Special Processes
      3. The Mutex States
      4. Handling Exits
      5. System Messages
      6. Trace and Log Events
      7. Putting It Together
      8. Dynamic Modules and Hibernating
    2. Your Own Behaviors
      1. Rules for Creating Behaviors
      2. An Example Handling TCP Streams
    3. Summing Up
    4. What’s Next?
  12. 11. System Principles and Release Handling
    1. System Principles
      1. Release Directory Structure
      2. Release Resource Files
      3. Creating a Release
      4. Creating the Boot File
      5. Creating a Release Package
      6. Start Scripts and Configuring on the Target
      7. Arguments and Flags
      8. The init Module
    2. Rebar3
      1. Generating a Rebar3 Release Project
      2. Creating a Release with Rebar3
      3. Rebar3 Releases with Project Dependencies
    3. Wrapping Up
    4. What’s Next?
  13. 12. Release Upgrades
    1. Software Upgrades
      1. The First Version of the Coffee FSM
      2. Adding a State
    2. Creating a Release Upgrade
      1. The Code to Upgrade
      2. Application Upgrade Files
      3. High-Level Instructions
      4. Release Upgrade Files
      5. Low-Level Instructions
      6. Installing an Upgrade
      7. The Release Handler
      8. Upgrading Environment Variables
    3. Upgrading Special Processes
    4. Upgrading in Distributed Environments
    5. Upgrading the Emulator and Core Applications
    6. Upgrades with Rebar3
    7. Summing Up
    8. What’s Next?
  14. 13. Distributed Architectures
    1. Node Types and Families
    2. Networking
      1. Distributed Erlang
      2. Sockets and SSL
      3. Service Orientation and Microservices
      4. Peer to Peer
    3. Interfaces
    4. Summing Up
    5. What’s Next?
  15. 14. Systems That Never Stop
    1. Availability
      1. Fault Tolerance
      2. Resilience
      3. Reliability
      4. Sharing Data
      5. Tradeoffs Between Consistency and Availability
    2. Summing Up
    3. What’s Next?
  16. 15. Scaling Out
    1. Horizontal and Vertical Scaling
    2. Capacity Planning
      1. Capacity Testing
      2. Balancing Your System
      3. Finding Bottlenecks
      4. System Blueprints
    3. Load Regulation and Backpressure
    4. Summing Up
    5. What’s Next?
  17. 16. Monitoring and Preemptive Support
    1. Monitoring
      1. Logs
      2. Metrics
      3. Alarms
    2. Preemptive Support
    3. Summing Up
    4. What’s Next?
  18. Index

Product information

  • Title: Designing for Scalability with Erlang/OTP
  • Author(s): Francesco Cesarini, Steve Vinoski
  • Release date: May 2016
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781449361563