The Future of Observability with OpenTelemetry

The Future of Observability with OpenTelemetry

by Ted Young
Released December 2021
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781098118426

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

The world of observability is in the midst of a seismic shift. Until recently, collecting telemetry data meant running multiple, uncoordinated pipelines. Now, OpenTelemetry provides users with a single integrated stream of data, containing all common forms of observability. With this report, CTOs, analysts, system architects, and DevOps practitioners will learn how OpenTelemetry will change the way we practice observability.

Author Ted Young, cofounder of the OpenTelemetry project, demonstrates how this collection of tools, APIs, and SDKs gives OSS libraries and services the ability to provide their own instrumentation. You'll save time, often in situations where time is critical, by focusing on how your system is changing without first having to identify what is changing.

By the end of this report, you'll learn how a modern observability pipeline works. You'll examine:

  • How the field of observability will change over the next two years
  • Why native instrumentation for OSS libraries is a revolutionary idea
  • How some current practices are very cumbersome wastes of time
  • The promises of AIOps: which ones are valid, and which are hype
  • How to budget for and capitalize on the upcoming shifts in technology
  • The quality of different vendor and technology offerings

Table of contents

  1. Foreword
  2. 1. Myths and Historical Accidents
    1. Transactions and Resources: What It’s All About
    2. The “Three Browser Tabs” of Observability
    3. How We Actually Observe Our Systems in the Real World
    4. Not Three Pillars, but a Single Braid
  3. 2. The Value of Structured Data
    1. Attributes: Defining Keys and Values
    2. Events: The Basis for Everything
    3. Resources: Observing Services and Machines
    4. Spans: Observing Transactions
    5. Tracing: Like Logging, Only Better
    6. Metrics: Observing Events in Aggregate
    7. Metrics Connected to Events: A Single Unified System
    8. Automated Analysis and the Single Braid
    9. The Point: Automated Analysis Saves You Time
  4. 3. The Limitations of Automated Analysis
    1. Beware of Hype
    2. Time Is Our Most Precious Resource
  5. 4. Supporting Open Source and Native Instrumentation
    1. Observability Is Drowning in Solution-Specific Instrumentation
    2. Applications Are Locked In by Solution-Specific Instrumentation
    3. Solution-Specific Instrumentation for OSS Is Basically Impossible
    4. How Do You Pick a Logging Library?
    5. Decomposing the Problem
      1. Requirement: Separate Instrumentation, Telemetry, and Analysis
      2. Requirement: Zero Dependencies
      3. Requirement: Strict Backward Compatibility and Long-Term Support
    6. Separation of Concerns Is Fundamental to Good Design
  6. 5. OpenTelemetry: An Architectural Overview
    1. Signals
      1. Context
      2. Propagators
      3. Tracing
      4. Metrics
      5. Logs
      6. Baggage
    2. The OpenTelemetry Client Architecture
    3. Client Architecture: The Instrumentation API
      1. Providers
    4. Client Architecture: The SDK
      1. Samplers
      2. Exporters
    5. Client Architecture: Library Instrumentation
    6. The Collector
      1. Collector Architecture: Receivers
      2. Collector Architecture: Processors
      3. Collector Architecture: Exporters
      4. Collector Architecture: Pipelines
  7. 6. Stability and Long-Term Support
    1. Signal Lifecycle
    2. API Stability
    3. SDK and Collector Stability
    4. Upgrading OpenTelemetry Clients
  8. 7. Suggested Setups and Telemetry Pipelines
    1. Installing the OpenTelemetry Client
      1. Picking an Exporter
      2. Installing Library Instrumentation
      3. Choosing a Propagator
    2. Deploying a Local Collector
    3. Deploying a Collector-Processing Pool
    4. Adding Additional Processing Pools
    5. Managing Existing Telemetry with the Collector
    6. Switching Providers
  9. 8. How to Roll Out OpenTelemetry Across Your Organization
    1. The Primary Goal
    2. Pick a High-Value Target
    3. Centralize Telemetry Management
    4. Breadth Before Depth
    5. Work with Management
    6. Join the Community
    7. Thanks for Reading
  10. A. OpenTelemetry Project Organization
    1. The Specification
    2. Project Governance
    3. Distros
    4. The Registry
  11. B. OpenTelemetry Project Roadmap
    1. Core Components
    2. The Future
      1. eBPF
      2. RUM
      3. OpenTelemetry Control Plane
      4. Columnar-Encoded OTLP
  12. About the Author

Product information

  • Title: The Future of Observability with OpenTelemetry
  • Author(s): Ted Young
  • Release date: December 2021
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781098118426