Learn Grafana 10.x - Second Edition

Learn Grafana 10.x - Second Edition

by Eric Salituro
Released December 2023
Publisher(s): Packt Publishing
ISBN: 9781803231082

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Get up and running with building data pipelines and creating interactive dashboards to visualize, monitor, and present a wide variety of time-series data with this comprehensive introductory guide

Key Features

  • Install, set up, and configure Grafana for real-time data analysis, visualization, and alerting
  • Visualize and monitor data using data sources such as InfluxDB, Telegraf, Prometheus, and Elasticsearch
  • Explore Grafana's cloud support with Microsoft Azure, Amazon CloudWatch, and Google Cloud Monitoring
  • Purchase of the print or Kindle book includes a free PDF eBook

Book Description

Get ready to unlock the full potential of the open-source Grafana observability platform, ideal for analyzing and monitoring time-series data with this updated second edition. This beginners guide will help you get up to speed with Grafana’s latest features for querying, visualizing, and exploring logs and metrics, no matter where they are stored.

Starting with the basics, this book demonstrates how to quickly install and set up a Grafana server using Docker. You’ll then be introduced to the main components of the Grafana interface before learning how to analyze and visualize data from sources such as InfluxDB, Telegraf, Prometheus, Logstash, and Elasticsearch. The book extensively covers key panel visualizations in Grafana, including Time Series, Stat, Table, Bar Gauge, and Text, and guides you in using Python to pipeline data, transformations to facilitate analytics, and templating to build dynamic dashboards. Exploring real-time data streaming with Telegraf, Promtail, and Loki, you’ll work with observability features like alerting rules and integration with PagerDuty and Slack. As you progress, the book addresses the administrative aspects of Grafana, from configuring users and organizations to implementing user authentication with Okta and LDAP, as well as organizing dashboards into folders, and more.

By the end of this book, you’ll have gained all the knowledge you need to start building interactive dashboards.

What you will learn

  • Learn the techniques of data visualization using Grafana
  • Get familiar with the major components of Time series visualization
  • Explore data transformation operations, query inspector, and time interval settings
  • Work with advanced dashboard features, such as annotations, variable-based templating, and dashboard linking and sharing
  • Connect user authentication through Okta, Google, GitHub, and other external providers
  • Discover Grafana’s monitoring support for cloud service infrastructures

Who this book is for

This book is for business intelligence developers, business analysts, data analysts, and anyone interested in performing time-series data analysis and monitoring using Grafana. You’ll also find this book useful if you’re looking to create and share interactive dashboards or get up to speed with the latest features of Grafana. Although no prior knowledge of Grafana is required, basic knowledge of data visualization and some Python programming experience will help you understand the concepts covered in the book.

Table of contents

  1. Learn Grafana 10.x
Copyright © 2023 Packt Publishing
  2. Contributors
  3. About the author
  4. About the reviewers
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
    4. Download the example code files
    5. Conventions used
    6. Get in touch
    7. Share Your Thoughts
    8. Download a free PDF copy of this book
  6. Part 1 – Getting Started with Grafana
  7. Chapter 1: Introducing Data Visualization with Grafana
    1. Technical requirements
    2. Appreciating data and visualization
      1. Storing, retrieving, and visualizing data
    3. Why Grafana?
      1. Exploration
      2. Analysis
      3. Presentation
      4. Observability
      5. Choosing Grafana
    4. Installing Grafana
      1. Grafana in a Docker container
      2. Make and Makefile
      3. Grafana for macOS
      4. Grafana for Linux
      5. Grafana for Windows
      6. Grafana Cloud
    5. Connecting to the Grafana server
    6. Summary
    7. Further reading
  8. Chapter 2: Touring the Grafana Interface
    1. Technical requirements
    2. Exploring Grafana—the Home dashboard
    3. Introducing the Grafana search bar
      1. Grafana logo
      2. Organization menu
      3. Search box
      4. + (add) menu
      5. Help
      6. Grafana blog
      7. User menu
    4. Expanding Grafana’s main menu
      1. Home
      2. Starred
      3. Dashboards
      4. Explore
      5. Alerting
      6. Connections
      7. Administration
    5. Exploring the Grafana dashboard UI
      1. Grafana dashboard title bar
      2. Dashboard content
    6. Summary
  9. Chapter 3: Diving into Grafana's Time Series Visualization
    1. Touring the Grafana panel UI
      1. Creating a simple data source
      2. Creating a graph panel
    2. Generating data series in the Query tab
      1. What is a query?
      2. Query tab features
      3. Duplicating an existing query
      4. Transform
    3. Editing the panel settings
      1. Selecting panel visualizations
      2. The Panel options section
      3. The Tooltip section
      4. The Legend section
      5. The Axis section
      6. The Graph styles section
      7. The Standard options section
      8. The Data links section
      9. The Value mappings section
      10. The Thresholds section
    4. Monitoring with the Alert tab
    5. Further exploration
    6. Summary
  10. Part 2 – Real-World Grafana
  11. Chapter 4: Connecting Grafana to a Prometheus Data Source
    1. Technical requirements
    2. Installing the Prometheus server
      1. Installing Prometheus from Docker
      2. Configuring the Prometheus data source
    3. Exploring Prometheus
      1. Using Explore for investigation
      2. Configuring Grafana metrics
    4. Querying the Prometheus data source
      1. Typing in a metrics query
      2. Querying for process metrics
      3. Querying for memory metrics
    5. Detecting trends with aggregations
      1. Applying aggregations to our query data
    6. Understanding the data source limitations
      1. Querying limits for series aggregations
      2. Querying limits for time aggregations
      3. Exploring data source dashboards
    7. Summary
  12. Chapter 5: Extracting and Visualizing Data with InfluxDB and Grafana
    1. Technical requirements
    2. Making advanced queries
      1. Launching server Docker containers
      2. Writing the ETL script
      3. Running the script
      4. Setting up an InfluxDB database
      5. Configuring the InfluxDB data source
    3. Understanding the time series data visualization
      1. Displaying time-aggregated data
      2. Debugging queries with Query inspector
      3. Observing time interval effects
      4. Setting the minimum interval
    4. Setting the axis
      1. Setting axis units
      2. Converting temperature into Fahrenheit
      3. Autoscaling the Y axis
      4. Dual y axis display
    5. Working with legends
      1. Setting legend contents
      2. Enabling legend aggregations
    6. Summary
  13. Chapter 6: Shaping Data with Grafana Transformations
    1. About Grafana DataFrames and transformations
    2. Exploring the various transformation functions
      1. Installing the TestData data source
      2. Selecting transformations
      3. Adding fields
      4. Modifying fields
      5. Filtering results
      6. Reducing rows
      7. Combining tables
    3. Expanding analysis with a transformation
    4. Chaining transformations into a visualization pipeline
      1. Limitations of transformations
    5. Summary
  14. Chapter 7: Surveying Key Grafana Visualizations
    1. Technical requirements
    2. Launching server Docker containers
    3. Setting up the InfluxDB database
      1. Initializing the InfluxDB server
      2. Generating an API token
      3. Configuring the InfluxDB data source
      4. Building the Python Docker container
      5. Loading the data
    4. Reviewing the table visualization
      1. Comparing aggregations
      2. Overriding field settings
      3. Setting a display name
    5. Introducing the stat visualization
      1. Creating a stat visualization panel
      2. Setting stat styles
      3. Defining value mappings in a stat visualization
    6. Adding visual interest with a gauge
      1. Exploring the gauge options
      2. Setting the threshold values and colors
    7. Going linear with a bar gauge
      1. Exploring the bar gauge settings
      2. Building a bar gauge
    8. Summary
  15. Chapter 8: Surveying Additional Grafana Visualizations
    1. Technical requirements
    2. Launching server Docker containers
    3. Setting up the InfluxDB database
      1. Initializing the InfluxDB server
      2. Generating an API token
    4. Configuring the InfluxDB data source
      1. Building the Python Docker container
    5. Exploring spatial data with the Geomap visualization
      1. Ingesting a new earthquake dataset
      2. Mapping earthquake data with the Geomap visualization
    6. Displaying category data with a bar chart visualization
      1. Understanding histograms
      2. Producing histograms with transformations
    7. Displaying histogram data with the bar chart visualization
    8. Visualizing histogram data over time with the heatmap
    9. Summary
  16. Chapter 9: Creating Insightful Dashboards
    1. Technical requirements
      1. Launching server Docker containers
      2. Setting up the InfluxDB database
    2. Designing a dashboard
      1. Conveying information
      2. Determining the visual context
      3. Prioritizing elements of importance
    3. Creating a high information-density dashboard
      1. Ingesting the weather data
      2. Designing the dashboard
      3. Adding the first row
      4. Adding the second row
      5. Adding the third row
      6. Adding the fourth row
      7. Adding the fifth row
    4. Creating a high-information visibility dashboard
      1. Designing the dashboard
      2. Ingesting the earthquake data
      3. Adding the first row
      4. Adding the second row
      5. Adding the third row
    5. Parting thoughts
      1. Considering layout
      2. Automating ingestion
    6. Summary
  17. Chapter 10: Working with Advanced Dashboard Features and Elasticsearch
    1. Technical requirements
    2. Building the data server
    3. Connecting Grafana to Elasticsearch
    4. Querying with Elasticsearch
    5. Creating a template variable
      1. Adding template variables to the graph panel
      2. Templating additional variables
      3. Creating ad hoc filters
      4. Repeating rows and panels with template variables
    6. Creating a new dashboard
      1. Setting up the template variables
      2. Configuring the panels
    7. Linking dashboards
      1. Adding dashboard tags
      2. Locking down a template variable
      3. Creating dashboard links
    8. Annotating dashboards
      1. Annotating the graph panel
      2. Querying tagged annotations
      3. Creating Elasticsearch annotation queries
    9. Sharing dashboards
      1. Sharing dashboard links
      2. Sharing dashboards by exporting
      3. Sharing dashboard snapshots
    10. Summary
  18. Chapter 11: Streaming Real-Time IoT Data from Telegraf Agent to Grafana Live
    1. Technical requirements
    2. Streaming real-time data from Telegraf to Grafana
      1. Setting up Grafana for streaming
      2. Installing the Telegraf agent
      3. Configuring the Telegraf agent
      4. Running Telegraf
    3. Streaming IoT data with MQTT and Mosquitto
      1. Describing the pipeline architecture
      2. Configuring the Telegraf MQTT consumer
      3. Installing the Mosquitto broker
    4. Generating messages to an IoT data pipeline
      1. Examining the simulated metrics data
      2. Reviewing the mqtt_pub.py script
      3. Running the mqtt_pub.py script
      4. Exploring the IoT data
    5. Summary
  19. Chapter 12: Monitoring Data Streams with Grafana Alerts
    1. Technical requirements
    2. Monitoring and observability
      1. Monitoring processes
      2. Monitoring system resources
      3. Monitoring applications
      4. Capturing metrics for alerting
    3. Alerting in Grafana
      1. Alert rules
      2. Labels
      3. Notification policies
      4. Contact points
    4. Defining alert rules
      1. What condition…
      2. What metrics…
      3. How long…
      4. Monitoring systems
      5. Monitoring applications
    5. Alert messaging to contact points
      1. Configuring an email contact point
      2. Configuring a PagerDuty contact point
      3. Configuring a Slack contact point
    6. Routing alerts with notification policies
      1. Specific routing
      2. Mute timings
      3. On your own
    7. Summary
  20. Chapter 13: Exploring Log Data with Grafana’s Loki
    1. Technical requirements
    2. Loading system logs into Loki
      1. Networking our services
      2. Installing Promtail
      3. Installing Loki
      4. Launching Grafana
      5. Creating a Loki data source
    3. Visualizing Loki log data with Explore
    4. Simulating logs with flog
      1. Configuring promtail
      2. Promtail in Docker Compose
    5. Alternative Docker log capture
    6. Querying logs and metrics with Explore
    7. Summary
  21. Part 3 – Managing Grafana
  22. Chapter 14: Organizing Dashboards and Folders
    1. Managing dashboards and folders
      1. Naming a dashboard
      2. Dashboard naming tips
      3. Working with dashboard folders
      4. Tips to manage dashboard folders
    2. Starring and tagging dashboards
      1. Marking dashboards as favorites
      2. Tagging dashboards
    3. Building and running dashboard playlists
      1. Creating a playlist
      2. Displaying a playlist
      3. Displaying playlists in normal mode
      4. Displaying playlists in TV mode
      5. Displaying playlists in Kiosk mode
      6. Displaying playlists with auto fit panels
      7. Editing a playlist
    4. Exploring the Dashboard list panel
      1. Setting Dashboard list panel options
    5. Duplicating dashboards
      1. Internal dashboard duplications
      2. External dashboard duplications
    6. Summary
  23. Chapter 15: Managing Permissions for Users, Teams, and Organizations
    1. Understanding key permission concepts
      1. Organizations
      2. Users
      3. Roles
      4. Teams
    2. Adding users
      1. Adding users – by invitation only
      2. Adding users – a self-service model
    3. Setting permissions
      1. Setting organization roles
      2. Setting folder permissions
      3. Setting dashboard permissions
    4. Establishing teams
      1. Setting up a team
      2. Permission rules
    5. Administering users and organizations
      1. Managing users
      2. Organization admin and Super Admin roles
      3. Managing organizations
    6. Summary
  24. Chapter 16: Authenticating Grafana Logins Using LDAP or OAuth 2 Providers
    1. Authenticating with OpenLDAP
    2. Setting up an OpenLDAP server
      1. Configuring Grafana to use LDAP
      2. Securing the bind password
      3. Testing the Grafana configuration
      4. Adding a user to OpenLDAP
      5. Looking up a user in Grafana
    3. Authenticating with OAuth 2
      1. GitHub OAuth 2 authentication
      2. Google OAuth 2 authentication
    4. Authenticating with Okta
    5. Summary
  25. Chapter 17: Cloud Monitoring AWS, Azure, and GCP
    1. Configuring an AWS CloudWatch data source
      1. Creating the policy
      2. Creating the user
      3. Configuring a new Grafana connection for AWS CloudWatch
    2. Configuring a Microsoft Azure Monitor data source
      1. Registering the Grafana application
      2. Setting the application role
      3. Generating application secrets
      4. Configuring a new Grafana connection for Azure Monitor
      5. Configuring Azure Log Analytics
    3. Configuring a GCM data source
      1. Enabling a Google Cloud API
      2. Creating a Google service account
      3. Configuring a new Grafana connection for GCM
    4. Summary
  26. Index
    1. Why subscribe?
  27. Other Books You May Enjoy
    1. Packt is searching for authors like you
    2. Share Your Thoughts
    3. Download a free PDF copy of this book

Product information

  • Title: Learn Grafana 10.x - Second Edition
  • Author(s): Eric Salituro
  • Release date: December 2023
  • Publisher(s): Packt Publishing
  • ISBN: 9781803231082