Kubernetes Security and Observability

Chapter 5. Observability

In this chapter we will discuss the difference between monitoring and observability in the context of Kubernetes deployments. We will explain best practices and tools for implementing observability in your Kubernetes cluster. In the next chapter we will cover how you can use observability to secure your cluster.

Observability has been a topic of discussion recently in the Kubernetes community and has garnered a lot of interest. We begin by understanding the difference between monitoring and observability. We then look at why observability is critical to security in a distributed application like Kubernetes, and review tools and reference implementations for observability. While observability is a broad topic and applies to several areas, we will keep the discussion focused on Kubernetes in this chapter. Let’s start by looking at monitoring and observability and how they are different.

Monitoring

Monitoring is a known set of measurements in a system that are used to alert for deviations from a normal range. The following are examples of types of data you can monitor in Kubernetes:

• Pod logs

• Network flow logs

• Application flow logs

• Audit logs

Examples of metrics you can monitor include the following:

• Connections per second

• Packets per second, bytes per second

• Application (API) requests per second

• CPU and memory utilization

These logs and metrics can help you identify known failures and provide more information about the symptom to help you remediate the ...