The Fundamentals of Telemetry Pipelines

Chapter 5. Embracing Compliance

Finance is about the money you make. Compliance is about the money you keep.

Anonymous

In Chapter 4, I imagine we all universally agreed that $65 million is a lot of money. As the amount of one single observability bill, it feels a bit much. Well, in this chapter the stakes are raised just a touch higher. Millions are for lightweights. Let’s talk about billions: $30.6 billion, to be in the approximate ballpark. Give or take a few million.

Fines for financial services noncompliance have reached as high as $30.6 billion in some extreme cases. Admittedly, this was for a host of irregularities ranging from misdemeanors to out-and-out fraud and enabling money laundering on an industrial scale, and so not necessarily what you can expect from mishandling your telemetry data.

But that’s just financial noncompliance. What about health care data compliance, like HIPAA? For those hoping to operate in Europe, what about General Data Protection Regulation (GDPR)?

Whatever business you’re in, there is likely a reasonable expectation that you will comply with various rules and regulations. Even if you’re a small service provider just trying to obey local restrictions around how you safely handle PII, there are rules.

And observability tools hate rules. Limiting data access to specific individuals, ensuring that data is anonymized, ensuring that data can be processed and dropped if necessary—all of these things give creators of observability tools nightmares. ...