Jan Machacek on reactive systems and shifting business demands

In this podcast episode, O’Reilly’s Jeff Bleiel talks with Jan Machacek, CTO at Cake Solutions, a global consulting firm that specializes in building reactive systems. They discuss Machacek’s experiences in building reactive microservice-based systems and how these systems can help companies keep up with changing business demands.

Machacek believes when designing large systems, it’s really important for an engineering team to avoid falling for a distributed monolith. “A distributed monolith is a massive anti-pattern in microservices: it is a result of not clearly separating the microservices’ contexts,” he says. “So, what we end up having is a system with tangled service interaction or code dependencies, and hidden data flows. Such a system can actually be successful—in the sense that it works now—but the flexibility to change the services that make up the system is greatly reduced.”

According to Machacek, a system’s elasticity is about how the system copes with increased demand: the load it processes as well as the features it includes. A well-designed and well-engineered microservices system is a building block for handling the processing load as well as for designing a flexible platform to evolve its functionality. “If the system you are designing is indeed resilient, responsive, and message driven, that makes it a lot easier for the system to be elastic,” he says.

Machacek has built—as well as rebuilt—entire monolithic systems to take advantage of reactive principles. Some of the projects he’s worked on include an exercise analysis platform, document processor, a secure messaging system, mobile media delivery, and many more. “In order for a reactive system to be resilient, architects and developers really need to think about these systems and understand what is going to happen when the system’s components fail,” he says. “And this is not an academic exercise—some components will fail. This is where engineering can help to uncover and force business decisions; for example, what happens when our user database is gone and we still need to log in a user? The system’s design and implementation must make it possible to provide options for the business.”

Accepting that failures happen makes it critical to be able to discover them. And so, every microservice in a reactive system needs to provide timely responses, even if only to say that it is failing. According to Machacek, there’s nothing worse than sending a message to a service and receiving silence. “Service silences or timeouts are horrific,” he says. “It’s much better to know that the service rejects the request because it’s unable to handle it. This allows for implementation of back pressure, or at least smart circuit breakers, which give the system the ability to recover from partial failures.”

All of these concepts are complex to architect and implement at first. However, if you persevere and deliver responsive, resilient, elastic, and message-driven services (all of them thoroughly tested), then deploy them in an appropriate infrastructure (again, thoroughly tested, focusing especially on resilience and elasticity), you’ll end up with the holy grail of systems: one that can truly evolve with the demands that the users place on it.

For more on reactive systems and shifting business demands, download a free preview chapter from Machacek’s forthcoming book, “Reactive Systems Architecture.”

This post and podcast is a collaboration between O’Reilly and Cake Solutions. See our statement of editorial independence.