Microservices need architects

For the past two years, microservices have been taking the software development world by storm. Their use has been popularized by organizations adopting Agile Software Development, continuous delivery and DevOps, as a logical next step in the progression to remove bottlenecks that slow down software delivery. As a result, much of the public discussion on microservices is coming from software developers who feel liberated by the chance to code without the constraints and dependencies of a monolithic application structure. While this “inside the microservice” perspective is important and compelling for the developer community, there are a number of other important areas of microservice architecture that aren’t getting enough attention.

Specifically, as the number of microservices in an organization grows linearly, this new collection of services forms an unbounded system whose complexity threatens to increase exponentially. This complexity introduces problems with security, visibility, testability, and service discoverability. However, many developers currently treat these as “operational issues” and leave them for someone else to fix downstream. If addressed up front—when the software system is being designed—these aspects can be handled more effectively. Likewise, although there is discussion on techniques to define service boundaries and on the linkage between organizational structure and software composition, these areas can also benefit from an architectural approach. So, where are the architects?

From enterprise architect to software architect

In my experience working with a wide range of organizations at various stages of microservices adoption, the architects are often interested observers but not the leaders of the movement. I believe this relates to their interpretation of their own roles. Following a corporate trend of IT centralization, the first decade of this century was a heyday for enterprise architects who worked directly for CIOs and CTOs. These EAs took a “command and control” approach, focusing on IT governance through enterprise standards, compliance, and application rationalization. However, mobile opportunities, cloud infrastructure and Agile practices have reversed that trend as organizations disperse in order to optimize for delivery speed over enterprise consistency. To add value in this new paradigm, architects need to change their goals and approach.

The “Evolutionary Architect” chapter of Sam Newman’s excellent book, “Building Microservices”, describes how architects can help organizations succeed with microservices, likening the role to that of a town planner who maintains high level control over the city’s zones while allowing the builders freedom with them. This type of approach is vital in order to deal with the inevitable growth and change that comes from a service-based software system. Roy Fielding provided a definition of “software architecture” in his 2000 PHD dissertation along with a set of common terms. In this same work, he defined the REST architectural style that applies directly to microservice integration and interoperability. In fact, the field of software architecture offers today’s architects a mature body of knowledge whose lessons can be applied practically to deal with many of microservice architecture’s most challenging issues.

Microservices’ architectural needs

In my opinion, there are three factors critical to the long term success of microservices implementations that are not getting enough attention. The first is service design. Many online discussions mention domain-driven design (DDD), seams, and cohesion, but organizations are struggling with how to apply these concepts practically. As early as 1972, David Parnas’ paper on software modularization emphasized the need for functional cohesion within modules and loose coupling between them. These guidelines have been fleshed out in the years since by Fielding’s REST, Vaughn Vernon’s application of DDD, and Greg Young’s work on CQRS and Event Sourcing. These resources and many others are useful in the hands of architects seeking to make microserivces succeed at their organizations.

Secondly, although there is an industry focus on the intricacies of each individual microservice, there is much less deliberation about how to shape the way these services interact with one another. As stated above, an organization’s superset of microservices forms a complex system that can lead to numerous issues unless there is a conscious focus on system design. One key to dealing with this complexity is understanding the difference between the essential complexity necessary for this software system to deliver its functional scope and the accidental complexity resident in the technological components. Fred Brooks’ 1986 paper, “No Silver Bullet”, examines how to use iterative development to deal with the former while minimizing the latter. Tackling an organization’s microservice system complexity proactively through evolvable system design is a fundamental task best performed by its architects.

Lastly, microservice architecture’s Agile and DevOps lineage entangles its effectiveness with its adopting organization’s culture and practices. “Conway’s Law”—from Mel Conway’s 1967 paper “How Do Committees Invent?”—is experiencing a revival for illustrating how software system architecture mimics organizational structure. This concept was expanded upon in a 2012 Harvard Business School study that linked the coupling of an organization’s structure to the modularity of its software products. This symbiotic relationship means that architects must also contribute to organizational design in order to represent the interests of the associated software systems.

So for enterprise architects, application architects, and solution architects out there who want to make a difference in this new world, I challenge you to dig into the teachings of software architecture and apply them in this new context. Tackle the role using iterative, evolutionary methods with a dose of systems thinking. Help guide your organization in the areas of service design, system design, and even organizational design. Become a software architect who leads your organization to success through microservice architecture’s current principles, over its potential pitfalls and eventually with its descendant disciplines.

---

This post is a collaboration between O’Reilly and CA Technologies. See our statement of editorial independence.