Cover | Table of Contents

Table of Contents

Chapter 1: Motivation

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

WE LIVE IN HARD TIMES. THE SOCIAL MARKET ECONOMY IS BEING REPLACED BY A GLOBAL MARKET economy, and the marketing guys rule the world. As a consequence, you have to be fast and flexible to survive. It's a renaissance of Darwinism:

It is not the strongest of the species that survive, nor the most intelligent, but the ones most responsive to change.

The key is flexibility. For all major companies and large distributed systems, information technology (IT) flexibility is paramount. In fact, IT has become a key business value enabler.

At the same time, processes and systems are also becoming more and more complex. We have left the stage where automation was primarily a matter of individual systems, and are fast moving toward a world where all those individual systems will become one distributed system. The challenge is maintainability.

It turns out that the old ways of dealing with the problems of scalability and distribution don't work anymore. We can no longer harmonize or maintain control. Centralization, the precondition for harmonization and control, does not scale, and we have reached its limits. For this reason, we need a new approach—an approach that accepts heterogeneity and leads to decentralization.

In addition, we have to solve the problem of the business/IT gap. This gap is primarily one of semantics—business people and IT people appear to speak and think in entirely different languages. The new approach must bring business and IT much closer than ever before.

Service-oriented architecture (SOA) is exactly what's needed. It's an approach that helps systems remain scalable and flexible while growing, and that also helps bridge the business/IT gap. The approach consists of three major elements:

• Services, which on the one hand represent self-contained business functionalities that can be part of one or more processes, and on the other hand, can be implemented by any technology on any platform.
• A specific infrastructure, called the enterprise service bus (ESB), that allows us to combine these services in an easy and flexible manner.
• Policies and processes

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Characteristics of Large Distributed Systems

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

SOA is a concept for large distributed systems. To understand SOA, you have to understand the properties of large distributed systems.

First, large systems must deal with legacies. You can't introduce SOA by designing everything from scratch. You have to deal with the fact that most of the systems that are in use will remain in use. This also means that establishing SOA is not a project like designing a new system. It involves changing the structure of an existing system, which means you have to deal with old platforms and backward-compatibility issues. In fact, SOA is an approach for the maintenance of large system landscapes.

By nature, all large systems are also heterogeneous. These systems have different purposes, times of implementation, and ages, and you will find that the system landscapes are accretions of different platforms, programming languages, programming paradigms, and even middleware. In the past, there have been many attempts to solve the problems of scalability by harmonization. And, yes, harmonization helps. Withdrawing old platforms or systems that are no longer maintainable is an important improvement. But chances are that your systems will never be fully harmonized. Right before you remove the last piece of heterogeneity, a company merger, or some other change will open Pandora's box again.

One reason for the heterogeneity is that large systems and their data have an incredibly long lifetime. During this lifetime, new functionality that promotes the business is developed by adding new systems and processes. Removing existing systems and data may seem to have no business value, but such changes are investments in the maintainability of your system. Often, these investments come too late, and become incredibly expensive because the systems are out of control, and all the knowledge about them is gone.

By nature, large systems are complex. For this reason, finding out the right places for and determining the effects of modifications can be tough. As [Spanyi03] states:

There is no such thing as a "quick fix . . . ". Organizations are complex business systems, within which a change in any one component is likely to have an impact on other components.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

The Tale of the Magic Bus

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Once upon a time, there was a company that had grown over years and years. Over the course of those years, the system landscape became infected with a disease called "mess." As a consequence, the people lost control over their system, and whenever they tried to improve it, either the effort required proved too high, or they made things even worse.

The company asked several experts, sages, and wizards for help, and they came up with a lot of ideas introducing new patterns, protocols, and system designs. But as a consequence, things only got worse and worse, so the company became desperate.

One day, a prince called Enterprise Integrate came along, and claimed that he had the solution. He told the CEO of the company, "Your problem is your lack of interoperability. When you have such a mess of systems and protocols, it is a problem that you have to create an individual solution for each kind of connection. Even if you only had 10 different platforms and 5 different protocols, if you wanted to enable each platform to communicate with each other platform, you would need over 100 different solutions. And you have many more than 10 platforms." The exact way this number was arrived at was not passed on, but some sketches of the processing led to the conclusion that each possible connection of two platforms was combined with the average number of used protocols.

"Look at my drawing," the prince continued (it's reproduced here in ). "This is how your problem gets solved. We create a Magic Bus."

Figure : The original drawing of the Magic Bus

"What's a Magic Bus?" the CEO asked.

The prince answered, "A Magic Bus is a piece of software that reduces the number of connections and interfaces in your system. While your approach might require up to n × (n−1)/2 connections for n systems (and twice as many interfaces), the Magic Bus requires only one connection and interface for each system. That means that for n systems, the number of interfaces is reduced by a factor of n−1 (a factor of 9 for 10 systems, 29 for 30 systems, and 49 for 50 systems)."

Convinced by these numbers, the CEO agreed to switch to this new technique. The praise for it was so strong that all over the country, the bards started to write songs about the Magic Bus. The most famous was written by a band called The Who, who praised the bus with the following words (see [MagicBus] for the complete lyrics of the song):

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

What We Can Learn from the Tale of the Magic Bus

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

These days, we often praise the "bus-age wonder." Although the idea of an IT bus is pretty old, recently, there has been a renaissance of this concept. It started with the introduction of the enterprise application integration bus (EAI bus), which was later replaced by the enterprise service bus (ESB). This has become so important that there has been a public flame war about who invented the ESB (see [ESB Inventor]).

Buses represent high interoperability. The idea behind them is that instead of creating and maintaining individual communication channels between different systems, each system only has to connect to the bus to be able to connect to all other systems. Of course, this does simplify connectivity, but as the preceding tale revealed, this approach has drawbacks. Connectivity scales to chaos unless structures are imposed. That's why we replaced global variables with procedures, and business object models with modules and components. And it will happen again when we start to deal with services in an unstructured way.

Thus, your first lesson is that in order for large systems to scale, more than just interoperability is required. You need structures provided by technical and organizational rules and patterns. High interoperability must be accompanied by a well-defined architecture, structures, and processes. If you realize this too late, you may be out of the market.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

History of SOA

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Surprisingly, it is hard to find out who coined the term SOA. Roy Schulte at Gartner gave me the exact history in a private conversation in 2007:

Alexander Pasik, a former analyst at Gartner, coined the term SOA for a class on middleware that he was teaching in 1994. Pasik was working before XML or Web Services were invented, but the basic SOA principles have not changed.

Pasik was driven to create the term SOA because "client/server" had lost its classical meaning. Many in the industry had begun to use "client/server" to mean distributed computing involving a PC and another computer. A desktop "client" PC typically ran user-facing presentation logic, and most of the business logic. The backend "server" computer ran the database management system, stored the data, and sometimes ran some business logic. In this usage, "client" and "server" generally referred to the hardware. The software on the frontend PC sometimes related to the server software in the original sense of client/server, but that was largely irrelevant. To avoid confusion between the new and old meanings of client/server, Pasik stressed "server orientation" as he encouraged developers to design SOA business applications.

Gartner analysts Roy W. Schulte and Yefim V. Natis published the first reports about SOA in 1996. See [Gartner96] and [Gartner03] for details.

The real momentum for SOA was created by Web Services, which, initially driven by Microsoft, reached a broader public in 2000 (see for details about the history of Web Services). To quote [Gartner03]:

Although Web Services do not necessarily translate to SOA, and not all SOA is based on Web Services, the relationship between the two technology directions is important and they are mutually influential: Web Services momentum will bring SOA to mainstream users, and the best-practice architecture of SOA will help make Web Services initiatives successful.

Soon, other companies and vendors jumped in (including major IT vendors such as IBM, Oracle, HP, SAP, and Sun). There was money to be made by explaining the idea, and by selling new concepts and tools (or rebranded concepts and tools). In addition, the time was right because companies were increasingly seeking to integrate their businesses with other systems, departments, and companies (remember the B2B hype).

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

SOA in Five Slides

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The rest of this book will discuss several aspects of SOA in practice. That means we'll go a bit deeper than the usual "five slides" approach, which presents SOA in such a simple way that everybody wonders what's so complicated and/or important about it.

Still, to give you an initial idea about the essence of what you will learn, here are my five slides introducing SOA. Bear in mind that these five slides give an oversimplified impression. The devil is in the details. Nevertheless, this overview might look a bit different from the usual advertisement for SOA.

Service-oriented architecture (SOA) is a paradigm for the realization and maintenance of business processes that span large distributed systems. It is based on three major technical concepts: services, interoperability through an enterprise service bus, and loose coupling.

• A service is a piece of self-contained business functionality. The functionality might be simple (storing or retrieving customer data), or complex (a business process for a customer's order). Because services concentrate on the business value of an interface, they bridge the business/IT gap.
• An enterprise service bus (ESB) is the infrastructure that enables high interoperability between distributed systems for services. It makes it easier to distribute business processes over multiple systems using different platforms and technologies.
• Loose coupling is the concept of reducing system dependencies. Because business processes are distributed over multiple backends, it is important to minimize the effects of modifications and failures. Otherwise, modifications become too risky, and system failures might break the overall system landscape. Note, however, that there is a price for loose coupling: complexity. Loosely coupled distributed systems are harder to develop, maintain, and debug.

Distributed processing is not a technical detail. Distributed processing changes everything in your company. Introducing new functionality is no longer a matter of assigning a specific department a specific task. It is now a combination of multiple tasks for different systems. These systems and the involved teams have to collaborate.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Chapter 2: SOA

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

THE GOAL OF THIS CHAPTER IS TO INTRODUCE SOA AS A CONCEPT. MY AIM IS TO OUTLINE THE fundamental aspects of SOA, and to show you the circumstances in which its use is appropriate. The important point is that SOA is a paradigm (or concept, or philosophy) that leads to a value system for large distributed systems with different owners.

I will cite, compare, and discuss definitions from various existing sources, such as the OASIS SOA Reference Model, Wikipedia.org, and some books. I will show how and why these definitions differ, and point out the key aspects of SOA that emerge.

It is hard to find an exact definition of the term SOA. The problem is not that there aren't any definitions; the problem is that there are many different definitions. To give you an idea of how they are similar and dissimilar, a selection of published definitions are sidebars in this chapter. You'll find some common phrases and attributes as you read them, but you will also find a lot of differences in the context, level of abstraction, and wording.

However, at least all definitions agree that SOA is a paradigm for improved flexibility.

In computing, the term "Service-Oriented Architecture" (SOA) expresses a software architectural concept that defines the use of services to support the requirements of software users. In a SOA environment, nodes on a network make resources available to other participants in the network as independent services that the participants access in a standardized way. Most definitions of SOA identify the use of Web services (i.e., using SOAP or REST) in its implementation. However, one can implement SOA using any service-based technology.

...

Unlike traditional point-to-point architectures, SOAs comprise loosely coupled, highly interoperable application services. These services interoperate based on a formal definition independent of the underlying platform and programming language (e.g., WSDL). The interface definition encapsulates (hides) the vendor and language-specific implementation. A SOA is independent of development technology (such as Java and .NET). The software components become very reusable because the interface is defined in a standards-compliant manner.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Definitions of SOA

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

It is hard to find an exact definition of the term SOA. The problem is not that there aren't any definitions; the problem is that there are many different definitions. To give you an idea of how they are similar and dissimilar, a selection of published definitions are sidebars in this chapter. You'll find some common phrases and attributes as you read them, but you will also find a lot of differences in the context, level of abstraction, and wording.

However, at least all definitions agree that SOA is a paradigm for improved flexibility.

In computing, the term "Service-Oriented Architecture" (SOA) expresses a software architectural concept that defines the use of services to support the requirements of software users. In a SOA environment, nodes on a network make resources available to other participants in the network as independent services that the participants access in a standardized way. Most definitions of SOA identify the use of Web services (i.e., using SOAP or REST) in its implementation. However, one can implement SOA using any service-based technology.

...

Unlike traditional point-to-point architectures, SOAs comprise loosely coupled, highly interoperable application services. These services interoperate based on a formal definition independent of the underlying platform and programming language (e.g., WSDL). The interface definition encapsulates (hides) the vendor and language-specific implementation. A SOA is independent of development technology (such as Java and .NET). The software components become very reusable because the interface is defined in a standards-compliant manner.

SOA is not a concrete architecture: it is something that leads to a concrete architecture. You might call it a style, paradigm, concept, perspective, philosophy, or representation. That is, SOA is not a concrete tool or framework you can purchase. It is an approach, a way of thinking, a value system that leads to certain concrete decisions when designing a concrete software architecture.

This aspect of SOA has a very important consequence: you can't buy SOA. There is no tool or recipe you can use to make everything work. While applying this paradigm to your concrete situation, you must make specific decisions that are appropriate for your circumstances.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

SOA Drivers

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Of course, flexibility is dealt with very differently on different layers and in different components. So, one important question is which kinds of software systems this paradigm is appropriate for. As it turns out, SOA copes well with many difficult-to-handle characteristics of large systems.

As businesses grow, they become more and more complex, and more and more systems and companies become involved. There is constant integration and constant change. SOA is well suited to dealing with complex distributed systems. According to the OASIS SOA Reference Model (see [OasisSoaRM06]), it is a paradigm for "organizing and utilizing distributed capabilities."

A more IT-conforming term for "distributed capabilities" would be "distributed systems," or, as Wikipedia's SOA definitions say, "nodes of a network" or "resources of a network."

SOA allows entities that need certain distributed capabilities to locate and make use of those capabilities. In other words, it facilitates interactions between service providers and service consumers, enabling the realization of business functionalities.

The OASIS SOA Reference Model's definition of SOA continues by stating that those distributed capabilities "may be under the control of different ownership domains." This is a very important point that is often suppressed in SOA definitions. You won't find it in any of the other definitions quoted in this chapter, but it is the key for certain properties of SOA, and a major reason why SOA is not only a technical concept.

SOA includes practices and processes that are based on the fact that networks of distributed systems are not controlled by single owners. Different teams, different departments, or even different companies may manage different systems (see ). Thus, different platforms, schedules, priorities, budgets, and so on must be taken into account. This concept is key to understanding SOA and large distributed systems in general.

Figure : Distributed systems with different owners

The ways you deal with problems and make modifications in environments with different owners and in environments where you control everything will, by necessity, vary. You cannot implement functionality and modify behavior the same way in large systems as in smaller systems. (See for a discussion of the most important aspects of large systems.) One important consideration is that "politics" come into play: you have to compromise with others, and you have to accept that different priorities and solutions exist. Because you can't control everything, you have to accept that you may not always be able to do things your way.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

SOA Concepts

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Here are the key technical concepts of SOA that allow it to cope with the system characteristics just described:

• Services
• Interoperability
• Loose coupling

Software development is the art of abstraction. We have to abstract reality in such a way that only the relevant aspects of a problem are handled. However, we all know that we can abstract from different perspectives. SOA aims to abstract by concentrating on the business aspects of a problem. The fundamental term introduced here is service. In essence, a service is an IT representation of some business functionality. The goal of SOA is to structure large distributed systems based on the abstractions of business rules and functions.

This gives a clear structure to the IT systems we design and develop. Although internally they are, of course, technical systems, the external interfaces should be designed in such a way that business people can understand them. Externally, you should not see the technical details. The smart consequence of this approach is that at this level of abstraction, platform-specific details don't matter. Thus, platforms can be heterogeneous.

Besides this broad definition, it is not clear what exactly a service is or can be. There are very different opinions about the exact attributes a service must, can, or should have. I will discuss services in detail in the next chapter (presenting several different existing definitions for services as well). As a rule of thumb, however, you can consider a service to be the IT representation of self-contained business functionality, such as "create a customer," "get a customer's contracts," "transfer money," "turn on the radio," "calculate the best route for my car," and so on.

With heterogeneous systems, the first goal is to be able to connect those systems easily. This is usually called "high interoperability." High interoperability is not a new idea. Before SOA, we had the concept of enterprise application integration (EAI), and regarding interoperability, SOA is nothing new.

However, for SOA, high interoperability is the beginning, not the end. It is the base from which we start to implement business functionality (services) that is spread over multiple distributed systems.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

SOA Ingredients

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Having read that the key technical concepts for SOA are services, interoperability, and loose coupling, you might conclude that all you have to do to enable SOA is to introduce services, interoperability, and loose coupling.

But as I stated earlier, you can't buy SOA. What's important is that you introduce these concepts in the appropriate fashion. That is, you have to find the right degree of centralization, you have to set up the corresponding processes, and you have to do your homework. A lack of these "ingredients" is what I most often find as the problem in real SOA projects. To establish SOA successfully, you have to care for your infrastructure, architecture, and processes (including the metaprocess, governance).

Infrastructure is the technical part of SOA that enables high interoperability. The infrastructure of a SOA landscape is called an enterprise service bus (ESB). This term is taken from enterprise application integration, where it was called the EAI bus or just enterprise bus.

The key feature of the ESB is that it enables you to call services between heterogeneous systems. Its responsibilities include data transformation, (intelligent) routing, dealing with security and reliability, service management, monitoring, and logging.

will discuss the purpose and properties of an ESB in detail.

Architecture is necessary to restrict all the possibilities of SOA in such a way that it leads to a working, maintainable system. SOA concepts, SOA tools, and SOA standards leave room for specific decisions that you must make in order to avoid a mess. You have to classify different types of services, you have to decide on the amount of loose coupling, you have to restrict the data model of service interfaces, you have to define policies, rules, and patterns, you have to clarify roles and responsibilities, you have to decide on the infrastructure technology, you have to decide which (version of) standards to use, and so on.

In this book, I will very often state that you have to decide something according to your concrete situation (requirements and context). By making these decisions, you will build your system architecture (or system of systems of architecture).

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

SOA Is Not a Silver Bullet

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

SOA has been widely hyped, and is starting to become mainstream. With popular new concepts, there is always a danger that people will try to apply them everywhere. People like to try out cool new things (although, on the other hand, they fear change), and when everyone's saying that you have to use SOA, it's easy to fall into the trap of making everything SOA-like. (If you have a hammer, everything looks like a nail.)

As I wrote earlier, SOA is the ideal solution for very special circumstances: heterogeneous distributed systems with different owners. As you will see throughout this book, though, there's a price to pay for dealing with heterogeneity and different owners, and providing flexibility, scalability, and fault tolerance.

See [Erl05] pp. 54–55

Contemporary SOA represents an open, extensible, federated, composable architecture that promotes service-orientation and is comprised of autonomous, QoS-capable, vendor diverse, interoperable, discoverable, and potentially reusable services, implemented as Web services.

SOA can establish an abstraction of business logic and technology, resulting in a loose coupling between these domains.

SOA is an evolution of past platforms, preserving successful characteristics of traditional architectures, and bringing with it distinct principles that foster service-orientation in support of a service-oriented enterprise.

SOA is ideally standardized throughout an enterprise, but achieving this state requires a planned transition and the support of a still evolving technology set.

...

Contemporary SOA supports, fosters, or promotes: vendor diversity, intrinsic interoperability, discoverability, federation, inherent reusability, extensibility, service-oriented business modeling, layers of abstraction, enterprise-wide loose coupling, organizational agility.

For this reason, if you don't have the type of system I've just described, think about not using SOA. If you have everything under control (i.e., a homogenous system and/or no different owners), SOA might be pointlessly expensive for you. For example, SOA is not the right approach just to separate a frontend from a backend. Of course, SOA will help to introduce different horizontal layers, but there are other approaches (such as modules and libraries) that are probably more appropriate. However, even if you fall into this category, the rules, problems, and principles of system abstractions are the same. So, you can still benefit from different topics presented in this book, even if you decide not to use SOA.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

SOA Is Not a Specific Technology

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Many SOA definitions include the term Web Services, but SOA is not the same as Web Services. SOA is the paradigm; Web Services are one possible way to realize the infrastructure by using a specific implementation strategy. This is an important distinction.

Web Services are emerging as the de facto standard for SOA implementations. Most discussions about SOA more or less claim that it should be implemented with Web Services (for example, in his definition Thomas Erl writes: "Contemporary SOA represents an . . . architecture that . . . is comprised of . . . services, implemented as Web services"). The good news is that the evolution of such a standard will lead to more freedom of choice: instead of using proprietary technology or making everything by yourself, you will be able to buy existing solutions at reasonable prices (open source solutions also are available).

This does not mean that building SOA with Web Services will solve all your problems. Web Services might help provide the infrastructure, but you will still have to construct the architecture, and set up all the complicated processes that are necessary for successful SOA. In addition, the Web Services standard is not as mature as it seems, and Web Services introduce some problems other implementation strategies don't have. I will discuss these issues in detail in .

Note that it is possible to implement SOA with other technologies (CORBA, MQ, Tibco, etc.). I have seen SOA landscapes that use different implementation strategies (see ).

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

SOA Versus Distributed Objects

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

There have been many different approaches to dealing with distributed systems. One approach, the initial concept of CORBA, was to use distributed objects. The idea was to enable remote access to objects of external systems. You were able to call methods of objects, including setters and getters, remotely. That is, for each access to an attribute, you were calling a remote function.

This was a very fine-grained kind of interface to remote systems, and, as a consequence, the approach was the base for the idea of having one general business object model spanning distributed systems.

In practice, it turned out that this approach didn't scale. Having one business object model that was used by each connected system was hard to organize, and introduced too much centralization, and too many dependencies.

In fact, SOA is the exact opposite of the concept of distributed objects. With SOA, data is exchanged between different systems, and each system operates on its local (redundant) copy with its own local methods and procedures. Unlike with distributed objects, this approach decouples the systems and lets them scale.

Note that version 2.3 of CORBA introduced value objects, or Objects by Value (OBV), which allow you to copy chunks of data (objects) from one system and operate on them locally. You can implement SOA using this technology (provided you don't use OBV with operations). This way of using CORBA has nothing to do with the concept of distributed objects.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

SOA Terminology

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Let's talk a bit about terminology. As usual with evolving concepts, different terms are often used for the same things, and a given term can have different meanings. The Glossary at the end of the book contains definitions of the most important SOA terms. However, I would like to introduce here a few terms you'll see throughout this book. I'll present others later, when we get into the details.

To begin, we'll need terms for the roles systems have when they communicate via services. The terms I use in this book are "provider" and "consumer":

• A provider is a system that implements a service (a business functionality) so that other systems can call it.
• A consumer is a system that calls a service (uses a provided service).

Requestor is often used as another word for consumer (especially in the Web Services world). This is a more technical term based on the fact that a service consumer sends a request to the provider to call a service (although this is not always the case, as you'll see in ). I will use "consumer" throughout this book.

You might also hear and use the usual terms for roles in distributed systems, client and server. Indeed, a provider is the server of services, while the consumer is the client using them. However, the terms "client" and "server" are used in so many different contexts that I prefer to and recommend that you use "provider" and "consumer" instead.

If you want to use a generic term for both the provider and the consumer, you can use the term participant. A service participant is a system that is a service provider or a service consumer (or both). Again, there are alternatives in common usage. For example, in the Java? world, the term service agent has evolved.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Summary

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

• Here is my summarizing definition of SOA:

  SOA is an architectural paradigm for dealing with business processes distributed over a large landscape of existing and new heterogeneous systems that are under the control of different owners.

• The key technical concepts of SOA are services, interoperability, and loose coupling.
• The key ingredients of SOA are infrastructure, architecture, and processes (including the metaprocess of establishing SOA, governance).
• The key success factors for SOA are understanding, governance, management support, and homework.
• SOA is neither a specific technology nor a silver bullet. There are places where SOA is appropriate and places where it is not.
• Web Services are one possible way of realizing the infrastructure aspects of SOA. Using Web Services is often recommended because it seems to be becoming established as the standard technology.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Chapter 3: Services

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

IN THIS CHAPTER, WE'LL EXPLORE THE MEANING OF THE TERM "SERVICE" AND ITS ASSOCIATED concepts (such as interfaces and contracts).

As for SOA, there are multiple definitions of the term "service." For this reason, I will again quote definitions from various existing sources, such as the OASIS SOA Reference Model, Wikipedia.org, and some books. I will then present my personal definition of the term.

To get a complete picture of what services are, you'll have to read further in this book. The goal of this chapter is to provide a general definition that will serve as a base for later discussions in this book.

The OASIS SOA Reference Model [OasisSoaRM06] states:

The noun "service" is defined in dictionaries as "The performance of work (a function) by one for another."

This can mean everything or nothing. As with the term SOA, it is hard to find an exact, useful definition of the term "service" because so many definitions exist. Again, I have collected some of these definitions and put extracts of them in sidebars throughout the chapter. You'll find some common phrases and attributes as you read them, but you will also find a lot of differences in the context, level of abstraction, and wording. As in the previous chapter, I'll begin by presenting my understanding of services.

A "Service" is (ideally) a self-contained, stateless business function that accepts one or more requests and returns one or more responses through a well-defined, standard interface. Services can also perform discrete units of work such as editing and processing a transaction. Services should not depend on the state of other functions or processes. The technology used to provide the service, such as a programming language, does not form part of this definition.

SOA is focused on business processes. These processes are performed in different steps (also called activities or tasks) on different systems. The primary goal of a service is to represent a "natural" step of business functionality. That is, according to the domain for which it's provided, a service should represent a self-contained functionality that corresponds to a real-world business activity. In other words, business people should be able to understand what a service does (see the "" sidebar).

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Services

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The OASIS SOA Reference Model [OasisSoaRM06] states:

The noun "service" is defined in dictionaries as "The performance of work (a function) by one for another."

This can mean everything or nothing. As with the term SOA, it is hard to find an exact, useful definition of the term "service" because so many definitions exist. Again, I have collected some of these definitions and put extracts of them in sidebars throughout the chapter. You'll find some common phrases and attributes as you read them, but you will also find a lot of differences in the context, level of abstraction, and wording. As in the previous chapter, I'll begin by presenting my understanding of services.

A "Service" is (ideally) a self-contained, stateless business function that accepts one or more requests and returns one or more responses through a well-defined, standard interface. Services can also perform discrete units of work such as editing and processing a transaction. Services should not depend on the state of other functions or processes. The technology used to provide the service, such as a programming language, does not form part of this definition.

SOA is focused on business processes. These processes are performed in different steps (also called activities or tasks) on different systems. The primary goal of a service is to represent a "natural" step of business functionality. That is, according to the domain for which it's provided, a service should represent a self-contained functionality that corresponds to a real-world business activity. In other words, business people should be able to understand what a service does (see the "" sidebar).

Note that a business-driven interface is different from a technically driven business interface. For example, a technically driven business interface might be designed as follows:

customerOP(action,  // "create", "read", "change", "delete"
           id,      // customer id or null
           name,    // new customer name or null
           address, // new customer address or null
           account) // new customer bank account or null

It is a business interface because it allows you to create a new customer, read or modify his/her data, or delete that data. However, this interface is technically driven, probably by a direct mapping to a corresponding database interface.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Interfaces and Contracts

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Technically, a service is an interface for (multiple) messages that return information and/or change the state of an associated entity (backend). Or, as Yefim V. Natis, a Gartner analyst who co-coined the term SOA, wrote in [Gartner03]:

Essentially, SOA is a software architecture that starts with an interface definition and builds the entire application topology as a topology of interfaces, interface implementations and interface calls. SOA would be better-named "interface-oriented architecture."

According to this, it is difficult to make the definition of "service" more concrete. Everything that can be used as an interface, and represents a self-contained business functionality can be a service.

However, technically and semantically, there are different kinds of interfaces. One type of interface is a signature, which describes the input parameters, output parameters, and possible exceptions.

Simply knowing the signature is not enough to use a service, though. As a consumer of a service, you want to—indeed, must—know the complete behavior and semantics of the service. In other words, the interface should be well defined.

The behavior of a well-defined interface should be unambiguous, but in practice, this goal is often hard to reach. In addition, some aspects of a service might be consumer-specific. This especially applies to nonfunctional aspects, such as attributes related to Quality of Service (QoS) and service-level agreements (SLAs). For this reason, you might lay out the relevant information in contracts. A contract is the complete specification of a service between a specific provider and a specific consumer. From a consumer's point of view, it defines "everything you have to know when using this service," so that (ideally) no doubts remain.

In practice, you usually start to describe a service with a well-defined interface. Then, when a specific consumer wants to use the service, you define a specific contract based on the well-defined interface. The individual contracts will reflect the necessary resources you need to provide the service according to the specific nonfunctional commitments. For instance, while all consumers use (or can use) the same formal interface to call a service, consumers may make different numbers of calls at different times.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Additional Service Attributes

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

If you go through all the service definitions in this chapter (and other definitions outside this book), you will find references to a lot of additional attributes that services "may," "must," or "should" have. However, there is no common understanding about whether these additional attributes are really required for services. Here are some typical questions to which different people will give different answers:

• Do services have to be stateless?
• Do they have to be implemented as Web Services?
• Should they be coarse-grained and/or composable?

Care must be taken here. My definition of a service is that it should be an IT representation of a business functionality defined by a (well-defined) interface. Services should also typically be self-contained. Some other attributes usually apply, but I don't believe that any of these other attributes are fundamentally required.

In practice, things differ. In your concrete SOA, you may require some—but not all—of the additional service attributes discussed here, and you may find that some kinds of services have attributes that others do not. For this reason, we usually classify different kinds of services according to their attributes (see for details).

The details of the possible attributes services can have can be pretty complicated. The various attributes will be discussed throughout this book, according to the general topics of the chapters. However, to provide an initial overview, I'll introduce them here. Where appropriate, I will refer you to other places in the book where you can find further details.

All definitions of SOA agree that it is a design goal that services be self-contained (independent, autonomous, autarkic). However, some dependencies always exist. For example, services may share at least some fundamental data types (such as string). Be careful with more complicated fundamental data types, though. The goal is to minimize dependencies so that a SOA is appropriate for distributed systems with different owners. See for details.

Systems Architecture Service:

In the context of Enterprise Architecture (A.K.A. Systems or Organizational Architecture or Modeling) the term "Service" refers to a discretely defined set of contiguous and autonomous business or technical functionality.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Summary

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

• Here is my summarizing definition of "service" in the context of SOA:

  A service is the IT realization of some self-contained business functionality.

• By focusing on the business aspects, a service hides technical details and allows business people to deal with it.
• Technically, a service is an interface for (multiple) messages that are exchanged between provider(s) and consumer(s).
• The complete description of a service from a consumer's point of view (signature and semantics) is called a "well-defined interface" or "contract." A contract is agreed individually between a certain provider, and a certain consumer, and usually includes nonfunctional aspects such as SLAs.
• There are several attributes that services may (or, according to some definitions, must) have. According to my understanding, they are situation-dependent; services will almost always have different attributes, and should be classified according to those attributes.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Chapter 4: Loose Coupling

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

SOA APPLIES TO LARGE DISTRIBUTED SYSTEMS. SCALABILITY AND FAULT TOLERANCE ARE KEY TO THE maintainability of such systems. Another important goal is to minimize the impact of modifications and failures on the system landscape as a whole. Thus, loose coupling is a key concept of SOA.

This chapter will discuss the motivations for this concept (for large distributed systems), exploring variations of loose coupling from the technical and organizational points of view. This topic will demonstrate that SOA is a paradigm that leads to special priorities when designing large systems. However, again, there is no rule prescribing which kind or level of loose coupling you should employ. This decision must be made based on your specific circumstances.

We live in crazy times. The market rules, which means you won't usually have enough time to create well-elaborated, robust system designs. If you're not fast enough, flexible enough, and cheap enough, you'll soon find yourself out of the market. Thus, you need fast, flexible, and cheap solutions.

Fast and cheap solutions, however, can't be well designed and robust. Consequently, you will have to deal with errors and problems. The important point here is fault tolerance. The most important thing is that your systems run. According to [ITSecCity02], a flightbooking system failure may cost $100,000 an hour, a credit card system breakdown may cost $300,000 an hour, and a stock-trading malfunction may cost $8 million an hour. As these figures show, fault tolerance is key for large distributed systems. When problems occur, it is important to minimize their effects and consequences.

Loose coupling is the concept typically employed to deal with the requirements of scalability, flexibility, and fault tolerance. The aim of loose coupling is to minimize dependencies. When there are fewer dependencies, modifications to or faults in one system will have fewer consequences on other systems.

Loose coupling is a principle; it is neither a tool, nor a checklist. When you design your SOA, it is up to you to define which kinds and amount of loose coupling you introduce. However, there are some typical topics you might want to consider when you think about loose coupling in your system. lists them (this list is an extension of a list published in [KrafzigBankeSlama04], p. 47).

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

The Need for Fault Tolerance

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

We live in crazy times. The market rules, which means you won't usually have enough time to create well-elaborated, robust system designs. If you're not fast enough, flexible enough, and cheap enough, you'll soon find yourself out of the market. Thus, you need fast, flexible, and cheap solutions.

Fast and cheap solutions, however, can't be well designed and robust. Consequently, you will have to deal with errors and problems. The important point here is fault tolerance. The most important thing is that your systems run. According to [ITSecCity02], a flightbooking system failure may cost $100,000 an hour, a credit card system breakdown may cost $300,000 an hour, and a stock-trading malfunction may cost $8 million an hour. As these figures show, fault tolerance is key for large distributed systems. When problems occur, it is important to minimize their effects and consequences.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Forms of Loose Coupling

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Loose coupling is the concept typically employed to deal with the requirements of scalability, flexibility, and fault tolerance. The aim of loose coupling is to minimize dependencies. When there are fewer dependencies, modifications to or faults in one system will have fewer consequences on other systems.

Loose coupling is a principle; it is neither a tool, nor a checklist. When you design your SOA, it is up to you to define which kinds and amount of loose coupling you introduce. However, there are some typical topics you might want to consider when you think about loose coupling in your system. lists them (this list is an extension of a list published in [KrafzigBankeSlama04], p. 47).

Table : Possible forms of loose coupling in SOA

	Tight coupling	Loose coupling
Physical connections	Point-to-point	Via mediator
Communication style	Synchronous	Asynchronous
Data model	Common complex types	Simple common types only
Type system	Strong	Weak
Interaction pattern	Navigate through complex object trees	Data-centric, self-contained message
Control of process logic	Central control	Distributed control
Binding	Statically	Dynamically
Platform	Strong platform dependencies	Platform independent
Transactionality	2PC (two-phase commit)	Compensation
Deployment	Simultaneous	At different times
Versioning	Explicit upgrades	Implicit upgrades

This table is far from being complete, but it's pretty typical for large distributed systems. Note again that this is not a checklist. There is no SOA certification saying you conform when all or at least 50 percent of the forms of loose coupling are in use. However, it would be very strange if none of these forms of loose coupling were used in your SOA. If this were possible, your system would appear not to have the common requirements of large distributed systems with different owners. That's OK, but you shouldn't call your solution SOA. (Well, it may help you get money and resources for it, but beware of false impressions.)

If there is such a fine list of aspects of loose coupling, and minimizing dependencies is good, you might be wondering why you don't simply use all these forms of loose coupling in each SOA. The answer is that there is a price to pay for loose coupling, in that it makes systems more complex. That means more development, and/or maintenance effort.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Dealing with Loose Coupling

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The forms of loose coupling discussed in this chapter are only some (more or less typical) examples. Again, note that there are no hard and fast rules: you will have to decide on the appropriate amount of loose coupling for your specific context and architecture.

I have seen very different decisions made with regard to different types of coupling. As I mentioned earlier, the policy of one of my customers was to avoid asynchronous communication whenever possible, based on the experience that it led to race conditions at runtime that were very hard, or even impossible, to reproduce in a development environment, and therefore almost impossible to fix. Another customer in the same domain had a policy that synchronous calls were allowed only for reading service calls because the performance was not good enough for writing service calls.

Note that you might also have to decide about combinations of forms of loose coupling. For example, one important decision you'll have to make is whether an ESB should be separated from a backend via a protocol, or via an API (see ). Separating via an API usually means that the ESB provides libraries each backend or backend adapter has to use. So, deployment and binding become issues. On the other hand, using a common API, you can hide some aspects of synchronous or asynchronous communications inside the ESB.

You might ask which forms of loose coupling are typical. To my best knowledge, there is no answer. All I can say is that the larger systems are, the more loosely they should be coupled.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Summary

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

• Loose coupling is a fundamental concept of SOA (and large distributed systems in general) aimed at reducing dependencies between different systems.
• There are different forms of loose coupling, and you will have to find the mixture of tight and loose coupling that's appropriate for your specific context and project.
• Any form of loose coupling has drawbacks. For this reason, loose coupling should never be an end in itself.
• The need to map data is usually a good property of large systems.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Chapter 5: The Enterprise Service Bus

Content preview·Buy PDF of this chapter|