Across all industries, executives are demanding more value from their strategic business processes. What process is strategic to a given company may vary dramatically by industry, but a common theme is that CEOs want their IT organizations to measurably improve the flow of data and information driving key business decisions. Whether it’s a financial services firm seeking a competitive advantage by guaranteeing a higher volume of faster foreign exchange trades, a retail chain looking to accelerate the flow of store data back to brand managers at corporate headquarters, or a building materials supplier striving to optimize order flow through a complex distribution chain, there are common and significant technical challenges to be overcome. Information is locked up in applications within different departments and organizations, and it is both time-consuming and costly to pry that data loose. In short, the enterprise is far from integrated.
The past several years have seen some significant technology trends, such as Service Oriented Architecture (SOA), Enterprise Application Integration (EAI), Business-to-Business (B2B), and web services. These technologies have attempted to address the challenges of improving the results and increasing the value of integrated business processes, and have garnered the widespread attention of IT leaders, vendors, and industry analysts. The Enterprise Service Bus (ESB) draws the best traits from these and other technology trends.
The ESB concept is a new approach to integration that can provide the underpinnings for a loosely coupled, highly distributed integration network that can scale beyond the limits of a hub-and-spoke EAI broker. An ESB is a standards-based integration platform that combines messaging, web services, data transformation, and intelligent routing to reliably connect and coordinate the interaction of significant numbers of diverse applications across extended enterprises with transactional integrity.
An extended enterprise represents an organization and its business partners, which are separated by both business boundaries and physical boundaries. In an extended enterprise, even the applications that are under the control of a single corporation may be separated by geographic dispersion, corporate firewalls, and interdepartmental security policies.
In an event-driven enterprise, business events that affect the normal course of a business process can occur in any order and at any time. Applications that exchange data in automated business processes need to communicate with each other using an event-driven SOA to have the agility to react to changing business requirements. An SOA provides a business analyst or integration architect with a broad abstract view of applications and integration components to be dealt with as high-level services. In an ESB, applications and event-driven services are tied together in an SOA in a loosely coupled fashion, which allows them to operate independently from one another while still providing value to a broader business function.
In the realm of SOA, events are represented in an open XML format and flow through a transparent pipeline that’s open to inspection and subject to intermediation...
— John Udell, InfoWorld
Service components in an SOA expose coarse-grained interfaces with the purpose of sharing data asynchronously between applications. Using an ESB, an integration architect pulls together applications and discrete integration components to create assemblies of services to form composite business processes, which in turn automate business functions in a real-time enterprise.
An ESB provides the implementation backbone for an SOA. That is, it provides a loosely coupled, event-driven SOA with a highly distributed universe of named routing destinations across a multiprotocol message bus. Applications (and integration components) in the ESB are abstractly decoupled from each other, and connect together through the bus as logical endpoints that are exposed as event-driven services.
With its distributed deployment infrastructure, an ESB can efficiently provide central configuration, deployment, and management of services that are distributed across the extended enterprise.
The common goal of applying technologies such as SOA, EAI, B2B, and web services is to create an architecture for integration that can be pervasive across an extended enterprise and beyond. For an integration infrastructure to achieve this pervasiveness, it must have the following characteristics:
It must adapt to suit the needs of general-purpose integration projects across a variety of integration situations, large and small. Adaptability includes providing a durable architecture that is capable of withstanding evolutions in protocols, interface technology, and even process modeling trends.
It must link together applications that span the extended enterprise using a single unified approach and a common infrastructure.
It must extend beyond the boundaries of a single corporate IT data center and into automating partner relationships such as B2B and supply-chain scenarios.
It must have simplicity of design and low barriers to entry, enabling the everyday IT professional to become a self-empowered integration architect.
It must provide an SOA across the pervasive integration that enables integration architects to have a broad, abstract view of corporate application assets and automated business processes.
It needs the flexibility and ability to react to and meet the needs of changing business requirements and competitive pressures.
Tip
In an ESB, applications and event-driven services are tied together in an SOA in a loosely coupled fashion. This allows them to operate independently from one another while still providing value to a broader business function.
The ESB architecture addresses these needs, and is capable of being adopted for any general-purpose integration project. It is also capable of scaling pervasively across enterprise applications, regardless of physical location and technology platform. Any application can plug into an ESB network using a number of connectivity options, and immediately participate in data sharing with other applications that are exposed across the bus as shared services. This is why the ESB is often referred to as an integration network or integration fabric.
An ESB provides a highly distributed approach to integration, with unique capabilities that allow individual departments or business units to build out their integration projects in incremental, digestible chunks. Using an ESB, departments and business units can continue to maintain their own local control and autonomy in individual integration projects, while still being able to connect each integration project into a larger, more global integration network or grid.
Web services have bestowed newfound importance on service-oriented architectures by providing a standards-based approach to interoperability between applications. The main objective of web services has been to provide a service abstraction that allows interoperability between applications built using disparate platforms and environments. The achievement of this goal will provide an easier path to pervasive integration between applications.
With the advent of the ESB there is now a way to incorporate web services and SOA into a meaningful architecture for integrating applications and services into a backbone that spans the extended enterprise in a large-scale fashion. An ESB makes web services, XML, and other integration technologies immediately useful with the mature technology that exists today.
The core tenets of SOA are vital to the success of a pervasive integration project, and are already implemented quite thoroughly in the ESB. The web services standards are trending in the right direction, but remain incomplete with respect to the enterprise-grade capabilities such as security, reliability, transaction management, and business process orchestration. The ESB is based on today’s established standards in these areas, and has real implementations that are already being deployed across a number of industries. The ESB is quite capable of keeping in step with the ongoing evolution of the web-services equivalents of these capabilities as they mature. Chapter 12 provides a more detailed discussion on this subject.
An ESB applies web services and other complementary standards by combining them with technology concepts and best practices learned from EAI brokers. However, an ESB is more than simply a web-services veneer on top of the same old EAI hub.
Traditional formalized approaches to integration have their pros and cons. Chapter 1 shows some of the high-level traits of integration approaches, which range from the least desirable on the lower left point of origin, to the most desirable on the upper right quadrant.
Traditional EAI brokers, which include those that are built upon application servers, use a hub-and-spoke architecture. A hub-and-spoke architecture has the benefit of centralized functions, such as management of routing logic and business rules, but does not scale well across departmental or business unit boundaries. Chapter 2 will examine the huge price of early attempts at integration using EAI hubs, as well as their moderate success.
Application servers can interoperate through standard protocols, yet they link things together in a tightly coupled fashion, and intertwine the integration logic and application logic together.
EAI brokers provide increased value by separating the application logic from the integration and process routing logic, yet still suffer from the hub-and-spoke architecture.
Message Oriented Middleware (MOM) provides the ability to connect applications in a loosely coupled, asynchronous fashion. However, MOM by itself requires low-level coding in an application. Using a traditional MOM, along with custom coding techniques, can get you a long way toward a distributed integration solution. However, without a higher level of abstraction of the routing logic, this approach also suffers from having integration logic hard-wired and intertwined with the application logic. Depending on the MOM being used, even the distributed characteristic might be limited because some traditional MOM infrastructure is not capable of spanning physical network boundaries very well.
Finally, in an ESB, services can be configured rather than coded. Process flow and service invocations can transparently span the entire distributed bus. An ESB provides a highly distributed integration environment that spans well beyond the reach of hub-and-spoke architectures, and a clear separation of business logic and integration logic such as routing and data transformation. An ESB architecture forms an interconnected grid of messaging hubs and integration services, with the intelligence and functionality of the integration network distributed throughout.
Chapter 6 further describes the contrast between integration using an application server architecture and integration using an ESB. MOM concepts are discussed in Chapter 5. “The Accidental Architecture” in Chapter 2 continues to discuss the separation of business process routing and business logic.
A key characteristic of an ESB is to provide the underpinnings to support the needs of distributed, loosely coupled business units and business partners automating supply chains. These capabilities in an ESB were born out of necessity, as a result of middleware vendors working with industry professionals who were trying to create an architecture for large-scale integration. These industry professionals included IT architects in large corporations, and innovators in the e-Marketplace trading hub community who needed to build a B2B trading exchange backbone based on shared services, messaging, XML, and numerous connectivity options, while adhering to industry standards for each component. Chapter 3 will discuss the many catalysts that contributed to the creation of the ESB concept.
At the same time, the biggest needs that had yet to be addressed included how to effectively provide integration capabilities such as application adapters, data transformation, and intelligent routing in a way that could be used for general-purpose integration projects across a variety of integration situations. Also required was a more universal technology and an architectural approach that could be used to connect applications beyond the needs of individual tactical integration projects.
IT professionals had been disappointed by some previous technology trends such as CORBA and EAI. CORBA had the right idea with SOA, but turned out to be too complex to implement and maintain due to its reliance on tightly coupled interfaces between applications and services. EAI also suffered from steep learning curves and expensive barriers to entry on individual projects (more on this in the next chapter). What was really needed was a simple approach to SOA, with an architecture that could be adapted to suit the general needs of any integration effort, large or small. In addition, there needed to be a durable architecture that was capable of withstanding evolutions in protocols, interface technology, and even process modeling trends. The ESB concept was created to address all these needs.
Since ESB was first introduced in 2002, the ESB approach to integration has been adopted by numerous significant vendors in the middleware, integration, and web-services markets. Its acceptance continues to grow steadily.
Analyst firms such as Gartner Inc., IDC, and ZapThink have been tracking and writing about the ESB technology trend since early 2002. In a report issued in 2002 (DF-18-7304), Gartner Inc. analyst Roy Schulte wrote the following:
A new form of enterprise service bus (ESB) infrastructure—combining message-oriented middleware, web services, transformation and routing intelligence—will be running in the majority of enterprises by 2005.
These high-function, low-cost ESBs are well suited to be the backbone for service-oriented architectures and the enterprise nervous system.
Those four pillars—MOM, web services, transformation, and routing intelligence—represent the foundation of any good ESB. This book will focus on the role of each piece and many other required components as we explore the ESB. We will examine what the ESB can do for an enterprise, and the role that each basic component plays. We will also explore some advanced topics, including architectural overviews of practical uses across a number of industries.
A number of middleware and integration vendors have either built, or are in the process of building, something that matches the description of the ESB. And the list is growing all the time. The Appendix lists all the known vendors. Some vendors say they are providing an ESB already; some have announced plans to create one; some are just using the terminology in marketing materials but don’t really have anything substantial behind it. This technology category is destined to become as hot as application servers were in the late ’90s, when more than 25 vendors were competing for the same technology space.
A few vendors in this list deserve special mention. Sonic Software pioneered the concept, and shortly thereafter a number of other smaller vendors got on board, saying they also were providing an ESB or were in the process of building one. Once the incumbent integration companies such as webMethods, SeeBeyond, and IBM finally got “on the Bus” and announced their intent to build an ESB, the ESB term really began to get widespread notice as a growing technology category with staying power.
At the time of this writing, Microsoft has not publicly made any statements regarding its Indigo project and the term ESB. However, some journalists and analysts made that connection when Indigo was first announced. A November 30, 2003 ComputerWorld article, “Developer interest piqued by Microsoft technologies,” quoted Roy Schulte of Gartner Inc. regarding Indigo:
Roy Schulte, an analyst at Gartner Inc. in Stamford, Conn., noted that Indigo is a superset of Microsoft’s Messaging Queuing (MS MQ) technology, as well as the company’s Component Object Model (COM), COM+, .Net remoting and Web services support. “Think of this as a simplification, a unification of communication middleware on behalf of Microsoft’s plan,” he said, adding that he sees Indigo as a very good enterprise service bus (ESB).
Indigo is based on messaging and has the notion of combining MSMQ and web services. That could provide the basis for a messaging bus. However, the rest of its integration capabilities are locked into BizTalk, which is a hub-and-spoke integration server. To qualify as an ESB, both a distributed message bus and distributed integration capabilities need to exist.
If nothing else, the completion of Indigo will make applications and services built on the Microsoft platform even more attractive as endpoints to connect into an ESB. The inclusion of Indigo into the Microsoft platform and development environment will facilitate making applications capable of being loosely coupled and messaging-aware.
Due to the fast flurry of vendors trying to gain attention in the growing ESB product category, combined with the number of industry analysts and journalists reporting with their opinions on it, understandably there is much confusion as to what an ESB actually is. This section serves to outline the main characteristics of an ESB.
As illustrated in Chapter 1, the ESB can form the core of a pervasive grid. It is capable of spanning an extended enterprise and beyond, having a global reach across departmental organizations, business units, and trading partners. An ESB is also well suited for localized integration projects, and provides flexible underpinnings enabling it to adapt to any type of integration environment.
Applications plug into the bus as needed, and are capable of having visibility and of sharing data with any other applications or services that are plugged into the bus. While web-services interfaces are an integral part of an ESB architecture, all applications do not have to be modified to become true web services to participate in the ESB. Connectivity is achieved through multiple protocols, client API technologies, legacy messaging environments, and third-party application adapters.
Standards-based integration is a fundamental concept of an ESB. For connectivity, an ESB can utilize J2EE components such as the Java Message Service (JMS) for MOM connectivity, and J2EE Connector Architecture (JCA or J2CA) for connecting to application adapters. An ESB also integrates nicely with applications built with .NET, COM, C#, and C/C++. In addition, an ESB can easily integrate with anything that supports SOAP and web-services APIs, which includes de facto standard web-services toolkit implementations such as Apache Axis. For dealing with data manipulation, an ESB can use XML standards such as XSLT, XPath, and XQuery to provide data transformation, intelligent routing, and querying of “inflight” data as it flows through the bus. For dealing with SOA and business process routing, an ESB can use the Web Services Description Language (WSDL) to describe abstract service interfaces, and Business Process Execution Language for Web Services (BPEL4WS), WS-Choreography, or some other XML-based vocabulary such as ebXML BPSS, to describe abstract business processes.
Don’t worry if you don’t know what all these buzzwords mean. Though this book is not intended to be an exhaustive reference or tutorial on all these individual technologies, they will be explained in sufficient detail in the context of how they relate to an ESB.
These standards-based interfaces and components are put together in a meaningful way that comprises an open-ended pluggable architecture. An ESB provides an infrastructure that supports both industry-standard integration components as well as proprietary elements through the use of standardized interfaces. Chapter 1 shows a simplified view of an ESB that is integrating a J2EE application using JMS and JCA, a third-party packaged application using a JCA-compliant application adapter, a .NET application using a C# client, and two external applications using web services.
The ESB draws from traditional EAI broker functionality in that it provides integration services such as business process orchestration and routing of data, data transformation, and adapters to applications. However, integration brokers are usually highly centralized and monolithic in nature. The ESB provides these integration capabilities as individual services that can work together in a highly distributed fashion, and can be scaled independently from one another. In Chapter 6 you will learn more about the ESB “service container,” a core concept of the ESB, which allows the selective deployment of integration services.
A key part of any integration strategy is the ability to readily convert data formats between applications. Many applications do not share the same format for describing similar data.
Data transformation is inherently part of the bus in an ESB deployment. Transformation services that are specialized for the needs of individual applications plugged into the bus can be located anywhere and accessible from anywhere on the bus. Because the data transformation is such an integral part of the ESB itself, an ESB can be thought of as solving the impedance mismatch between applications.
An ESB gives you all the required core capabilities for virtually any integration project, and can be augmented with layered technology to handle more specific uses. For example, specialized capabilities such as Business Process Management (BPM) software can process workflow-related business processes, and collaboration servers can provide specialized services for managing business partners. Specialized third-party translators can provide data conversion from external data formats such as EDI into a target Enterprise Resource Planning (ERP) system or onto the general bus as an internal canonical XML representation.
In an ESB-enabled, event-driven SOA, applications and services are treated as abstract service endpoints, which can readily respond to asynchronous events. The SOA provides an abstraction away from the details of the underlying connectivity and plumbing. The implementations of the services do not need to understand protocols. Services do not need to understand how messages are routed to other services. They simply receive a message from the ESB as an event, and process that message. The ESB gets the message to anywhere else it needs to go.
In an ESB SOA, custom integration services may be created, extended, and reused as ESB functionality. Application endpoints, which are exposed as services, can be constructed together with specialized integration enablers to form composite services and process flows that can be recombined and reused for various purposes, with the goal of automating business functions in a real-time enterprise.
Chapter 7 will discuss SOA in the ESB in more detail.
An ESB’s process flow capabilities range from simple sequences of finite steps to sophisticated business process orchestration with parallel process execution paths using conditional splits and joins. These can be controlled by simple message metadata or through the use of an orchestration language such as BPEL4WS.
The process flow capabilities of the ESB make it possible to define business processes that are local to an individual department or business unit, and that can also coexist within a larger integration network. This is something a hub-and-spoke integration broker or a BPM tool can’t do very well on its own. Chapter 7 will examine the details of a distributed processing capability that provides highly distributed business process orchestration without the need for a centralized processing or rules engine.
Process flow in an ESB can also involve specialized integration services that perform intelligent routing of messages based on content.
Because the process flow is built on top of the distributed SOA, it is also capable of spanning highly distributed deployment topologies (even spanning oceans at times) without the need to be painfully aware of the physical network boundaries or multiple protocol hops between applications and services on the bus (Chapter 1).
The connections between nodes on the ESB are firewall-capable. The security between applications and the ESB, and even between the ESB nodes themselves, is capable of establishing and maintaining the most stringent authentication, credential-management, and access control.
Reliability is achieved by having an enterprise-capable MOM at the core of the ESB. The MOM core provides asynchronous communications, reliable delivery of business data, and transactional integrity. As you will learn in Chapter 5, this is not your traditional MOM technology from a decade ago. Requirements have evolved and matured since then, and a MOM core of an ESB must be capable of meeting today’s requirements.
Traditional hub-and-spoke EAI broker approaches tend to have organizational boundary problems, which are sometimes caused by the physical limitations of the EAI broker’s incapability of easily spanning firewalls and network domains. More importantly, even if a hub-and-spoke architecture is capable of being stretched out across organizational boundaries, it still does not allow the local autonomy that individual business units need to operate semi-independently of one another. One of the biggest problems associated with extending the reach of integration beyond the departmental level is the issue of local autonomy versus centralized control.
As part of the business culture in most large corporate environments, each department or business unit needs to operate independently of one another. However, they still rely on shared resources, and reporting and accounting information that funnels into a common business function.
In such an environment, it is not reasonable to impose an integration strategy that requires all message traffic to flow through a centralized message broker sitting in headquarters. This is not simply a technical obstacle; it is a corporate culture issue as well. In an environment of loosely coupled business units, it does not make sense for things such as business process flow between localized applications, or security domains, to be managed by a single centralized corporate IT function. Loosely coupled business units within an organization need to operate independently of one other. Each should have its own IT function and not have to think in terms of routing all its message traffic, or delegating control of its business rules and security domains, through a centralized integration broker at one location or the other (Chapter 1).
Figure 1-5. Separate business units lack the necessary autonomy if using a centralized hub-and-spoke integration broker
Local business units and departments need to have control over their own local IT resources, such as the applications running at their site. The integration infrastructure should support deployment topologies to support that business model with practicality. The ESB provides this deployment model, allowing for local message traffic, integration components, and adapters to be locally installed, configured, secured, and managed, while still being able to plug together the local integration domains into a larger federated integration network with an integrated security model (Chapter 1).
Figure 1-6. Autonomous and federated, an ESB allows organizations to cooperatively federate operations across organizational boundaries
The distributed characteristics of the ESB are achieved by abstraction of endpoint definitions from the physical deployment details and underlying wire protocols, along with orchestration and routing of data between those endpoints. The federated characteristics are achieved by the ability of the ESB to segregate and selectively traverse application domains and security boundaries.
In some business models it doesn’t make sense to have local IT staff at each remote location, although there is still a need for a loosely coupled, autonomous yet federated integration network. To illustrate this point, let’s examine a simple case study for an ESB deployment in the retail industry. A video rental chain can have hundreds or thousands of remote locations that all contain the same set of applications, and all operate in the same fashion with regard to inventory management, accounting, and reporting (Chapter 1).
Figure 1-7. A video retail chain with thousands of remote stores, all containing the same set of applications
Using an ESB, an integration blueprint can be established to handle the local communication between the applications at a remote store. This includes the interface definitions to in-store applications, the routing of message traffic and message channel management, and security permissions. It may also include integration components such as an application adapter, protocol adapter, or data transformation. This integration blueprint, or template, can be deployed and customized at each site and act independently of all the other stores (Chapter 1).
Figure 1-8. ESB configuration blueprint deployed at each remote location and remotely configured and managed
This remote deployment blueprint capability is not unique to retail—it has applications across other industries as well. The unified remote management of federated integration realms is a key contributor to the success of any ESB deployment in a highly distributed environment.
XML is an ideal foundation for representing data as it flows between applications across the ESB. The data that is produced and consumed by a vast array of applications can exist in a variety of formats and packaging schemes. While it is certainly possible for the ESB to carry data using any form of packaging or enveloping scheme that you like, there are tremendous benefits to representing in-flight data as XML, including the ability to use specialized ESB services that combine data from different sources to create new views of data, and to enrich and retarget messages for advanced data sharing between applications. Chapter 4 will explore a fundamental benefit of XML—the ability to insulate an organization from the need to synchronize upgrades between applications—and will discuss the philosophy behind distributed XML transformation services in more detail.
An ESB eliminates latency problems by providing real-time throughput into in-flight data as it travels between applications across the ESB. Currently, one of the most popular integration methods is nightly batch processing. However, bulk batch-processing integration strategies, nightly or otherwise, are prone to high margins for error and can cause delays in information retrieval. The resulting high latency in getting up-to-date information can be costly. Chapter 9 discusses the details of this, and explores how an ESB can be used to refactor your organization from nightly batch processing toward real-time throughput of business data.
Operational awareness refers to the ability of a business analyst to gain insight into the state and health of business operations. An infrastructure that allows the timely tracking and reporting of data as it flows across an organization in the form of business messages in a business process is an invaluable tool in helping to achieve operational awareness. A separate category of products known as Business Activity Monitoring (BAM) has emerged to address the many issues of operational awareness.
One of the benefits of using XML as the native data format for the ESB is that messages are not treated as opaque chunks of data. If all data between applications and services is formatted as XML documents, underpinnings are provided by the ESB that allow you to layer advanced capabilities on top of the ESB to gain real-time insight into the business data that flows through your enterprise. These capabilities, whether they are inherently part of the ESB or are enabled as an extension of it, represent a natural part of the common infrastructure that includes the routing, process flow, and underlying plumbing, and that don’t require a separate third-party BAM product be bolted onto the side.
Auditing and tracking capabilities that are a base part of an ESB allow you to monitor and track the health of your business processes and the flow of messages throughout your SOA. Value-added services such as data caching, data collection and aggregation, and visual rendition of XML data can create a substrate for generating operational alerts, notifications, and reporting that can provide real-time insight into the state of your business as the data traverses your enterprise (Chapter 1).
Figure 1-9. Value-added services enabling operational awareness provide real-time insight into live business data
Tracking and reporting of data in the ESB is made possible by defining auditing and tracking points within a process flow, which in turn provide insertion points for collecting important content from business messages as they travel through a business process. Examples of trackable data are the business messages themselves, and the process events indicating whether a message has passed through a particular set of business process steps.
Advanced value-added services can provide the data collection services, the query mechanisms, and the reporting capabilities that enable all this data to be gathered and presented in a meaningful way. XML persistence services can provide caching and aggregation points that can collect data to be transformed for the purpose of providing data to feed into other applications, or to feed into human-readable reporting mechanisms that can be used by business analysts. This means that data flowing through the ESB can be analyzed in real time to produce live information about the nature of your business—for example, to provide a realistic snapshot of where your inventory is at all times within a supply chain.
One of the primary differentiating qualities of an ESB is its ability to allow incremental adoption, as opposed to being an all-or-nothing proposition. In the post-Y2K spending slowdown, budgets for multimillion-dollar IT projects are just not what they used to be. There is some indication that IT funding is getting freed up to solve the short-term tactical integration needs, but budgets are still being highly scrutinized at an executive level. At the same time, however, there is still a desire to implement larger corporate-wide strategic initiatives—all of which rely heavily on integration and reuse of existing IT assets.
Chapter 1 illustrates how an ESB can be used for small projects that each build into a much larger integration network. We’ll see how this is accomplished as we delve into the details throughout this book.
The federated/autonomous capabilities of the ESB also contribute to the ability to adopt an ESB one project at a time. Incrementally staged deployments of ESB integration projects can provide immediate value while working toward the broader corporate initiatives.
This notion of incremental adoption is also further supported by the ability to bridge into an existing integration broker hub and legacy message brokers. Integration broker hubs and their traits are explored in more detail in Chapter 2.
Many nascent technologies suffer from the issue of gaining adoption by trying to find a problem to solve. ESB concepts, on the other hand, have evolved out of necessity via industry-leading architects working with vendors in the technology community to define and build it, so ESB has been adopted as it has been built. ESBs are already being put to use in a variety of industries, including financial services, insurance, manufacturing, retail, telecom, energy, food distribution, and government. Here are some examples.
A leading subprime lender implemented an ESB to reduce application processing costs by 60%. This was accomplished by creating a unified view of customer and lending data across an eCredit system, third-party credit bureaus, and their backend systems.
Leading banks have implemented Straight Through Processing of financial transactions using an ESB, at a considerable saving over manual processing.
A Derivatives Trading system relies on an ESB to process more than 100,000 transactions a day for 1,200 users, accounting for several billion dollars in revenue.
The world’s largest life and health reinsurer, with $20 billion in annual revenue, generated significant savings using an ESB as a business process management solution to streamline the exchange of back-office transactional information between the main headquarters and the insurance brokers who market and manage their policies.
A manufacturer of countertops and flooring is using an ESB to improve supply chain predictability and reduce out-of-stock conditions by implementing a co-managed inventory system and “availability to promise” (ATP) query system. In phase 1 of the deployment, the ESB is being used to link the manufacturer and 60 of its distributors in a supply-chain network.
The deployment model of the ESB allows the manufacturer to deploy ESB service containers at the distributor sites. This is an alternative to deploying an integration broker at each remote distributor.
A major manufacturer of lighting, televisions, and medical imaging equipment is using an ESB to create a unified integration backbone to connect all its data centers across its global business units, and to create a unified view of product data and billing information to customers worldwide.
Using a standards-based, centralized management framework, a national retail video chain is in the process of adopting an ESB infrastructure to dynamically configure and manage 1,800 remote stores from a central management and configuration console.
The world’s largest mail-order company ($12 billion in revenue) relies on an ESB to order products from its many suppliers.
A web portal at a major phone carrier relies on an ESB to provide real-time analytics on click-through tracking (two-hour response versus 30-day response), processing 16 million messages per day.
The second-largest U.S. telecom carrier provider, a $43 billion company, uses an ESB to provide information from internal systems to competitive carriers.
A major European food distribution network (a £1.2 billion division) implemented an ESB in eight weeks and saved $3 million using a centralized hub-and-spoke integration broker approach. The ESB automates the distribution network by managing the buying, selling, and logistical coordination of the supply chain that ranges from the distribution of meats and produce to the grain that feeds the livestock.
In this food distribution network, the ESB is integrating applications spanning three different operating companies and many third-party trading partners, resulting in increased operational efficiency, significant cost savings, and an easier methodology to integrate new systems.
A U.S. government agency is working with an ESB to integrate multiple government systems to comply with the USA PATRIOT Act. The PATRIOT Act allows the government to track transactions for the purpose of preventing money from getting to terrorists. The project involves using the ESB as the integration between portal servers and various backend systems across multiple government agencies to provide a unified view of data.
In summary, an ESB has the following characteristics:
- Pervasiveness.
An ESB can be adapted to suit the needs of general-purpose integration projects across a variety of integration situations. It is capable of building out integration projects that can span an entire organization and its business partners.
- Highly distributed, event-driven SOA.
Loosely coupled integration components can be deployed on the bus across widely distributed geographic deployment topologies, yet are accessible as shared services from anywhere on the bus.
- Selective deployment of integration components.
Adapters, distributed data transformation services, and content-based routing services can be selectively deployed when and where they are needed, and can be independently scaled.
- Security and reliability.
All components that communicate through the bus can take advantage of reliable messaging, transactional integrity, and secure authenticated communications.
- Orchestration and process flow.
An ESB allows data to flow across any applications and services that are plugged into the bus, whether local or remote.
- Autonomous yet federated managed environment.
An ESB supports local autonomy at a departmental and business unit level, and is still able to integrate in a larger managed integration environment.
- Incremental adoption. An ESB can be used for small projects.
Each individual project can build into a much larger integration network, which can be remotely managed from anywhere on the bus.
- XML support.
An ESB can take advantage of XML as its “native” datatype.
- Real-time insight.
An ESB provides the underpinnings to enable real-time insight into live business data. BAM enablement is built right into the ESB fabric.
You should now have enough information about ESBs to whet your appetite. In the later, more detailed chapters, you will learn more about the underlying technical aspects. The next few chapters will discuss the evolution of the ESB, its current state of integration, the benefits of adopting XML in a generic data exchange architecture to mediate between diverse data representations, and asynchronous messaging and MOM.
Get Enterprise Service Bus now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
