What is an ESB and what is it good for?

Question

At a previous job, there was lots of talk about "Enterprise Service Bus" (ESB). I read parts of a conceptual book about it, but never really understood how you would implement/integrate it in concrete terms. I'm familiar with SOA/queueing/directory services/etc. but I don't understand what exactly an ESB is.

Is it a concrete thing (service/server/broker/etc.) that you just hook all your apps up to it in different ways, or is it more just a conceptual way to design systems?

Any explanations or links to good examples would be greatly appreciated. Thanks.

Answer 1

It's a fairly high level concept of abstraction. The central concept is that the ESB provides the middleware and interfaces that allow businesses to connect their applications without writing code.

This could include mediation to reconcile incompatible protocols, data, and interaction.

The idea of a central bus on which everything passes gives opportunity for additional layers of abstraction. Using industry standards to "plug" other applications, clients, and such into this bus makes it so that connecting new services, data sources, clients with disparate needs is relatively easy.

Actual implementations

As far as actual implementations, that's the domain of very large enterprise support businesses. While it's very buzzwordy, the goal is an ideal that on some small level can be understood through comparison with the internet:

Similarity to Internet

One big communications bus with widely different uses and data, but all running standardize protocols.

One can, in fact, write an HTTP to FTP connector that would allow browsers to access FTP sites without invoking an FTP client (usually built into the browser now).

Mashups

Mashups demonstrate an interesting implementation - take some bus route data from the San Francisco authority, map from google, and sushi bar locations from yahoo with ratings and run a simple query that gives you the closest sushi bar, weighting it so that you'd be willing to travel a little further for a better bar.

All completely different services, incompatible by themselves, but using standard connectors (yahoo pipes, for instance) they can be pulled together into a cohesive and useful whole.

-Adam

Answer 2

Disclaimer: I work for IBM and consult on WebSphere ESB, an IBM product designed to build ESBs with. The following are my opinions and don't necessarily reflect IBM's position.

An ESB is different things to different people, unfortunately.

To me, an ESB is a any technology that you can insert into an SOA (Service-Oriented Architecture), allowing you to connect disparate systems together. It often performs the functions of protocol transformation, message modification, routing, logging, acting as a security gateway, and so on. For example, you might use an ESB to expose a service previously only available as a Web Service as a JMS-based service as well.

In this respect, ESB implementations (or to be more precise, software sold to build ESBs with - such as that I consult on) are often technologically similar to what used to be known as a messaging or queueing broker, although the purpose is somewhat different, because (as the acronyms imply) it's oriented around services rather than moving messages from one place to another. How important the distinction is technologically is a matter of opinion.

Answer 3

My experience with commercial ESB is it's an overblown and expensive technology that creates as many problems as it solves. The ESB will link new systems and legacy, messages will fly over the bus and everything will be able to talk to everything else seamlessly. Throw in some resilience, orchestration and you have a very powerful piece of enterprise application software.

The problem comes when you try to use them for real, the overhead of writing for the bus, creating the message structures and so on, can outweight the benefits. As a high cost item, the ESB is seen as the panacea for all tech issues which it isn't, too much time is spent on the bus and not on the applications/data being connected. It's often the case that multiple competing standards will fight for supremacy in the same organisation leading to the classic tech dominated silos that these systems should actually be fixing.

IMHO it is far better to use create a small number of specific interfaces, typically using web services between only those system that need it.

Answer 4

It's basically a conceptual way to design a system - software companies try to sell you more by sticking on the 'ESB' sticker on it and managers like because an ESB looks good from a 'higher-level'.

An ESB is basically a MOM (message oriented middleware) with an added data model and structure definition management. You have a common data definition for all applications and adapters on that bus (could be XML with a shared XSD). Anything that connects MUST send it's information adhering to this data definition. The ESB supports loading, sharing and versioning of this common data definition. When connecting a new component to an ESB, you can expect more 'compatibility' out of the box than when connecting it to a MOM. Each component on that bus is conceptually treated as a 'resource' - so there's an additional abstraction introduced to decouple sender from receiver.

Example: let's say you want to connect application A with application B in a standard message oriented middleware, let's take JMS. You talk to your colleagues working on application B, agree on a topic, message type and fields and send it (pseudo code): sendJms("TRADE.MSFT", {MapMessage trader="pete" price=101.4 vol=100})

If you do the same thing in a service oriented architecture, you need to

1. install additinal software
2. learn a lot of new concepts
3. define your new Java component in the ESB's admin gui
4. implement some interfaces that are controlled by the ESB
5. sendJms( getDestination(), {MapMessage trader="pete" price=101.4 vol=100}) - note the destination is injected from the ESB

The first time it's probably a bit painful, but I guess you can get used to it, just like you can get used to EJB's ;-)

You could say a MOM system is 'untyped' (dynamic structure) while an ESB is 'typed' (static structure). The trade-offs of raw Messaging vs. ESB are similar to other untyped/typed choices:

• REST vs. SOAP
• unvalidated XML vs. XML validated with an XSD
• Groovy vs. Java
• interpreted language vs. compiled language

For smaller projects it's nice to hash out functionality quickly (e.g. Groovy code) but for larger projects it's good to have a debugger (e.g. Java), to be warned in advance when things break and to have a standard for people before they commit to the project.

So if your project suffers from too many people breaking the system by checking in unvalidated changes - move towards more structure (ESB instead of MOM). If your projects suffers from not getting enough things done in time - choose the simpler, untyped solution. If it's both - get a consultant (just kidding ;-)

Answer 5

Well, that depends on who you ask... Many would say it's a piece of "Middleware" that connects various pieces of "business logic" together to take the coding out of messaging between these modules. I think that's a general enough definition, but I'm sure you've already gotten there via Wikipedia and whatnot.

Those who would have the great ESB save the world see it as it's most commonly presented, a hub for doing everything. Most ESB implementations effort to encapsulate all repetitive tasks we see in business software. That means most ESBs take care of data transfer, security, logging, protocol translation, event systems, api exposure via web services, etc.

I think that's as clear as I can be... Hope that helps.

Answer 6

Take a look at my presentation "Spoilt for Choice - How to choose the right ESB".

I explain when to use an ESB, Integration Suite, or just an Integration Framework (such as Apache Camel). I also discuss the differences between open source and proprietary ESBs.

Answer 7

Beyond a the standard definition you can get from Wikipedia. I find it to be a great tool for connecting a bunch of legacy systems across multiple platforms and technologies. It's also a good tools for building distributed workflows and state management systems( such as general ledger).

However, it's rather expensive, complex and inconvenient to maintain and extend which makes it a poor technology choice as a general purpose tool for scaling you applications.