Is a standard implementation of an ESB on one server, or a cluster of servers?

Question

I already used SOAP, but I never worked with SOA, ESB and other enterprise application integration patterns. And I find documentation about ESBs quite confusing.

There is something I am not sure to understand with an ESB. I know it is rather a concept describing the network of an enterprise as a bus and not really a concrete thing, but still.

I can understand that an ESB provides protocol and message conversions with legacy services, allows orchestration and that the logic of the message destination is done by the ESB. But I also thought of the ESB as a middleware between different ESB servers (and not only with web service interfaces).

If I take the example of ServiceMix, I thought that it would be natural to have several ServiceMix platforms on different servers which interact through a common bus/protocol (NMR? JMS?). Therefore my service on ServiceMix (a) created with CAMEL (by using some web services for instance) could consume a service on ServiceMix (b) also created with CAMEL.

So if my service need another service, I would just mention its identifier, and the ESB would route my request to the correct ServiceMix platform.

But when I read about the examples of ServiceMix, it seems to me that ServiceMix is prefentially used as a standalone application server. Not a cluster of servers.

Is an ESB simply a boosted application server? (let apart the integration capabilities it provides)

Is there actually multiple ESBs in an SOA? Are they linked with a protocol internal to the ESB? Or do the services implemented on an ESB (a) must provide external interfaces like SOAP so an ESB (b) can use its services?

Answer 1

ESB is often used as a marketing concept where you draw it as a bar in the middle where everything can be simply plugged in. This vastly oversimplifies Enterprise Integration as inside the ESB the same mess can occur that you had outside before.

Often ESBs are implemented as an application server with adapters and a routing core. My personal opinion is that it is not a good architectural choice to do integration with one app server. The reason is that you leak too much information out of the system to be integrated this way. The central platform will know inner details of the app it integrates with like DB structure or proprietary connection protocols.

On the other hand the idea of a platform that can care about routing and transformations is a good concept that can help a lot if done correctly.

I think a better architecture is the concept of having a common transport and format in the SOA and create services with models that lean to the business language and thinking. To make this happen you need to have the adapters near the systems you integrate. This leads to a decentralized ESB that looks more like what you describe. Keep in mind though to make the common transport and dataformat very standardized.

I have good experiences with using a messaging system as the central transport and pure XML or SOAP as data format. Another good option is REST. It is also possible to combine both.

Servicemix or Karaf + Camel + CXF can host adapters to the applications as well as routing and transformation logic. Additionally you can embedd adapters into your applications using camel and CXF. JMS or REST will allow to connect containers and external adapters.

Answer 2

Simply put, Enterprise Service Bus is more of an architectural concept. The idea is to realise integrations in terms of business services. An ESB can actually be implemented as a set of policies and guidelines where applications agree upon a given set of formats and a given communication protocol (SOAP/WS, Messaging, etc), a way to describe services etc.

ESB servers (Service Mix, IBM WebSphere Message Broker, JBoss ESB, Mule ESB, MS BizTalk Server, you name it) essentially provides a toolbox for realising this. The number of servers, the topology, what protocols/data formats to use etc. often needs to be decided for each situation and enterprise environment. An application environment built up of batch controlled legacy/main frame systems needs a very different integration setup then for instance an event driven Java EE or .NET stack of applications.

Answer 3

Old topic - but I think you need to build an ESB around a Message Bus system.

ESBs may provide other ways of access as well like HTTP, FTP etc. ESBs inherently can host applications or let you connect seamlessly to another external application, agnostic of protocol as it does the mediation internally. You can scale it as much as you want. e.g. you can have the same app deployed in 3 instances of the ESB and load balance on the Message oriented middleware - i.e. you send your message on a queue and all ESB instances are listening on it and only one picks it up on a Point to Point model. You can then scale further by having a cluster of the MoM and send message to any of the instances - both hosting the same queue.