How to implement competing consumer using http

Question

I really enjoyed the "Microservices: Java, The Unix Way" presentation by James Lewis.

In this presentation, James' talks about implementing a user registration service by writing a queue processing engine that reads events off an atom feed and exposes an endpoint that multiple consumers can read from to process and turn into user entities.

The notes on the specific slide (around 18:40 in the video) say that this was implemented using the competing consumer EIP:

"Queue Processing Engine implemented the competing consumer pattern using conditional GET, PUT and Etags against the atom collection exposed by the event queue"

This kind of queue (and the way they talk about having heterogeneous consumers) suggests that it is a publish-subscribe channel.

I don't really understand how this can be implemented, the EIP book says that competing consumers only works:

[...] with Point-to-Point Channels; multiple consumers on a Publish-Subscribe Channel just create more copies of each message

I assume the queue processor exposes a REST resource that the competing consumers call making GET requests for new items, but where do the the PUT requests and etags come into it?

Answer 1

A colleague spoke to Martin Fowler and James Lewis and to summarise a half remembered summary, they implied that you just don't have multiple consumers of the queue.

Just have one consuming service and have monitoring in place to ensure that you are alerted if it goes down.

Answer 2

The use of entity tags with the PUT method in this context is explained in RFC 5023, The Atom Publishing Protocol, Section 9.5:

After editing, the client can PUT the Entry and send the ETag entity value in an If-Match header, informing the server to accept the entry on the condition that the entity value sent still matches the server's.

   PUT /edit/first-post.atom HTTP/1.1
   Host: example.org
   Authorization: Basic ZGFmZnk6c2VjZXJldA==
   Content-Type: application/atom+xml;type=entry
   Content-Length: nnn
   If-Match: "e180ee84f0671b1"

   <?xml version="1.0" ?>
   <entry xmlns="http://www.w3.org/2005/Atom">
     <title>Atom-Powered Robots Run Amok</title>
     <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
     <updated>2007-02-24T16:34:06Z</updated>
     <author><name>Captain Lansing</name></author>
     <content>Update: it's a hoax!</content>
   </entry>

The server however has since received a more recent copy than the client's, and it responds with a status code of 412 ("Precondition Failed").

   HTTP/1.1 412 Precondition Failed
   Date: Sat, 24 Feb 2007 16:34:11 GMT

In other words, the client doesn't want to edit a resource if someone else has already done so, so it sends the entity tag in the If-Match header with the PUT request. The client is saying to the server, "Only accept my edit if no one else has edited this resource already."