Question

We had a discussion over this topic at the work not long ago.

We have the following situation:

  • A legacy database which is currently the base for the main operational application in our company.
  • A new content management system that should receive master data information from the mentioned database. That master data will always be owned by the legacy database, but it is needed in the CMS as a basis to construct its own data. You could think of it as categories or grouping data.
  • A trigger system in the legacy database that watches over certain tables. When an insert/update/delete occurs, it puts a notification into a database queue.
  • This database queue is watched by a (so called) loader service, which gets the data and puts it into a Kafka topic (Kafka is used for a lot of microservices in our systems).
  • A (again so called) sync service which is subscribed to the Kafka topic. Whenever something is published in the topic the service consumes it and pushes it to the CMS.

Here you can see a diagram that shows the architecture at high level:

enter image description here

For me this is (sort of) an event-driven architecture. An event happens in the legacy DB and other services get to react to it.

However, a colleague from me insisted in that it could not be considered as an event-driven architecture because the owner of the master data that provoked the event was the legacy database. Basically, my colleague's argument was that each service should own its own data and send event messages to other services (via a mediator), and that the other services should react to these events and build their data based on their own needs.

Now, I agree that each service will have to use the data from the event as it seems fit for it. But I do not agree that this argument means the architecture I explained is not event-driven. I have searched over the Internet and did not find a clear statement indicating this data ownage as being a prerequisite for an architecture to be considered as event-driven.

Is it really necessary in an event-driven architecture that each actor owns the data he works on? I.e, do I really need to not use the master data of the legacy database in the CMS? And/or should the CMS instead (ideally) create its own data and somehow map it to the legacy database data?

Some clarifications:

  • I know the presented architecture may not be the best you can have, but working with legacy systems sometimes forces you to tweak things.
  • I also know this architecture means that the services are coupled at the data level, but it is what we have for now, again due to the legacy system.
Was it helpful?

Solution

Event Driven Architecture

Properties:

  • Some state somewhere has changed.
  • The event represents the fact that this state has changed.
  • There is something waiting for, then receiving, and then reacting to an event.

If the system fits into that description its event-driven. It may be other things but events are driving (at least some part of) it.

Actor

You didn't go into depth on this in your question body, but i see it lurking in your questions title. I feel that this is probably where your colleague is coming from.

Properties:

  • May receive messages
  • May send messages
  • Cannot modify state it does not own.
  • Cannot read state it does not own.
  • state it owns cannot be read by a third party.
  • state it owns cannot be modified by a third party.

If it has these properties its an Actor.

I think your colleague is thinking in these terms, but has two misunderstandings:

  1. What constitutes an actor's own state.
    • A Database by definition is a separate actor. It receives some form of SQL-like message, manages its own state, sends back results, and perhaps generates other messages (events).
    • An actor may of course contain a Database actor internally, but it must do so completely, otherwise the state is shared and an actor cannot read/modify unowned state, or to state that a third-party can modify.
  2. How actors send and receive messages.
    • Your colleague appears to believe that this is only possible by some form of co-coordinator. This is not a bad solution for loosely coupled actors. But it is not the only way to organise communication.
    • Messages can come from different message-streams such as Network, GUI, Database. Each has a different message protocol, subscription, and cancellation means.
    • An actor can have more than one address on a given message stream allowing it to receive duplicates, or even receive different event sets.
    • Message passing can be accomplished through many means including lossless and lossy substrates (TCP vs UDP)
    • It is even possible to use a Reader/Writer circular buffer. While it is a shared structure, all state is clearly owned by one and only one actor at a time.

OTHER TIPS

Its a tricky one, but I can see your colleagues point.

If you consider an Insert 'event' coming into the system from some user input, it does indeed seem like an event based system. Where the Input event causes action on the db and cms. But then really the owner of that data could be considered to be the user. So both systems own their own data.

But If we consider an Update, effecting a subset of columns on a single row, or perhaps many rows. The event itself depends on existing data in the DB. You couldn't really add another consumer that had not received all previous events.

In my view your setup is more like replication than an Event based system.

Additionally you further muddy the waters by using Kafka, a streaming database, rather than a message queue.

Of course it hardly matters what you call things, is there some aspect of 'event based' systems that you wish your system to have?

Licensed under: CC-BY-SA with attribution
scroll top