Breaking the monolith – How to design your system for both flexibility and scale – Part 4: The Message

This post is part of a series. you’d probably like to read it from the beginning. Or check out the previous post in the series

Those of us who had the chance to experience the exquisite pleasure of getting a formal education in the field of software engineering were told a fabulous tale. This tale of extreme bravery and audacity  included mythical creatures and ideas – like a zero-downtime zero-loss infrastructure, instant globally synchronizing data (allowing the creation of what is sometimes referred to as the “single point of truth”),  and of course – the object oriented system.

Well, as all fairy tales go, this one also had mostly turned out to be a bunch of gross simplifications of real life issues threw together to protect the innocent minds of the young, unschooled or the straight up stupid.


The problem is, that this whole object-oriented design thing (we’ll call it OOD from now on, if you don’t mind) is close enough to how things work in the real world to fool some of the people most of the time, And a very delicate touch is needed to separate truth from fiction.

After decades of software development, as we bravely march into the cloud age, we have seen that the OOD approach is all well and good right up until you need to change something. Maybe you want to change how a certain feature work, maybe you need to deploy a new version, or maybe, god forbid, you need better performance to service your growing audience (of Justin Bieber death metal cover  lovers, perhaps?). If your well crafted product is designed in an “Object Oriented” manner, you now, as they say, have two problems.

where have we gone wrong?

By designing your system around what is constant you probably left out it’s ability to roll with the punches. That because those entities in your system that became object are doing something which we in the industry like to professionally call “a big no-no” – they implicitly lock data and computation together. Moreover, to add insult to injury, using the ODD approach is a very big temptation to unintentionally develop your system in a bottom up manner, another one of the HPC software engineering mortal sins.

What you should have done is to write down those basic entities on a piece of paper (or whiteboard, depending how you like to roll) and then, instead of running off to code the hell out of them into objects, to write next to them what are the basic events your system should handle.

Think of an instant messaging app, they may only have so many instances of their three basic entities (message, user and group, yes?) on their server in any given moment, but many orders of magnitude more events (message on transit, unread messages, message ack.,etc.),especially if you combine the whole lifetime of the system. Using an object-oriented design would have probably cost them in performance – as you should know the state of each object in the system in any given moment. This holds you from scaling out without having to pay synchronicity tax and worst, you now have to pay the bills for storing all that data.

Let’s examine the  event oriented take on the same system-we will call whichever data the servers need to hold the ‘state’ of the system. so the state of the servers in the event oriented approach need to be:

  1. which users belong in this group – a very simple embarrassingly parallel key-value store
  2. for each of these recipients, what is their push token (how do I contact them) – an even simpler embarrassingly parallel key-value store

the system is in fact agnostic of a specific user state, and hence able to scale easily, and as I like to say  much better.

The Way It’s Done

So let’s assume for a moment that I persuaded  you into trying that event-thing (though you still think it’s just a hype and no one will remember it a year from now). How do one start on designing an event oriented system, or worse, how do you transform an existing, object-oriented system into a sparkly event oriented one?

You’d be surprised to know, that you already have all the tools you need to design an event-driven system from scratch – it is the same set of ideas you used to unearth the underlying objects defining the system (and then, by regression, the different actions these object perform), only this time you will describe the different events the system should handle, and from that you will extrapolate which entities in the system these events affect. Try and thing what is the business problem this software is meant to solve – these are your events.

In the other hand, a “brownfield” project of turning an OOD system into an event-driven one is a whole lot complicated matter, and as such will be the subject of a future post in the series.

In the meantime let me leave you with a bit of a brain teaser:

In “Who Moved My State?”, the author, Miro Samek claims (and to some degree proves) that event-driven software,  though essential, tends to be unpredictable and oversimplified at best. What tools and paradigms do we now have so that effect would be diminished, taking into account that the system requirements will inevitably change (how can your system keep making sense after years of bad coding habits)? And, assuming Samek is right, how do you explain why every internet scale company out there is using event-driven processing?


Adam Lev-Libfeld

A long distance runner, a software architect, an HPC nerd (order may change).

Latest posts by Adam Lev-Libfeld (see all)

Breaking the monolith – How to design your system for both flexibility and scale – Part 4: The Message

Leave a Reply

Your email address will not be published. Required fields are marked *