Dispelling the eventual consistency FUD, using event sourcing

So your organization is toying with the idea of using Event Sourcing. The benefits look promising, but you can’t seem to shake off the nagging doubt of that one aspect of adoption - Eventual Consistency because of its alignment with CQRS. Your organization’s systems have always been following the principle of Solid Transactional Consistency (maybe you are in the Banking Business), and CQRS/ES becomes a big no-no because of just this one aspect. In addition, everyone within your organization finds those examples of order items eventually appearing in your cart extremely amusing. Finally, something to agree upon on that Friday Zoom bonding call!

In the CQRS/event sourcing space, we are equally guilty of not countering the FUD with enough examples and literature that gives relevant data points for folks wanting to adopt these patterns. What if I told you that your systems built with CQRS/ES are as transactionally consistent as systems built without these patterns using the more formal storage methods!

Well, let’s walk through an example actually to prove it. Before we get into it, a quick 30,000 feet view of these patterns - CQRS helps you split your Domain Model into two distinct partitions, the Command Side for processing “Commands,” which result in a state change, and the Query Side for processing “Queries” to retrieve state. Event Sourcing proposes the storage of the state of your application as a sequence of immutable events in an event log. State changes are validated directly against that event log, while Queries are executed against projections derived from that log. This allows the event log to be considered a reliable source of truth.

We will probably take the most commonly used example of Money Withdrawals to demonstrate this. But, first, we need to consider the only invariant that the customer has the required balance in their bank accounts to perform a withdrawal. For example, a customer might make multiple withdrawals in a short time frame, and at the end of each withdrawal, the balance needs to be in a consistent state. Else you could have customers make more withdrawals than allowed, and the bank shuts shop.

Traditionally, you deal with the consistency problem using a database with a single mutable record representing the account balance. Every withdrawal operation loads the balance record, checks the withdrawal amount against the current balance, and accordingly, the system decides whether to permit the withdrawal or not. An important point to note here is that before the customer intends to make a withdrawal, they are presented with a view of their current balance. This View is constructed by loading the same mutable Account Balance record.

Essentially you use the same model:

• To display the balance, which helps the customer to make an intent to make a withdrawal.
• To help the system decide on whether the intention to make a withdrawal is permitted or not.

This approach gives you the perceived benefit of guaranteed “read” consistency. However, while the information is presented on screen, there is no guarantee that the data isn't modified in the meantime. Thus, the user's decision is always made on stale data. In addition to that, you have to deal with problems around scalability, availability, and model complexity. CQRS/ES offers a great way to help address these kinds of problems with strong consistency guarantees.

How exactly is that achieved? Going back to our original example, say we now have decided to adopt Event Sourcing. So instead of a single mutable account balance record, we now store all the operations (All the withdrawals and the deposits) that happen against that account as a sequence of events. 

When the customer now makes the intent (i.e., a Command) to do a withdrawal transaction against a particular account, this will load the past events from the Event Store against that account (i.e., the Withdrawals/Deposits) and then replay them to arrive at the current state for the account balance. This state is then used to decide on whether to permit the withdrawal or not. In short, it is guaranteed that Command processing in an Event Sourced system will always be done against the latest and current state, not an eventual state. 

This takes us back to the View as the customer will intend to perform a withdrawal only after they are presented with the balance. In Event Sourcing-based systems, the balance (aka the balance projection) that needs to be displayed is not retrieved from the Event Store. Instead, it is retrieved from another datastore which constructs and stores the balance the way it is used in this case as a display. The construction and storage are done asynchronously by subscribing to the events emitted once it is committed to the Event Store. Essentially it might not reflect the current state, and hence we term the balance projection as eventually consistent. 

To summarize, Command Processing in Event Sourcing will always be performed against the latest state and not an eventually consistent state. However, projections used to query the state are always eventually consistent due to the asynchronous nature of its construction and storage. While the example shown here is a bit trivial to demonstrate the concept, it holds for very complex command processing. It should be noted that in any system being built, there will be islands of consistency in a river of constant change. This holds for both Strongly Consistent and Eventually Consistent systems.

Axon provides Event Processors that significantly scale up Projections' construction in almost near real-time once events have been published. In addition, it also provides Subscription Queries that listen to any updates to your Projections and delivers them in real-time to interested clients resulting in an enhanced user experience.