Software Design Blog

Journey of Rinat Abdullin

Domain-Driven Design, Event Sourcing, Rx and Marble Diagrams

Recently I’ve been taking a deep dive into the Reactive Extensions for .NET. This is a wonderful project that comes out of Cloud Programmability Division at Microsoft. Rx helps do to really wonderful things with asynchronous and parallel programming in unreliable distributed world. Do you remember what LINQ did to the collections (and much more)? Well, Rx is going to do (and doing) the same to events, asynchronous and distributed programming.

So if you have to deal anything that requires more than a single thread, process or machine to run - you’ll be impacted in some way or the other.

The thing I love the most about what Eric Meijer and his team are doing - the underlying logic which is simple plain and so beautiful. Even the entire IObservable/IObserver thing could be logically derived from IEnumerable by inverting the concepts logically.

Just watch these videos, if you are interested in the details here (highly recommended):

What does this have to do with CQRS, Domain-Driven Design and Event Sourcing?

When I’m building some complex interactions and systems, I’m trying to follow the same principle. System design and evolution are not guided by the intuition or creativity. Pure logic and mathematical reason are the most reliable advisors here (it feels like proving a theorem). It’s really hard to explain in words, but logically solid design can handle a lot of things with really little effort (and code) just because of it’s nature. Basically it allows to say to new challenges: “we did not code for this, but since this is logical, it already is in the design, you just need to enable this”.

While I’m just learning CQRS/DDD/ES I want to do it in the logical way that will prevent me from doing bad and illogical things (which will impact my systems later in their lifecycle). Way of thinking that Eric Meijer shows, seems to help here. Besides, it helps to merge the ideas of Pat Helland (almost infinitely scalable systems) into the overall CQRS concepts.

Reactive Extensions benefit a lot from the marble diagrams representing parallel processes and their interactions. So I’ve tried to re-apply these principles to my understanding of CQRS/DDD/ES. By mechanically reflecting the logic, here’s what I’ve got for the command handler:

Command Handler in CQRS/DDD/ES

Essentially this picture reflects the flow of commands (marked with D for Do) incoming into the handler. Commands can come as a single or as a batch (that’s the marble or tick in this reality). Commands coming together either succeed or fail together.

Each incoming tick translates into a new parallel reality for an aggregate (just like SelectMany in Rx, where you get observable of observable of T). Then we flatten successful commands onto the change stream (each change is a unit of work), while errors are flattened into the failure stream.

There is a message dispatcher that catches up upon these streams and sends domain events, command failures and confirmations back to the message bus. This way subscribers will have logically complete information about what’s happened in the business logic. Subscribers could include sagas and event handlers working with read models (which will be used to provide clients with feedback that includes potential command failures).

Note a few differences upon the traditional CQRS with event sourcing approach as advocated by Greg Young:

  • Aggregate Root can react to incoming commands by providing a finite stream of events, optionally terminating with error (there is a direct translation from Rx observers here: OnError and OnComplete).
  • We save successful change (command(s) and resulting events) atomically as unit of work. Persisting commands will actually let us ensure command idempotency in the cruel world of scalable message queues (with “at least once” deliveries). Essentially this is reflection of Helland’s activities that we need to have in the same scope with the aggregates that own them.
  • Command failures are also saved (although the stream could be different from the event source). We don’t need to save the intermediate events though - just command(s) and the resulting failure.
  • Message dispatcher runs upon the failure and change streams, publishing domain events into the message bus. We have logically complete information here, that could eventually be provided for the client (sender) for the retrieval.
  • theoretically we can easily process commands for a single aggregate in multiple threads, serializing access to the change and failure streams.

Things that do not fit in, yet:

  • In CQRS practice there are cases, when there would traditionally be a call to the domain service (in the command handler and before the aggregate root). This does not fit in here. In a Rx world this would probably be a separate async command handler. Yet this would require an additional layer of translation (another command handler and event handler).
  • events and commands are some sort of duals. They are both messages that bring along different intent and help to organize CQRS architecture (this comes from the DDD realm). Yet, this duality (and the rules coming out of it) brings additional complexity, when you need to chain multiple operations (first call unreliable operation, then pass the result to the aggregate, then do something else). I think I’m missing some aspect of commands and events that would help to lay out interactions in a clean and logical way.

That’s a theory and logic as I see them today. This means:

  • tomorrow perception might evolve into something different that better brings together known patterns and constraints.
  • in theory there is no difference between theory and practice. In practice - there is a difference.

If I could bring in sagas, view event handlers and domain services into the picture in a logical way (and without breaking any existing scalability and consistency constraints), I’d be a happy man.

We’ll see how it goes (see xLim 4: CQRS in Cloud series for any latest materials on this topic). Meanwhile I’m really interested in your thoughts on this subject.