Learning Distributed Systems: Sockets and Event Store

I spent some quality time this weekend catching up on the theory of distributed systems. Pieter had done an amazing job with ZeroMQ as both a community and reference doc.

Armed with new insights (and some severe brain-rewiring) I gave another try to sketching a prototype of simple event store on .NET platform. Simplest possible design was selected:

Clients do Request-reply via TCP socket to the Store (unmarshaller that appends messages to file stream). There is a non-blocking reader (just like the one in LMAX) that catches up with writer and broadcasts events to the network via PGM.

Surprisingly enough, when you start this entire setup with 6 clients trying to push events (16 byte messages) to a single store (served by 1 thread), you get 20000 messages per second in a durable mode.

Obviously there is no real replication (binary star would work here for fail-over) or load balancing and partitioning (majordomo pattern shows the direction for the design). Besides, reliable delivery of published events is not spelled out in the code as well (i.e. handling cases when even handler goes down, misses a few events or wants to replay everything).

Still, the snippet is on github, should you be interested (it's really simple).

So far I personally really like the approach enforced by the entire ZMQ project. It requires paradigm shift from the development model I'm used to, but potentially allows to gain better performance without touching anything outside the infrastructure of a usual CQRS/DDD system with event sourcing. This, and better utilization of cloud resources (or computing resources in general) make this a promising direction for further studies. We'll see how it goes.

Update: this article continues in Tech Layer of CQRS Systems: Pushing it Further