Working on Event Store
Wednesday, March 21, 2012 at 15:31It looks like Lokad might publish another open source project soon. Namely: extremely simple event store.
I need a decent ES store to support immediate data replication and projection rebuilding across multiple data-centers and clouds. In a sense, Lokad has grown to position, where we need to support Digital Nervous System concept (high level way to represent interactions between multiple bounded contexts in a growing company). Implementing this with pure files/blobs is not enough, since we need dead-easy way to support scenarios like this one (without driving design crazy):
The simplest way to achieve that functionality is by enhancing basic tape stream concept (from Lokad.CQRS) with 2 more extremely simple and powerful things:
- Task Parallel Library (which is already used in almost all of our servers)
- ZeroMQ sockets
Project will be dead-simple and open source (so that we'd get some community feedback and help to move push the state of the art). If you would need something more flexible and feature-rich, you could either use EventStore of Jonathan Oliver or wait till Gregory Young moves forward with his proper 9 yards implementation.
Logic behind this project will be based on highly consistent model (no dynamo-DB style) with replication and simplified persistence concept from bitcask. It will include lessons learned from LMAX guys and practical experience with Azure/File streams. Focused and KISS.
Reasoning for this project is to solve immediate problem with event stream replication and management at Lokad, but we'd also like to use this as an opportunity to move things forward for everybody. As you probably know, Vaughn Vernon is working on the next DDD book, that might include some material on effective design of Aggregate Roots with Event Sourcing (AR+ES). I would love to see this simple and open source event store get stable before this book comes out.
Yes, I know, that it sounds like non-enterprisey approach and severe case of "Not Invented Here" syndrome. So if you know any event store that fits all these requirements, please drop me a line:
- Open source with permissive license for any kind of use (no limitations like in NServiceBus or RavenDB);
- Simple and based on practical experience;
- capable of working in hybrid scenarios (both on-premises and in the cloud);
- .NET;
- Minimal dependency footprint and xcopy deployment.
Reader Comments (7)
Great; looking forward to a KISS event store...
Sounds great! Looking forward to it. How will this fit in with the ideas described in your "Pushing It Further" post? 0Mq Majordomo router to handle partioning?
Current implementation of EventStore is just a first step in this direction. It's going to be just a minimum viable solution needed to get event delivery and replication running in our systems
Would be great to hear your thoughts on Disruptor... maybe a blog post?
Andre, there should be a distributed podcast (it was recorded earlier) with the LMAX guys. I think they are much better in talking about their project, than I ever would be able to
I’ve watched some of their presentations… would be great to read more specific info on Task Parallel Library vs Disruptor for example, also using Disruptor.Net with ZeroMQ
1. TPL and Disruptor are two completely orthogonal concepts. TPL provides a set of abstractions and scheduling on top of thread pools.
2. ZeroMQ is probably something that would play well with disruptors. In fact, you can dive into buffers in ZeroMQ itself (which are similar to disruptor or general TCP buffers)