Focus for next iterations in Lokad.CQRS Sample Project

This is just a quick overview of things that I have in mind for the next iteration of Lokad.CQRS Sample Project.

Please keep in mind, that Lokad.CQRS reflects current state of things, as spread across the frontier of development in Lokad projects. It is probably as opinionated as 37signals are opinionated about their projects.

While moving forward, one can carry only so many things in his bag. So we discard less important things and keep life-savers sharpened up. There are often harsh decisions without compromises, while focusing on simplicity.

If something changes, it is already based on real pain that caused this change in our internal projects. And while inflicting this change I also keep in mind:

• my own team members who then eventually will give me happy faces about upgrading their own individual projects;
• Lokad.CQRS community whom I don't want to disappoint;
• new cloud projects that are lined up on the plate and that will use latest state of Lokad.CQRS Sample project as a kick-start. It would be better to keep friction minimal (including friction of remembering non-intuitive things).

In essence, Lokad.CQRS is based on immediate pain and need to still keep on moving forward. This creates filter that ensures that changes are practical and cheap. This also helps our own development at Lokad to stay more real (and hence capable of dealing with upcoming challenges)

So here's the list of things that I consider important for the next turn. Obviously, if "Everything is important", then nothing turns out to be important. So I'd appreciate help in prioritizing this list.

Web User Interfaces and Experience

We've been cranking up so many new Web UIs at Lokad recently, that friction has dropped considerably. Partially this can be attributed to paradigm shifts, partially small changes in infrastructure and nice leverage of latest Bootstrap and jQuery libraries.

This approach actually allows to target mobile devices with the same UI, which is pretty cool.

Since currently Lokad.CQRS sample project does not have any WebUI guidance, I believe adding it is of the primary importance.

Self-maintenance infrastructure

A few things are going away from our latest projects. In particular, Audit view Rebuilds are no more. Instead we are getting this functionality off our minds and letting server handle this (both in on-premises mode and in the cloud). This simplifies development a lot.

Also, importance of separating Bounded Contexts and keeping them lean - also becomes extremely important. It helps with the deployment and maintenance of complex setups. This affects how we think and structure systems.

Domain development

Domain development is a complicated part that cost me a few months of development lately. The most important outcome is that: AggregateFactory approach is a bit limiting. Sometimes you need a lot of services or to access multiple aggregates in one handler.

I know, that the latter is not recommended as a general rule of thumb. However in practice things work out differently.

Fortunately, there is a rather simple way to upgrade aggregate factories to more conventional command handlers.

Cross-Cloud Capabilities

So far we have been focusing only on two major deployment modes: on-premises and Windows Azure cloud. In order to have greater flexibility and reliability for certain scenarios, it is important to start targeting other .NET-capable cloud environments as well (RackSpace and Amazon AWS being the primary goals). Scenarios include:

• continuous data replication between clouds.
• persistent virtual machines for highly-specific integration scenarios.
• fast failover scenarios; everybody remembers Amazon outage, right?

Event Sourcing Performance

Current ES performance is abysmal Lokad.CQRS, which is my fault. We were not optimizing for 10^6 events while developing first iteration. Now this starts showing up and will have to be fixed sooner or later.

What do you think?

If you have read that far - I thank you.

Here are three simple questions, that I'd (seriously) appreciate getting feedback on:

• What is the most important thing on this list for you?
• What hurts you the most, while learning Lokad.CQRS or working with it?
• From looking at the Lokad.CQRS core code, what is the least useful feature for you? (features are usually organized in folders)

Comments

[Kerry Street]

Rinat,

I perceive the Lokad.CQRS Sample Project as having two major community goals; learning and delivering.

1. Learning: Provide production-worthy sample code with its own documentation and tutorials for learning about event centric development and guidance on how to use it.

2. Delivering: Maintain an evolving and production-worthy codebase that can be used as the foundation to jumpstart new event centric projects on major cloud platforms.

The features in your post fall largely in the "Delivering" category. They evolve the project to the next set of enhanced functionality. This is important for the "Delivering" goal but less so for the "Learning" goal. At this point in time I think the project and the community needs more "Learning" love, than "Delivering" love. I think the next turn should primarily focus on the learning goals over the delivering ones. I believe this will increase community adoption and understanding of the project.

Many of us are still in the learning phase and need a reliable and thorough starting point. If you think of each CQRS/DDDD/ES expert as their own "bounded context", with their own similar but different language, it can be challenging for beginners to understand all the new concepts and moving parts of an end-to-end solution that uses this approach. It is encouraging that various members of the community are working together on Microsoft's CQRS guidance to clarify some of the ambiguity, but there is still a need for beginners to have ONE expert source that is complete and consistent within their domain. This single source eases the initial learning curve and results in understanding at least one approach more rapidly. I think that you and Lokad.CQRS can be that single source for now.

I am still basically a beginner with event centric development. I have read entries on blogs and blikis, watched recommended videos, listened to podcasts, and am in progress with the "blue book". I am ready to make something useful using Lokad.CQRS as the foundation. I have been doing "code archaeology" to understand the latest Lokad.CQRS sample: Digging around the solution, following the path of Program.cs, and just trying to figure out what is going on and place it in context with what I have learned about CQRS so far. This has been enjoyable but I have found myself thinking, "Digging through this code is fun, but a lost ancient civilization didn't create it. They are alive and I can just ask them to explain and document what is going on and how I should use it!" :)

If I was the product manager/evangelist for this project my prioritized backlog would look something like this:

Web User Interfaces and Experience items: Make it deployable to Azure so people can see and use a fully functional end-to-end solution on this platform.

A Lap Around the Lokad.CQRS Sample Project: Watch this incredible screencast as we walk you through every folder in the sample solution to explain what it does and how it relates to the CQRS/DDD/ES big picture you have been hearing about. What if anything is missing from that big picture and what are your options to fill the gaps?

Documentation: Determine ongoing documentation plan and how community can contribute back. As the code evolves (see Lokad.CQRS v1, v2, v3), how do we keep the documentation, tutorials, and samples up to date so that the community can always understand what they are looking at in the latest branch? Bliki? GitHub pull requests of doc files? Other?

From Sample to Success: A step-by-step series that walks you through getting your hands on the Lokad.CQRS bits to transforming it into a fully functional solution. The solution includes a small end-to-end example that takes advantage of native cloud and CQRS capabilities and shows how you add things beyond the Security and User Aggregates. Massive scale example for mobile clients anyone? What about handling synchronous query requirements with this approach?

After those were done, I would get back to your list of:

Event Sourcing Performance
Domain development
Self-maintenance infrastructure
Cross-Cloud Capabilities

I realize that documenting the code and creating tutorials for beginners on what you and the Lokad team already know may not be that exciting, but it feels like that is what is needed most right now. If you do not want to do it, I would be happy to try it once I figure it out myself. :)

I believe that this project is important for the community because it provides a glimpse into a real company's evolving production code that has been adapted to real customers and business problems. Most of the other public resources in this domain that I am aware of do not provide the level of detail and completeness of code that is needed by beginner and intermediate developers to deliver production-worthy distributed solutions. I think focusing more on the learning goals right now will increase adoption and understanding and I am willing to help in any way that I am able.

Wow, this is a LONG comment! Sorry about that.

Thank you again to you and Lokad for these contributions.

Kerry Street

[Rinat Abdullin]

Thank you very much for such a clear and thorough feedback. I'm still a beginner in CQRS/DDD area myself, but already forgot the perspective I had years ago before entering this field. Your clear and extensive comment really helps to get back on track. I really appreciate it.

Sharing material is not a problem, problem is that either it ages helplessly fast, as we learn and that every decent documentation article requires amount of time that I get only once per weekend.

I will think how to adapt Lokad.CQRS project and workflow around it to try to chase two rabbits:

* keep core project flexible so that it could represent current development frontier in Lokad.
* structure workflow so that it would be easy for us to share (mostly) in small bits and for the community to contribute towards some focused and coherent guide.

Please give me a few days to think about that.

Thanks again for motivation and clearly showing which pieces are missing and in which form they could be structured to help the community.

[Kerry Street]

Thanks Rinat. I look forward to hearing your thoughts and the great work to come.

@kcstreet