Software Design Blog

Journey of Rinat Abdullin

AppHarbor-style Lokad.CQRS Host

Just sharing a weird idea that surfaced earlier.

As you probably know, in addition to numerous improvements, Windows Azure has announced support for git deployments of web sites recently (nice move!). This can reduce development friction noticeably, but is still far from my dream low-friction scenario on AppHarbor, which works like this:

  1. Developer adds new feature, which affects both server-side and UI client. UI changes require completely different read model representations (say, for querying across multiple bounded contexts in a new way) and so new projections are defined as well.
  2. Changes are tested locally using file system abstractions and then are pushed into the master.
  3. Deployment server picks up the changes and begins deployment.
  4. New instance of application server is built and deployed side-by-side with the old one. When server starts up, it detects that projection code has changed and automatically rebuilds persistent read models (in essence, precomputes views). After initialization process is complete - server comes back online and swaps new persistent models in.
  5. New version of web UI is deployed and swapped with the old one as well (via the load balancer managed by deployment server).

This scenario works on AppHarbor, but is not a good fit for Windows Azure, since step 4 is not supported.

While trying to come up with various work-arounds for this limitation, an old idea surfaced under the new look. What if we rewrote deployment server in a way that is hard-wired specifically to Lokad.CQRS projects?

In essence such deployment would be somewhat similar to Lokad.Cloud.AppHost, but with a different twist. Instead of having a generic process as deployment cell unit, we would have bounded context as defined within Lokad.CQRS. Such bounded context can be described via:

  • Application Services (sets of command handlers);
  • View projections (sets of event handlers that update same persistent read model);
  • Tasks (long-running processes);
  • Ports (subscriptions to external events which trigger certain actions in response).

In my projects each bounded context is explicitly described in the code like this:

public static class SomeBoundedContext
{
  public static IEnumerable<object> Projections(IDocumentStore docs)
  {
    yield return new UserIndexProjection(docs.GetWriter<byte, UserIndexLookup>());
    // all other projections
  }
  public static IEnumerable<object> ApplicationServices(
    IDocumentStore docs, IEventStore store)
  {
    // set up some dependencies
    var storage = new NuclearStorage(docs);
    var id = new DomainIdentityGenerator(storage);
    var unique = new UserIndexService(storage);
    var passwords = new PasswordGenerator();

    yield return new UserApplicationService(store);
    yield return new SecurityApplicationService(store, id, passwords, unique);
    yield return new RegistrationApplicationService(store, id, unique, passwords);
    yield return id;
    // all other application services (groups of command handlers)
  }
  // etc for tasks and ports

In essence, each new CQRS/DDD project at Lokad starts by copying latest source code of Lokad.CQRS Sample Project and then starting iterative domain modeling process. Models are always expressed in form of Bounded Context (like the one posted above), optional specification tests and some optional client UI (all this stuff - runnable on a local development machine or server using file system for persistence and messaging). After all is ready for the first spin, code are wired to proper deployment adapters in Lokad.CQRS and pushed to the cloud.

The most boring and tedious part in this routine is actually setting up the infrastructure and Lokad.CQRS building blocks to host the bounded contexts and UI.

So all this got me thinking - what if it would be worth another try to build a .NET DDD Model Host from all that code and experience, which would:

  • provide same Lokad.CQRS infrastructure (including the new version of embedded event-store I’m building for Lokad projects) but as a server host, which is either started locally or in the cloud;
  • Load bounded context definitions and wire them to the message handlers, persistence adapters (i.e. files for local environment and Azure for cloud), message quarantine routines etc
  • Track changes in the code and invoke rebuilds of persistent read models, when necessary.
  • Manage swapping of application services between versions (along with exception handling, timeouts of individual command handlers etc).

Theoretically, this would allow following scenario.

  1. Developer starts a DDD Model Host somewhere.
  2. His TeamCity build server is configured to push bounded contexts (as dlls or nuget packages) to this Host. Essentially, whenever a code is pushed to certain branch, it will be started up in a separate app domain, just like AppHarbor does.
  3. Developer registers some web UIs in this Host. Host then polls them regularly for uptime (like here) and also checks some well-known endpoint for projection code. If new projection code is detected, it will immediately be pulled and rebuilt automatically, by rerunning event streams against this new code. Host will then be responsible for keeping these read models up-to-date.

In essence, among other things, this allows to have multiple thin web clients for various bounded context running against one application server, which would host multiple bounded contexts. Theoretically scaling out rules can also be defined.

There is nothing really new in this (yet another) attempt to make a framework out of Lokad.CQRS. Here are the primary differences from other existing approaches:

  • AppHarbor scenario for Lokad.CQRS deployments: is that we have a server that is strictly tailored to the Lokad.CQRS building blocks (application services, tasks, projections and ports) and relevant cross-cutting concerns. We can have multiple bounded contexts within the same worker, plus keep the ease of deploying same BC to various environments (on-premises and in the cloud) using the same codebase. Plus, we get web console for viewing all the stuff.
  • The only real difference from Lokad AppHost: strong tailoring towards Lokad.CQRS blocks and providing infrastructure (proper persistence and messaging abstractions are injected by the environment).
  • Difference from Lokad.CQRS deployments: bounded contexts are loaded not statically but rather dynamically; it becomes easier to deploy new BCs on an existing host.

Theoretically all this is doable and should reduce development and maintenance friction in my projects (both behavioral and big data deployments). However, from practical perspective it still might be simpler to keep on copying code from Lokad.CQRS and managing deployments manually on individual basis (less configuration code to write and hence less places for bugs to hide).

What do you think?

PS: it seems that Squarespace comments system went crazy recently and might prevent you from posting completely (as opposed to simply accepting comments to moderation queue, which is needed to stop all that spam from coming). If your comments don’t show up within a day, please feel free to kick me in the twitter: @abdullin