Lokad.CQRS - Messages, Commands, Events and Ping Pong

This is the next article in the Learning series for Lokad.CQRS Guidance. In the previous one we've talked about the big picture behind the project and compared Lokad.Cloud and Lokad.CQRS projects.

Command-Query Responsibility Segregation is heavily based on concepts of messaging and enterprise integration. In a sense, CQRS is a highly efficient message-based integration between the User Interface (whether it is a Web UI, Desktop UI or somehing else), application servers and some persistence media. So we are going to see a lot of messages around.

A message is just a packet of data that is sent from some component (sender) to an address (recipient). Messages are just like emails:

• both sender and recipient are expected to be able to understand the message;
• sender does not expect immediate response (and does not block the calling thread);
• sender designates the address to deliver to (some logical name), it will be the responsibility of infrastructure to make sure message will be delivered to the recipients associated with that address.

Actually, the last point is one of the reasons behind the scalability of CQRS. When we are sending message, we just name the address. This address could have a single recipient or multiple Azure Worker Roles with enough horsepower to handle the whole stream of CPU-intensive commands.

Commands and Events

In the world of Lokad.CQRS messages are usually either commands or events (this notion partially coming from the Domain Driven Design). At the code level both types are just messages. But if we separate them from the start (both in the code and in our mind), things get much more logical and simple.

Commands represent an instruction or request to do something. They are named in the imperative:

• Send Email Command
• Create New Account
• Relocate Customer Command

Commands are often sent from the User Interface to the application server. They capture the intent behind the action. They can be rejected by the server, if it decides that it is impossible to complete the command in the current state.

In a sense, command is like a remote method call that does not expect immediate response (the does not expect part is really important for scalability). Method name is name of the command, and parameters match to the properties of the command.

For example, command might look like this one:

BuildBasicExcelReportCommand
{
  "ResultsStorage": "reference-000754",
  "SolutionId": 102,
  "SyncId": "795a96b6-6089-4c8b-b965-9d8e0103d27f"
}

I prefer to print out messages for human investigation as JSON formatted entities. They are more readable than XML. Yet, this does not mean, that's how they go over the wire.

Events, on the other side, inform about something that has already happened. They can't be rejected (you can't change the past, since managing parallel realities is too expensive). By simply recording all events that happened in the system you get perfect audit log, much simpler integration, the ability to look back in time and derive various reports.

One of the events, that could possibly be generated by BuildBasicExcelReportCommand, might look like this:

ReportCreatedEvent
{
  "ReportId": "b57de418-7747-402d-a2ae-9d8e012b596b",
  "SolutionId": 102,
  "ReportType": 2,
  "Name": "Excel Report",
  "StorageContainer": "salescast-solution-000102",
  "StorageReference": "report-b57de418-7747-402d-a2ae-9d8e012b596b.xlsx",
  "ContentType": 
    "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
  "FileName": "SalescastReport.xlsx",
  "ReportSize": 2340545,
  "TotalSeconds": 98.0115131
}

Basically, as we see, commands and events in Lokad.CQRS are just both messages that differ by the naming and intent. This explicit convention helps so much down the road, that it is extremely important to follow it.

So we've talked about how messages are divided into commands and events. We'll cover more theory in the later articles. For now, let's get down to the code and some message sending.

Sample - Ping Pong

In the messaging world, there is a popular equivalent of "Hello World" program, which is called "Ping Pong". Basically "Ping Pong" samples just show, how to:

• define messages;
• send messages;
• receive and handle messages;
• configure and start up the solution to do all this.

We'll follow the classics, as well. Code for this tutorial is Sample-01 in the Lokad.CQRS codebase. You need Visual Studio 2010 and the latest Azure Tools for VS2010 (version 1.2 at the moment of writing).

We start by creating a new Azure Cloud Service with a WorkerRole.

In the same project we will define schema for the PingPongCommand, that just holds the current number of the ball and name of the game being played:

[DataContract]
public sealed class PingPongCommand : IMessage
{
  [DataMember]
  public int Ball { get; private set; }
  [DataMember]
  public string Game { get; private set; }

  public PingPongCommand(int ball, string game)
  {
    Ball = ball;
    Game = game;
  }

  public PingPongCommand Pong()
  {
    return new PingPongCommand(Ball+1, Game);
  }
}

Note, that you might be tempted to write messages with private read-only fields instead of the read-only properties (just to make the code look more compact). It is advised to avoid doing that. Later on down the road a lot of Lokad.CQRS will be based on the ability to inherit messages from multiple interfaces and to subscribe to only some of them. Since interfaces require properties, you might end up with some message classes that use fields and the others with the properties. This might get confusing.

PingPongHandler simply subscribes to the message, prints it to Trace, sleeps and sends back with the increased ball counter.

[UsedImplicitly]
public sealed class PingPongHandler : IConsume<PingPongCommand>
{
  readonly IMessageClient _sender;

  public PingPongHandler(IMessageClient sender)
  {
    _sender = sender;
  }

  public void Consume(PingPongCommand message)
  {
    const string format = "Ping #{0} in game '{1}'.";
    Trace.WriteLine(string.Format(format, message.Ball, message.Game));
    Thread.Sleep(1000);

    _sender.Send(message.Pong());
  }
}

Note, that for simplicity in these samples we inherit messages and handlers from interfaces declared in the Lokad.CQRS framework (Lokad.Cqrs.Default namespace). However, you don't have to do this! You don't even need to reference Lokad.CQRS from your domain or handlers, as long as your Lokad.Cqrs Engine host (Windows Azure Application Server in this case) binds these two together.

This actually creates an interesting possibility. Since Lokad.CQRS is non-intrusive, it allows you to cross-host absolutely the same business logic in another application service. For example, you could be able to swap out Azure WorkerRole host for a Windows.Service instance with NServiceBus. We'll talk more about this later.

Let's get back to the sample. For the simplicity there is a base CloudEngineRole class, we could inherit our WorkerRole from.

public class WorkerRole : CloudEngineRole
{
  protected override ICloudEngineHost BuildHost()
  {
    return new CloudEngineBuilder()
    // this tells the server about the domain
      .Domain(d =>
        {
          d.InCurrentAssembly();
          d.WithDefaultInterfaces();
        })
      // we'll handle all messages incoming to this queue
      .HandleMessages(mc =>
        {
          mc.ListenTo("sample-01");
          mc.WithSingleConsumer();
        })
      // when we send message - default it to this queue as well
      .SendMessages(m => m.DefaultToQueue("sample-01"))
      .Build();
  }

  public override bool OnStart()
  {
    DiagnosticMonitor.Start("DiagnosticsConnectionString");
    // we send first ping message, when host starts
    WhenEngineStarts += SendFirstMessage;

    return base.OnStart();
  }

  static void SendFirstMessage(ICloudEngineHost host)
  {
    var sender = host.Resolve<IMessageClient>();
    var game = Rand.String.NextWord();
    sender.Send(new PingPongCommand(0, game));
  }
}

In the code above we configure our Lokad.CQRS Application Server (engine) with:

• Domain feature - core information about the domain that is used by the almost all the other components.
• HandleMessages role - server process that receives incoming messages and processes them properly.
• SendMessages feature - helper feature that makes it more simple to send messages for your code; it does this by exposing IMessageClient implementation to the IoC Container.

We additionally tell the server to send the first message, when it starts.

Given all this code, launching Sample_01 in Azure Dev Fabric should show you some pings running around.

DefaultCloudEngineHost: Starting host
ConsumingProcess: Starting consumption for Queue x 1 (sample-1)
AzureQueueTransport: Starting 1 threads to handle messages on sample-1
Queue[sample-1]: Auto-created queue http://.../sample-1
Queue[sample-1]: Auto-created poison queue http://.../sample-1-poison
Queue[sample-1]: Auto-created discard queue http://.../sample-1-discard
Queue[sample-1]: Auto-created blob storage http://../sample-1
Ping #0 in game 'stet'.
Queue[sample-1]: PingPongCommand - ad5cbcb2-e943-4cc2-a22b-e528c7fe4db9
Ping #1 in game 'stet'.
Queue[sample-1]: PingPongCommand - a80f5c38-2828-44b3-9de0-eb509f4dbb7c
Ping #2 in game 'stet'.

Note that:

• Local Development Storage is used by Lokad.CQRS engine, unless specified otherwise.
• All required queues (this includes discard and poison queues) are created automatically.
• Blob storage that was auto-generated, is used for sending messages larger than 6144 byte limit of Azure Queue Client.
• We use DataContractSerializer by default.

Summary

This article was from Learning series for Lokad.CQRS Guidance. We've talked about messages and their types: commands and events. There also was the first code sample of actually using Lokad.CQRS Application Server in Windows Azure to send and handle some messages.

In the next article on this subject we'll probably talk more about interfaces, listening to messages and the Scheduler Role of Lokad.CQRS application server.

You can stay tuned to the updates by subscribing to this Journal.

So what do you think about this article and the project? Do you like, how it's starting so far? Please, feel free to ask questions and submit feedback.

Comments

[Yuri Menkov]

Rinat,

Can you point to a correct client implementation that will feed messages into the service?
I guess I can use SEND MESSAGE sample but I'm more interested to see a real life client example.

Any ideas?

[Rinat Abdullin]

You've raised an interesting point here!

Technically using IMessageClient is correct. Architecturally and by design - it's too simple and loose for the real-life scenarios. We have a guidance and practice for that (built around ability to bind Lokad.CQRS to your message interfaces, defined in separate assembly). I'll try to post a sample on that in the next few days.

[Yuri Menkov]

Thank you for getting back to me.

Here is how I use the client in my tests.

CloudClient client = new CloudClientBuilder()
.CloudStorageAccountIsDev()
.Domain(d =>
{
d.WhereMessagesAre<DeltaOffers.Workflow.Commands.ResourceCommand>();
d.UseDataContractSerializer();
d.WithDefaultInterfaces();
}).BuildFor("resourceinbox");

DeltaOffers.Workflow.Commands.ResourceCommand cmd = new DeltaOffers.Workflow.Commands.ResourceCommand(pkey, rkey, keyword);

client.SendMessage(cmd);

I can pass multiple command types through one queue and it seems to work ok.
Since most of it based on trial and error I think a simple END to END example would clear things up.

On the Worker Roles here is what I have:

return new CloudEngineBuilder()
// this tells the server about the domain
.Domain(d =>
{
d.InAssemblyOf<DeltaOffers.Workflow.Commands.ResourceCommand>();
d.InAssemblyOf<DeltaOffers.Workflow.Commands.HttpHeadCommand>();
d.WithDefaultInterfaces();
})
// we'll handle all messages incoming to this queue
.HandleMessages(mc =>
{
mc.ListenTo("resourceinbox");
mc.WithMultipleConsumers();
mc.NumberOfThreads = 5;
})
// when we send message - default it to this queue as well
.SendMessages(m => m.DefaultToQueue("resourceoutbox"))
.Build();

This does process 2 different commands.

I'm curious to see the PRO / CONS on doing something like this in a real life application.

As usual your feedback is greatly appreciated

[Yuri Menkov]

Oh, and one more thing: How do I disable DEBUG trace on the WorkerRole.
I did not dig into code (just used the built DLLs)

[Yuri Menkov]

I know I'm getting greedy but I would like to TEST this in a regular WINDOWS SERVICE host.
I think I could use the QUEUE as a message transport between different machines (not needing the cloud worker role).

Any Ideas?

class WorkflowService : ServiceBase
{
private WorkerRole worker;

public WorkflowService()
{
InitializeComponent();

worker = new WorkerRole();

}
}

[Rinat Abdullin]

Hi Yuri,

Re Windows Service: Yes you can easily do that and don't need to create worker role. Just build the cloud engine with the builder class and start the created host on service start, while stopping it, when service stops. All threads and specifics will be handled internally.

This is not greedy at all. As long as you stick to the design guidelines and recommendations recommended by CQRS, even more interesting possibilities are enabled for the future. That's by the design.

Re your sample:

InAssemblyOf<> just tells the domain about the assemblies, where the commands are located. You don't need to point domain at every command. I would also recommend to start protoBuf serialization instead of data contracts, as shown in the last sample - it will help you to avoid many problems down the road (aside from better performance and cross-platform support).

Otherwise, your code just looks fine.

What exactly do you mean by END to END sample?

Re Trace log: I'll try to expose the configuration to change logger today. There will probably be a blog post on that later.

[Yuri Menkov]

Excellent! Thank you for clarifications.
We are thinking about taking dependancy on your project for one of our larger cloud applications.

Just to clarify: END to END = Best recommended route.
(Client shell with CONFIG OPTIONS explained and Service Scenario)
Like you mentioned here there are few things that you are recommending
that I think make sense (like protoBuf)

In Any case we are very excited to explore this.
Thank you very much for your excellent work!