NServiceBus and Windows Azure Challenges

Cloud is going to be one of the next big things for NServiceBus. That's what Udi Dahan had mentioned in one of his recent podcast talks.

Indeed, there already are changes allowing to host NServiceBus within the worker role, while using Azure Queues as the messaging transport (we are replacing the MSMQ here).

Yet, this approach might have a few inherent problems.

First of all, Azure Queue Messages can't be larger than 8Kb. So in order for NServiceBus to support existing functionality (MSMQ has 4Mb limit) Azure Blob Storage will have to be used to keep the payload. That's how Lokad.Cloud handles persistence of large objects.

Yet, this brings us to another possible failure. MSMQ is a local file queue. This works around the unreliable network connections, while still ensuring guaranteed delivery. Azure, on the other hand, does not have such capabilities by design. Exception handling reliability policies built into the Azure Storage Client merely feature "retry actions" without any "guaranteed" or "transactional" delivery.

Third problem is the actual transactional model of the Azure Queues. They were build for the distribution and high availability. Consistency suffers a little bit in favor of more scalable lease policies (while checking out a message you get some time to delete it, or it'll be returned to the queue). This lightweight approach leaves some possibility that the message could be processed twice. And generally we don't want to have a single chance that "ChargeCustomerCommand" would be executed twice.

Although Azure Queues are not fit for the reliable enterprise integration scenarios out-of-the box, they are perfectly fit for the work load distribution scenarios, where transaction boundaries and consistency are not that important.

So it will be interesting to watch how NServiceBus team works around these limitations in order to deliver reliable enterprise communications framework for the cloud.

Comments

[Joannes Vermorel]

FYI, queue messages on Azure are now kept indefinitely (you are charged for the corresponding storage), and no more deleted after 7 days.

[Rinat Abdullin]

@Joannes, thanks for the correction!