Azure Queue Messages cannot be larger than 8192 bytes

We all know, that Windows Azure Queues are even more limited in size than MSMQ Queues. And the maximum size of a message is 8Kb.

Well, not quite.

If you are using Windows Azure Queue Client Library (Microsoft.WindowsAzure.StorageClient), then it will exhibit an interesting behavior. Take a look at the simple code:

using (var stream = new MemoryStream())
{
  _messageSerializer.Serialize(data, stream);
  if (stream.Position < 8192)
  {
    var bytes = stream.ToArray();
    // we should not get any exception here
    return new CloudQueueMessage(bytes);
  }
  // else save overflowing part to the Azure BLOB
}

If you try to save a byte buffer of size between 6144 and 8192 bytes, then azure queues will throw up an exception:

System.ArgumentException: Messages cannot be larger than 8192 bytes.
  at Microsoft.WindowsAzure.StorageClient.CloudQueueMessage..ctor(Byte[] content)

This does not make a lot of sense, right? What this exception actually means is:

Messages cannot be larger than 6144 bytes

If we look into the original DLLs, reason becomes obvious. Azure Client code does not really check for the byte size, it checks for the size of the message, after applying Base64 Encoding (which adds 4/3 overhead):

if (Convert.ToBase64String(content).Length > MaxMessageSize)
{
    throw new ArgumentException(...);
}

So that's definitely one potential improvement point for the Azure Client Libraries just in the constructor of CloudQueueMessage - making the exception text more clear and dropping the overhead of converting to Base64 just for the sake or performing a check.

PS: While we are at the subject of wishes for Azure, there was another one, that could significantly improve productivity of cloud solutions: MSMQ Azure (this especially applies to the cloud scalable solutions built with messaging and CQRS).

Comments

[Joannes Vermorel]

Christoph and me, already hit the same problem twice for Lokad.Cloud. You're now the third :-)

[Rinat Abdullin]

This just proves, how intuitive this exception message is))

[Steve Marx]

Queue messages have to be strings that can be safely embedded in XML. The .NET storage client library converts whatever you store to Base64 to make sure it meets this requirement. The resulting string (base64 encoded) is what can't exceed 8KB. I agree the exception should be clearer, and I'll notify the team that owns it.

[Christoph Rüegg]

The MaxMessageSize field is misleading the same way as well. The documentation describes it as "The maximum message size in bytes", without warning that the "message" is not the raw byte array but some transformed representation of it. Since byte[] is explicitly supported, maybe an additional field for the maximum byte length would help.