What's the best way to represent System.Decimal in Protocol Buffers?

Question

Following on from this question, what would be the best way to represent a System.Decimal object in a Protocol Buffer?

Answer 1

Well, protobuf-net will simply handle this for you; it runs off the properties of types, and has full support for decimal. Since there is no direct way of expressing decimal in proto, it won't (currently) generate a decimal property from a ".proto" file, but it would be a nice tweak to recognise some common type ("BCL.Decimal" or similar) and interpret it as decimal.

As for representing it - I had a discussion document on this (now out of date I suspect) in the protobuf-net wiki area; there is now a working version in protobuf-net that simply does it for you.

No doubt Jon and I will hammer this out more later today ;-p

The protobuf-net version of this (in .proto) is something like (from here):

message Decimal {
  optional uint64 lo = 1; // the first 64 bits of the underlying value
  optional uint32 hi = 2; // the last 32 bis of the underlying value
  optional sint32 signScale = 3; // the number of decimal digits, and the sign
}

Answer 2

Marc and I have very vague plans to come up with a "common PB message" library such that you can represent pretty common types (date/time and decimal springing instantly to mind) in a common way, with conversions available in .NET and Java (and anything else anyone wants to contribute).

If you're happy to stick to .NET, and you're looking for compactness, I'd possibly go with something like:

message Decimal {

    // 96-bit mantissa broken into two chunks
    optional uint64 mantissa_msb = 1;
    optional uint32 mantissa_lsb = 2;

    required sint32 exponent_and_sign = 3;
}

The sign can just be represented by the sign of exponent_and_sign, with the exponent being the absolute value.

Making both parts of the mantissa optional means that 0 is represented very compactly (but still differentiating between 0m and 0.0000m etc). exponent_and_sign could be optional as well if we really wanted.

I don't know about Marc's project, but in my port I generate partial classes, so you can the put a conversion between System.Decimal and Protobuf.Common.Decimal (or whatever) into the partial class.

Answer 3

I put together a patch for protobuf-csharp-port with hooks which generates protobuf classes with native Decimal and DateTime structs. Wire format wise, they are represented by two "built-in" proto messages.

Here is the link: https://code.google.com/p/protobuf-csharp-port/issues/detail?can=2&start=0&num=100&q=&colspec=ID%20Type%20Status%20Priority%20Milestone%20Owner%20Summary&groupby=&sort=&id=78

Answer 4

A slightly simpler to implement approach than Jon or Marc's is to store it as 4 sint32 values, which conveniently maps trivially to the output of Decimal.GetBits().

The proto file will look like:

message ProtoDecimal {
    sint32 v1 = 1;
    sint32 v2 = 2;
    sint32 v3 = 3;
    sint32 v4 = 4;
}

And the converter will be:

public decimal ConvertToDecimal(AbideDecimal value)
{
    return new decimal(new int[] { value.V1, value.V2, value.V3, value.V4 });
}

public ProtoDecimal ConvertFromDecimal(decimal value)
{
    var bits = decimal.GetBits(value);
    return new ProtoDecimal 
    {
        V1 = bits[0],
        V2 = bits[1],
        V3 = bits[2],
        V4 = bits[3]
    }
}

This might not be as simple in other languages, but if you only have to target C# then it will take up the same maximum of 16 bytes that the other approach will (although values like 0 might not be as compactly stored - I don't know enough about the intricate details of how protobuf stores ints), while being much clearer to dumb-dumb programmers like me :)

Obviously you will have to race the horses if you want to test performance but I doubt there is much in it.