How do I unpack a number larger than 64 bits in ruby?

Question

Let's say I have an arbitrary string that's ~1000 bytes long. (I'm studying crypto.) How can I unpack that into a BigNum? I understand how to pack it into 8-bit numbers, say, or 32-bit numbers.

s='I am a grumpy potato'
s.unpack('C*')
[73, 32, 97, 109, 32, 97, 32, 103, 114, 117, 109, 112, 121, 32, 112, 111, 116, 97, 116, 111]
s.upack('L*')
=> [1835081801, 1730175264, 1886221682, 1869619321, 1869898100]

Or, is there a straightforward way to combine, say, the 8-bit numbers into a BigNum? I could probably unpack into an array of 8-bit numbers and multiply each element of the array by subsequent powers of 8. But that seems too complicated to be the right way.

EDIT: What's the preferred way to turn the BigNum back into a string? I don't mean to_s, I mean taking the same pattern of bytes and interpreting it as a string?

Answer 1

I think your hunch about how to handle it is right. Unpack doesn't handle Bignums; they're classically fairly tricky specifically because they don't fit in a standard 64-bit int.

You could manually "unpack" it, via something like:

str.unpack("C*").reverse.each_with_index.inject(0) do |sum, (byte, index)|
  sum + byte * (256 ** index)
end

That is, reverse the list of bytes (if on a big endian system), iterate each byte, and multiply its value by 256^position. Ruby's BigNum kicks in once the value gets big enough, and you're able to convert byte strings into very large numbers without a hitch.

You can do the same in chunks of 32-bit (or 64-bit, if the platform supports it) ints, as well:

INT32_MAX = 256 ** [1].pack("L*").size
INT64_MAX = 256 ** [1].pack("Q*").size

str.unpack("L*").reverse.each_with_index.inject(0) do |sum, (byte, index)|
  sum + byte * (INT32_MAX ** index)
end

str.unpack("Q*").reverse.each_with_index.inject(0) do |sum, (byte, index)|
  sum + byte * (INT64_MAX ** index)
end

Answer 2

Thanks a lot for https://stackoverflow.com/a/17014450/204070. Works wonderful for me. The answer could be simplified to:

str.unpack("C*").inject(0) do |sum, (byte, index)|
  sum * 256 + byte
end