Why does the shortcut used by BitConverter when the start index is divisible by the size of the type being converted to work?

Question

I've recently been looking into how BitConverter works and from reading other SO questions I've read that it takes a 'shortcut' when the start index is divisible by the size of the type being converted to where it can just cast a pointer the byte at the index into a pointer to the type being converted to and de-reference it.

Source for ToInt16 as an example:

public static unsafe short ToInt16(byte[] value, int startIndex) {
     if( value == null)  {
          ThrowHelper.ThrowArgumentNullException(ExceptionArgument.value);
     }

     if ((uint) startIndex >= value.Length) {
          ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument.startIndex, ExceptionResource.ArgumentOutOfRange_Index);
     }

     if (startIndex > value.Length -2) {
          ThrowHelper.ThrowArgumentException(ExceptionResource.Arg_ArrayPlusOffTooSmall);
     }
     Contract.EndContractBlock();

     fixed( byte * pbyte = &value[startIndex]) {
          if( startIndex % 2 == 0) { // data is aligned 
              return *((short *) pbyte);
          }
          else {
              if( IsLittleEndian) { 
                   return (short)((*pbyte) | (*(pbyte + 1) << 8)) ;
              }
              else {
                   return (short)((*pbyte << 8) | (*(pbyte + 1)));                        
              }
          }
     }
}

My question is why does this work regardless of the endianness of the machine, and why doesn't it use the same mechanism when the data is not aligned?

An example to clarify:

I have some bytes in buffer that I know are in Big endian format, and I want to read a short value from the array at say, index 5. I also assume that my machine, since it is Windows, uses little endian.

I would use BitConverter like so, by switching the order of my bytes to little endian:

BitConverter.ToInt16(new byte[] { buffer[6], buffer[5] })

assuming the code takes the shortcut it would do what I want: just cast the bytes as they are in the order provided and return the value. But if it didn't have that shortcut code, wouldn't it then reverse the byte order again and give me the wrong value? Or if I instead did:

BitConverter.ToInt16(new byte[] { 0, buffer[6], buffer[5] }, 1)

wouldn't it give me the wrong value since the index is not divisible by 2?

Another situation:

Say I had an array of bytes that contained an short somewhere I want to extract already in little endian format, but starting at an odd offset. Woulnd't the call to BitConverter reverse the order of the bytes since BitConverter.IsLittleEndian is true and the index is not aligned, thus giving me an incorrect value?

Answer 1

The code avoids a hardware exception on processors that don't allow misaligned data access, a bus error. Which is very expensive, it is usually resolved by kernel code that splits up the bus accesses and glues the bytes together. Such processors were still pretty common around the time that this code was written, the tail-end of the popularity of RISC designs like MIPS. Older ARM cores and Itanium are other examples, .NET versions have been released for all of them.

It makes little difference on processors that don't have a problem with it, like Intel/AMD cores. Memory is slow.

The code uses IsLittleEndian simply because it is indexing the individual bytes. Which of course makes the byte order matter.

Answer 2

On most architecutre there is a performance hit in accessing data that isn't aligned at the proper boundary. On x86 the CPU will allow you to read from an unaligned address, but there will be a performance hit. On some architecture you'll get a CPU fault that the operating system will trap.

I'd guess that the cost of letting the CPU fix up the reading of unaligned data is greater than the cost of reading the individual bytes and doing the shift/or operations. Also, the code is now portable to platforms where an unaligned read will cause a fault.

Answer 3

Why does this work regardless of the endianness of the machine?

The method does a re-interpretation of bytes assuming that they were produced in the environment with the same endianness. In other words, endianness influences both the the order the input bytes are arranged in the array, and the order the bytes need to be arranged in the output short in the same way.

Why doesn't it use the same mechanism when the machine is Big Endian?

This is an excellent observation, and it is not immediately obvious why the authors didn't do the cast. I think the reason behind it is that if you cast a pbyte with an odd value to short*, the subsequent access to short would be unaligned. This requires a special opcode to prevent a hard exception, which some platforms generate on unaligned access.