Why can fixed size buffers only be of primitive types?

Question

We have to interop with native code a lot, and in this case it is much faster to use unsafe structs that don't require marshaling. However, we cannot do this when the structs contain fixed size buffers of nonprimitive types. Why is it a requirement from the C# compiler that fixed size buffers are only of the primitive types? Why can a fixed size buffer not be made of a struct such as:

[StructLayout(LayoutKind.Sequential)]
struct SomeType
{
  int Number1;
  int Number2;
}

Answer 1

Fixed size buffers in C# are implemented with a CLI feature called "opaque classes". Section I.12.1.6.3 of Ecma-335 describes them:

Some languages provide multi-byte data structures whose contents are manipulated directly by address arithmetic and indirection operations. To support this feature, the CLI allows value types to be created with a specified size but no information about their data members. Instances of these “opaque classes” are handled in precisely the same way as instances of any other class, but the ldfld, stfld, ldflda, ldsfld, and stsfld instructions shall not be used to access their contents.

The "no information about their data members" and "ldfld/stfld shall not be used" are the rub. The 2nd rule puts the kibosh on structures, you need ldfld and stfld to access their members. The C# compiler cannot provide an alternative, the layout of a struct is a runtime implementation detail. Decimal and Nullable<> are out because they are structs as well. IntPtr is out because its size depends on the bitness of the process, making it difficult for the C# compiler to generate the address for the ldind/stind opcode used to access the buffer. Reference types references are out because the GC needs to be able to find them back and can't by the 1st rule. Enum types have a variable size that depend on their base type; sounds like a solvable problem, not entirely sure why they skipped it.

Which just leaves the ones mentioned by the C# language specification: sbyte, byte, short, ushort, int, uint, long, ulong, char, float, double or bool. Just the simple types with a well defined size.

Answer 2

What is a fixed buffer?

From MSDN:

In C#, you can use the fixed statement to create a buffer with a fixed size array in a data structure. This is useful when you are working with existing code, such as code written in other languages, pre-existing DLLs or COM projects. The fixed array can take any attributes or modifiers that are allowed for regular struct members. The only restriction is that the array type must be bool, byte, char, short, int, long, sbyte, ushort, uint, ulong, float, or double.

I'm just going to quote Mr. Hans Passant in regards to why a fixed buffer MUST be unsafe. You might see Why is a fixed size buffers (arrays) must be unsafe? for more information.

Because a "fixed buffer" is not a real array. It is a custom value type, about the only way to generate one in the C# language that I know. There is no way for the CLR to verify that indexing of the array is done in a safe way. The code is not verifiable either. The most graphic demonstration of this:

using System;

class Program {
    static unsafe void Main(string[] args) {
        var buf = new Buffer72();
        Console.WriteLine(buf.bs[8]);
        Console.ReadLine();
    }
}
public struct Buffer72 {
    public unsafe fixed byte bs[7];
}

You can arbitrarily access the stack frame in this example. The standard buffer overflow injection technique would be available to malicious code to patch the function return address and force your code to jump to an arbitrary location.

Yes, that's quite unsafe.

Why can't a fixed buffer contain non-primitive data types?

Simon White raised a valid point:

I'm gonna go with "added complexities to the compiler". The compiler would have to check that no .NET specific functionality was applied to the struct that applied to enumerable items. For example, generics, interface implementation, even deeper properties of non-primitive arrays, etc. No doubt the runtime would also have some interop issues with that sort of thing too.

And Ibasa:

"But that is already done by the compiler." Only partly. The compiler can do the checks to see if a type is managed but that doesn't take care of generating code to read/write structs to fixed buffers. It can be done (there's nothing stopping it at CIL level) it just isn't implemented in C#.

Lastly, Mehrdad:

I think it's literally because they don't want you to use fixed-size buffers (because they want you to use managed code). Making it too easy to interop with native code makes you less likely to use .NET for everything, and they want to promote managed code as much as possible.

The answer appears to be a resounding "it's just not implemented".

Why's it not implemented?

My guess is that the cost and implementation time just isn't worth it to them. The developers would rather promote managed code over unmanaged code. It could possibly be done in a future version of C#, but the current CLR lacks a lot of the complexity needed.

An alternative could be the security issue. Being that fixed buffers are immensely vulnerable to all sorts of problems and security risks should they be implemented poorly in your code, I can see why the use of them would be discouraged over managed code in C#. Why put a lot of work into something you'd like to discourage the use of?

Answer 3

I understand your point of view...on the other hand I suppose that it could be some kind of forward compatibility reserved by Microsoft. Your code is compiled to MSIL and it is bussiness of specific .NET Framework and OS to layout it in memory.

I can imagine that it may come new CPU from intel which will require to layout variables to every 8 bytes to gain the optimal performance. In that case there will be need in future, in some future .NET Framework 6 and some future Windows 9 to layout these struct in different way. In this case, your example code would be pressure for Microsoft not to change the memory layout in the future and not speed up the .NET framework to modern HW.

It is only speculation...

Did you tried to set FieldOffset? See C++ union in C#