compliant variable length struct in C++

Question

In standard C you can end a struct with an array of size 0 and then over allocate it to add a variable length dimension to the array:

struct var
{
    int a;
    int b[];
}

struct var * x=malloc(sizeof(var+27*sizeof(int)));

How can you do that in C++ in a standard (portable) way? It is okay to have a constraint of max posible size and obviously doesn't have to work on the stack

I was thinking of:

class var
{
...
private:
  int a;
  int b[MAX];
};

and then use allocators or overload new/delete to under allocate based on the required size:

(sizeof(var) - (MAX-27)*sizeof(int)

But, while it seems to work, its not something I'd want to have to maintain.

Is there a cleaner way that is fully standard/portable?

Answer 1

What's wrong with simply doing a variant of the C way?

If the structure has to remain purely POD, the C way is fine.

struct var
{
    int a;
    int b[1];

    static std::shared_ptr<var> make_var(int num_b) {
        const extra_bytes = (num_b ? num_b-1 : 0)*sizeof(int);
        return std::shared_ptr<var>(
                new char[sizeof(var)+extra_bytes ],
                [](var* p){delete[]((char*)(p));});
}

since it's a POD, everything works just like it did in C.

---

If b is not guaranteed to be POD, then things get more interesting. I haven't tested any of it, but it would look more or less like so. Note that make_var relies on make_unique, because it uses a lambda destructor. You can make it work without this, but it's more code. This works just like the C way, except it cleanly handles variable amounts of types with constructors and destructors, and handles exceptions

template<class T>
struct var {
    int a;

    T& get_b(int index) {return *ptr(index);}
    const T& get_b(int index) const {return *ptr(index);}

    static std::shared_ptr<var> make_var(int num_b);
private:
    T* ptr(int index) {return static_cast<T*>(static_cast<void*>(&b))+i;}
    var(int l);
    ~var();
    var(const var&) = delete;
    var& operator=(const var&) = delete;

    typedef typename std::aligned_storage<sizeof(T), std::alignof(T)>::type buffer_type;
    int len;
    buffer_type b[1];
};
template<class T> var::var(int l)
    :len(0)
{
    try {
        for (len=0; len<l; ++len)
            new(ptr(i))T();
    }catch(...) {
        for (--len ; len>=0; --len)
            ptr(i)->~T();
        throw;
    }
}
template<class T> var::~var()
{
    for ( ; len>=0; --len)
        ptr(i)->~T();
}
template<class T> std::shared_ptr<var> var::make_var(int num_b)
{
    const extra_bytes = (num_b ? num_b-1 : 0)*sizeof(int);
    auto buffer = std::make_unique(new char[sizeof(var)+extra_bytes ]);
    auto ptr = std::make_unique(new(&*buffer)var(num_b), [](var*p){p->~var();});
    std::shared_ptr<var> r(ptr.get(), [](var* p){p->~var(); delete[]((char*)(p));});
    ptr.release();
    buffer.release;
    return std::move(r);
}

Since this is untested, it probably doesn't even compile, and probably has bugs. I'd normally use std::unique_ptr but I'm too lazy to make proper standalone deleters, and unique_ptr is hard to return from a function when the deleter is a lambda. On the off chance you want to use code like this, use a proper standalone deleter.

Answer 2

While this is not directly answering your question -- I would point to that a better practice in C++ is to use the STL lib for this sort of variable length array -- it is safe and simpler and understood by anybody who will maintain it after you.

class var
{
...
private:
  int a;
  std::vector<int> b; // or use std::deque if more to your liking
};

Now you can just new it up like any other class;

var* myvar = new var;

And you can use it just like a old type array without explicitly allocating the memory (although that is not what most ++ programmers do)

myvar->b[0] = 123;
myvar->b[1] = 123;
myvar->b[2] = 123;

Answer 3

Yes, you can, although you can't declare it as an array member. You can use a reference:

struct s {
    int ( & extra_arr )[];

    s() : extra_arr( reinterpret_cast< int (&)[] >( this[1] ) {}
};

In practice this will use a pointer's worth of storage, although in theory it doesn't need to. This class is not POD, attributable to that difference between theory and practice.

---

You can alternately put the reinterpret_cast into a nonstatic member function:

struct s {
    int ( & get_extra() )[]
        { return reinterpret_cast< int (&)[] >( this[1] ); }

    int const ( & get_extra() const )[]
        { return reinterpret_cast< int const (&)[] >( this[1] ); }
};

Now access requires function call syntax (inlining will eliminate the distinction in machine code besides the debug build), but there is no wasted storage and the object will be POD barring some other exception to the POD rules.

With a little ABI tweaking such as #pragma pack this can get you full C binary compatibility. Often such tweaking is needed for serialization applications anyway.

Also this one supports const-correctness, whereas the previous solution allows modification of a const object (because it's unaware that the array is part of the same object).

The boilerplate can be generalized into a CRTP base class (which in C++11 even still allows the derived class to be POD), or a preprocessor macro expanding to define either the C++ accessor or the C flexible member.

---

Note that neither of these solutions does anything more than the original C. The special member functions will not copy the flexible array, and the class cannot support a function parameter or subobject.

Answer 4

template <size_t MAX>
class var
{
   ...
private:
  int a;
  int b[MAX];
};

In each template instantiation MAX is a constant that can be used on loops. Then you can construct vars whith any length.

var<7> v7;
var<100> v100;

Or typedef them

typedef var<10> myVar;

Answer 5

A much cleaner way is to use inheritance:

class Parent
{
  public:
    virtual int get_b(unsigned int index) = 0;
  protected:  
    int a;
};

class Child1
: public Parent
{
  public:
    int get_b(unsigned int index)
    {
        return b[index];  // Should have index bounds checking.
    }
  private:
    int b[20];
};

The inheritance allows you to adjust the size and quantity of members of the Parent class.

Answer 6

Ok - (not putting this in question since I'm not sure about it) Since it seems to me from the current answers that there is currently no better way than over allocation, I was wondering if this would help maintenance wise:

template <class BASE, class T>
class dynarray
{
public:
    BASE base;
    const size_t size;
    T data[1]; // will be over allocated

    static dynarray * create(size_t data_size)
    {
        return new(data_size) dynarray(data_size);
    }
    void operator delete(void *p)
    {
        ::operator delete(p);
    }
private:
    void * operator new (size_t full_size, size_t actual)
    {
        if (full_size != sizeof(dynarray))
        {
            // inheritence changed size - allocate it all
            return ::operator new(sizeof(dynarray));
        }

        return ::operator new(sizeof(dynarray) + (actual-1)*sizeof(T));
    }
    void operator delete(void *p, size_t) // matching delete
    {
        ::operator delete(p);
    }
    dynarray(size_t data_size) : size(data_size)
    {
    }
};

usage is a bit stilted, but possibly better though:

typedef dynarray<double,int,27> dyn;
dyn * x=dyn::create(7);
x->data[5]=28;
x->base=5.3;

EDIT: changed implementation from under alloaction to over allocation

Answer 7

another way to do that is to use placement new

#include <cstdlib>

class var
{
    ...
private:
    int a;
    int b[1];
};

var * x = new(malloc(sizeof(var) + (27-1)*sizeof(var::b))) var;

in this case, contructor is called on allocated memory

to delete the structure use:

x->~var(); // only if var have a destructor
free(x);

or better, add a delete operator to var and use delete:

struct var {
    ...
    operator delete(void* ptr) throw() { free(ptr); }
};
var * x = ...
delete x;

The best and proper approach is to use a static function to create instances and to privatize the constructor: class var { public: ... static var * create(int size, ) throw() { new(malloc(sizeof(var) + (27-1)*sizeof(b)))) var(); } void operator delete(void *ptr) { free(ptr); }

private:
    int a;
    int b[1];
    var(<args>) { ... }
};

var * x = var::create(27);
delete x;

note: i use an array of size 1 because undeffined and 0 sized arrays are not supported by all compilators.

Answer 8

There was a defect report filed on this here for C: http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_051.html

The response is that declaring the array to be the largest size you will need (or actually you can also do the maximum possible size for an integer as well) is a "safer idiom" and that it is strictly conforming. The idea is that, instead of over allocating and thus running beyond the declared size of the array, you actually under allocate and only access memory inside the declared bounds of the array.

This should apply to C++ as well as long as it didn't change those rules, which it shouldn't have because it's meant to be pretty much compatible with C. If someone knows something C++-specific that invalidates this solution, please inform me.

As long as you hide this implementation behind a well-defined interface, there shouldn't be any maintenance problems.