Reducing the sizeof a vector using custom allocator?

Question

I'm using Linux on a 64-bit machine and I use g++4.8. The sizeof operator applied on a std::vector returns :

sizeof(std::vector<float>) = 24
sizeof(std::vector<double>) = 24
sizeof(std::vector<long double>) = 24

My question is : is there any way to create a vector with a smaller size based on the fact that my application will never need vectors of more than 1 GB of elements ? Because optimally, I could have a vector of 16 bytes : the pointer to the beginning (8 bytes), the current size (4 bytes) and the current capacity (4 bytes). Do I have to recode my own vector from scratch or can I reduce the size with a custom allocator ?

This size is very critical for me as I work on supercomputers with more than 100 TB of memory, and every byte I can gain on my basic classes could finally save several TB of memory.

Answer 1

No, you can't.

The original idea with allocators was that they could define objects for pointers and reference types to the data. However, that's not really allowed anymore; allocator<T>::pointer must be T*, and containers are freely allowed to assume that it is.

So if you truly need a vector to use some kind of compact pointer representation or something, you're going to have to implement one yourself.

The standard library containers are good defaults for most uses. But for those with specialized needs, you may have to replace them with something else. That would appear to be the case here.

Answer 2

Vectors are traditionally implemented using three pointers (begin, end, and end of storage). The only way to shrink them is to change the internal representation indeed.

Answer 3

This size is built in to the vector's implementation, and you can't reduce that conveniently (building in 32-bit mode would of course reduce the size due to smaller pointer sizes).

Perhaps if you're using trillions of vectors in your code you might want to try to reduce that number (combine them into a larger vector?) rather than trying to shave a couple bytes off each one.

Answer 4

There is something to clear up: vector size and capacity data become relatively very very small when vector size grows.

Lets say you have a double vector of size 1 million. Size and capacity data will still occupy 8 bytes in total and it is quite minute compared to the main data the vector holds.

If you frequently use very small sized vectors like from 1 to 50, then you can write a wrapper to arrays and use it and std vectors may not be a good choice for small sized data groups . I do not know whether your RAM is 100TB or it is just your hard drive capacity but keeping many small sized data structures may not be a good idea while it can result in severe memory fragmentations.