How can I create a std::vector with 64 bit indexes?
-
20-09-2019 - |
Question
I want to create a big std::vector
so operator[]
should receive long long
rather than unsigned int
, I tried writing my own allocator:
template <typename T>
struct allocator64 : std::allocator<T> {
typedef long long difference_type;
typedef unsigned long long size_type;
};
But when I try the following:
long long n = 5;
std::vector<int, allocator64<int> > vec(n);
vec[n-1] = 2;
I get the following warning for the second and third line:
warning C4244: 'argument' : conversion from '__int64' to 'unsigned int', possible loss of data
What am I missing? I thought the type for operator[]
and for the size constructor should come from allocator::size_type
.
I'm using VS9 (2008).
Solution
Maybe the STXXL library can help:
STXXL provides an STL replacement using an abstraction layer to storage devices to allow for the optimal layout of data structures. This allows for multi-terabyte datasets to be held and manipulated in standard C++ data structures, whilst abstracting the complexity of managing this behaviour efficiently. STXXL utilises multi-disk I/O to speed up I/O bound calculations. STXXL has been developed at the University of Karlsruhe.
OTHER TIPS
Do you really need to store more than numeric_limits<unsigned int>::max()
entries in the vector? O.o
If not then just cast n-1 to int explicitly: int(n-1)
Anyway, the argument is of type vector::size_type
(and not allocator::size_type, as far as I know) which is usually a typedef to size_t (but needs not be); it may be 4 bytes or 8 or other amounts; it is implementation defined.
Also see MSDN
The implementation of vector provided with your compiler already uses the largest integer type that it usefully can[*], usually size_t
. Suppose you're on a machine with a 32bit address space, and you say you want a vector with 2^33 elements. There's no way that can be allocated, and changing the parameter type of operator[]
would not give your computer the ability to allocate bigger blocks of memory.
So, make sure that you're compiling your code for 64bit Windows, and all should be well.
[*] That's not guaranteed by the standard. But compiler-writers don't actually enjoy putting in limits for no reason, even if it sometimes looks that way.