Scalability of variadic templates

Question

I am working on a large-scale software infrastructure in C++11 that makes extensive use of variadic templates. My question is the following: what is the scalability of this approach? First, is there an upper limit in the number of arguments that variadic templates can take? Second, is code bloat a major issue with state-of-the-art compilers when many arguments are used (and, by extension, many combinations of these arguments that would yield to many different implementations of the templated methods)?

Answer 1

I thought I'd have a go at finding out whether there is any limit on the number of template parameters for my particular compiler (g++ 4.8.1 on Linux). I used the following test case:

template <int I>
struct this_is_a_ridiculously_long_struct_name_youd_never_use_in_real_life {};

template <int Depth, typename... T>
struct A
{
    using e = this_is_a_ridiculously_long_struct_name_youd_never_use_in_real_life<Depth>;

    A() {};

    A<Depth - 1, e, e, e, e, e, e, e, e, e, e, e, e, e, e, e, e, e, e, e, e, T...> a;
};

template <typename... T>
struct A<0, T...>
{
};

int main()
{
    A<899> a;
}

The template recursion depth limit defaults to 900 in g++, hence the 899 parameter you see. The ridiculously long struct name is used to see whether I could generate any symbols which were too big for the linker to handle -- more on this in a sec.

In case you can't see what's going on in the test case, basically every instantiation of A creates a member variable which adds 20 extra template parameters. Partial specialisation is used to stop the recursion. By the end, A<0, ...> has somewhere in the region of 18000 template parameters.

What I found was that g++ handled this just fine. It took quite a while to think about it, and used a fair bit of memory, but I was unable to get it to fail simply by increasing the number of template parameters. Clang 3.1 also handled this without any problem once the template recursion depth was set sufficiently (i.e. 900).

Furthermore, although the mangled symbol names do indeed become huge, I was unable to break either nm or ld using them. (It's worth noting that the Linux/Itanium mangling scheme uses substitution so that repeated template parameters of the same type don't repeat the whole type name, but rather are marked S0, S1 etc.) A quick google doesn't seem to turn up any limit on ELF symbol length, but perhaps someone else knows whether such a limit exists.

In conclusion then, for g++ and clang on Linux at least, there doesn't seem to be any practical limit on the number of template parameters.

As to the second part of your question, regarding code bloat, it's very hard to say, particularly once compiler optimisation gets involved. It's easy to do recursion with variadic templates, but then it's easy for the compiler to get rid of intermediate types too. I can only suggest to try it and see.