STL Containers and Binary Interface Compatibility

Question

STL Binary Interfaces

I'm curious to know if anyone is working on compatible interface layers for STL objects across multiple compilers and platforms for C++.

The goal would be to support STL types as first-class or intrinsic data types.

Is there some inherent design limitation imposed by templating in general that prevents this? This seems like a major limitation of using the STL for binary distribution.

Theory - Perhaps the answer is pragmatic

1. Microsoft has put effort into .NET and doesn't really care about C++ STL support being "first class".

2. Open-source doesn't want to promote binary-only distribution and focuses on getting things right with a single compiler instead of a mismatch of 10 different versions.

This seems to be supported by my experience with Qt and other libraries - they generally provide a build for the environment you're going to be using. Qt 4.6 and VS2008 for example.

References:

• http://code.google.com/p/stabi/
• Binary compatibility of STL containers

Answer 1

I think the problem preceeds your theory: C++ doesn't specify the ABI (application binary interface).

In fact even C doesn't, but being a C library just a collection of functions (and may be global variables) the ABI is just the name of the functions themselves. Depending on the platform, names can be mangled somehow, but, since every compiler must be able to place system calss, everything ends up using the same convention of the operating system builder (in windows, _cdecl just result in prepending a _ to the function name.

But C++ has overloading, hence more complex mangling scheme are required. As far as of today, no agreement exists between compiler manufacturers about how such mangling must be done. It is technically impossible to compile a C++ static library and link it to a C++ OBJ coming from another compiler. The same is for DLLs.

And since compilers are all different even for compiled overloaded member functions, no one is actually affording the problem of templates.

It CAN technically be afforded by introducing a redirection for every parametric type, and introducing dispatch tables but ... this makes templated function not different (in terms of call dispatching) than virtual functions of virtual bases, thus making the template performance to become similar to classic OOP dispatching (although it can limit code bloating ... the trade-off is not always obvious)

Right now, it seems there is no interest between compiler manufacturers to agree to a common standard since it will sacrifice all the performance differences every manufacturer can have with his own optimization.

Answer 2

C++ templates are compile-time generated code.
This means that if you want to use a templated class, you have to include its header (declaration) so the compiler can generate the right code for the templated class you need.

So, templates can't be pre-compiled to binary files.

What other libraries give you is pre-compiled base-utility classes that aren't templated.

C# generics for example are compiled into IL code in the form of dlls or executables.
But IL code is just like another programming language so this allows the compiler to read generics information from the included library.

.Net IL code is compiled into actual binary code at runtime, so the compiler at runtime has all the definitions it needs in IL to generate the right code for the generics.

Answer 3

I'm curious to know if anyone is working on compatible interface layers for STL objects across multiple compilers and platforms for C++.

Yes, I am. I am working on a layer of standardized interfaces which you can (among other things) use to pass binary safe "managed" references to instances of STL, Boost or other C++ types across component boundaries. The library (called 'Vex') will provide implementations of these interfaces plus proper factories to wrap and unwrap popular std:: or boost:: types. Additionally the library provides LINQ-like query operators to filter and manipulate contents of what I call Range and RangeSource. The library is not yet ready to "go public" but I intend to publish an early "preview" version as soon as possible...

Example:

com1 passes a reference to std::vector<uint32_t> to com2:

com1:

class Com2 : public ICom1 {
    std::vector<int> mVector;

    virtual void Com2::SendDataTo(ICom1* pI)
    {
        pI->ReceiveData(Vex::Query::From(mVector) | Vex::Query::ToInterface());
    }
};

com2:

class Com2 : public ICom2 {
    virtual void Com2::ReceiveData(Vex::Ranges::IRandomAccessRange<uint32_t>* pItf)
    {
        std::deque<uint32_t> tTmp;
        // filter even numbers, reverse order and process data with STL or Boost algorithms
        Vex::Query::From(pItf)
            | Vex::Query::Where([](uint32_t _) -> { return _ % 2 == 0; })
            | Vex::Query::Reverse
            | Vex::ToStlSequence(std::back_inserter(tTmp));
        // use tTmp...
    }
};

You will recognize the relationship to various familiar concepts: LINQ, Boost.Range, any_iterator and D's Ranges... One of the basic intents of 'Vex' is not to reinvent wheel - it only adds that interface layer plus some required infrastructures and syntactic sugar for queries.

Cheers,

Paul