Language independent software libraries

Question

When trying to decide on the choice of a programming language for a specific task, people will often tell me: "use this language because it has the best library for the type of problems you are addressing".

I would rather choose my language for its intrinsic qualities as a programming medium. That implies that libraries should be more universally usable.

What work has there been to create programming libraries that could be programmed and used in different languages, being mechanically adapted from one to the other on the basis of a standardized interface language/formalism (or any other mean)? How succesful has it been?

For example, I found one system that claims to do that : Microsoft's .NET Framework. They say it is based on a Common Language Specification (CLS) which is defined in the ECMA-335 Standard: Common Language Infrastructure.

Are there others? What about free-software frameworks?

Pointers to relevant papers or web sites are an answer. A survey would be best.

Note (June 7, 2013): this question is more controversial than is visible. However, I cannot fathom the nature of the controversy. Some people support the question and/or make cogent comments. But the people who seem unhappy about it just downvote without a word of explanation. I am sure they have a good reason to do so. But it would better help our understanding of the issues, or possibly of the lack of issues, if they were a little more explicit as to what irks them.

Answer 1

Java Bytecode

Similar to Microsoft's CLS, the Java Bytecode that the Java virtual machine executes gives you the (theoretical) possibility of using libraries from one JVM-targeting language with another JVM-targeting language. For example, Java libraries can be used in Scala, which IMO is a much better language than Java itself. Libraries written in Scala could be used in Clojure, or in Jython, or in one of these languages.

However, although interoperability is theoretically possible because of the common byte code to which all these languages compile, it is said (my personal experience is fairly limited) to sometimes be complicated to actually get it working.

Another, in the context of your question probably equally big drawback is, that using libraries from other languages often forces you to program in that languages style - and consequently "rip" you out of the style of the language you are actually programming in. For example, due to the lack of closures/anonymous functions and higher-order functions, Java libraries often don't have "nice" interfaces in the functional programming sense. Scala libraries, on the other hand, are usually written in a functional way.

Multi-platform languages

Another way of addressing your problem are multi-platform languages such as Haxe.

Haxe can be compiled to all popular programming platforms with its fast compiler – JavaScript, Flash, NekoVM, PHP, C++, C# and Java (soon).

To be fair, I don't have any experience with Haxe, so all I can do is conjecture. That said, I see this approach as a double-edged sword. On the one hand, it allows you to implement libraries only once and then compile them to C++, PHP, etc., and use them from their. On the other hand, if you write a client or a library depending on other libraries in Haxe, you are probably limited to Haxe libraries, and, for example, cannot use the highly appraised C++ library Boost.

A similar, although assumably less powerful approach is followed by Apache Thrift.

Apache Thrift allows you to define data types and service interfaces in a simple definition file. Taking that file as input, the compiler generates code to be used to easily build RPC clients and servers that communicate seamlessly across programming languages. Instead of writing a load of boilerplate code to serialize and transport your objects and invoke remote methods, you can get right down to business.

General worries

I assume that the "out of style" problem mentioned in the context of JVM-targeting languages is actually more general and probably something that makes it difficult to truly achieve what you want. If you specify interfaces arbitrarily general, it is likely that they feel out of place when used in any concrete programming language.

A second problem related to interfaces is which data structures to use as input/output data structures. For example, which data structures should a language-independent sorting library accept and return - arrays, sequences, lists, vectors, streams? And how are these mapped from/to data structures that actually exist in a given concrete language?

Another general problem I can see is performance. A library that is implemented in one language and cross-compiled into another is probably of poorer performance than one that is directly implemented in that language - which is not automatically a problem, but can definitely sometimes be one.