Building linux binaries for multiple platforms

Question

Help me settle a score.

I have a piece of software written in C++ that's meant to run on as many linux distributions as possible and I need to figure out a strategy that's effective. I'm trying to ship binaries in this case not source code (might be good to know). It's already a commercial product and I have intellectual property issues that prevent me from open sourcing the product but also means I have to deal with a myriad of GPL issues.

The current line of reasoning has been to pick a least common denominator and build everything off that. That has two major implications that I find counter productive.

1. The C++ support in old versions of GCC lacks some more modern C++ features.
2. The least common denominator entails Red Hat Enterprise Linux 4 (RHEL4)

I definitely don't need the the entire C++11 feature set but I'd like to bring the C++ support up to that of Visual C++ 2010. I'm perusing the idea to use Clang/libc++ as opposed to GCC/libstdc++ where possible.

RHEL4 doesn't seem to have extensive cross platform support for building C++ applications, more over, I have little insight in the stability of ABI across different versions of linux but I'm concerned that RHEL4 is more trouble than worth. Trying to build for all distributions based on a few is not a viable strategy.

I'm under the assumption that compiling software for different distributions of linux is best accomplished by compiling the software for the target platform with tools on the target platform. I'm also currently operating under the assumption that you'll run into vast portability issues cross linux platforms if you do not embrace this. Not to talk about the many libraries that you can or cannot link with because of C++ ABI instability across platforms/distributions.

But I could be wrong, I'd like to hear from the people that deal with this on a regular basis. What will work and why? or more importantly, what will not work?

Answer 1

You may try to focus on a few major platforms rather than individual distributions. What I mean by that is to build on what I call the "foundational distros" (Debian and RedHat) and hope for the best on the others.

Most likely, Debian binaries (that are statically linked) will run just fine on Ubuntu and Mint and other Debian derived distributions. RedHat binaries will probably run on Centos, Scientific Linux and SuSE Linux. If you care about less popular distros (Say you have a lot of customers running some uncommon Linux), and neither your Debian or RedHat executable works on them or can be made to work somehow, then setup a VM of that distro and build one executable specifically for that flavor.

I've taken this approach in the past and it has worked well.

Answer 2

The best way to do that it to make your software some open-source free software (e.g. GPLv3+ license), then if your software is interesting enough it will become packaged in distributions (by the distribution maintainer).

You always want to provide distribution packages (e.g. .deb files for Ubuntu or Debian) because these are the easiest to install.

If you still want to make proprietary software (but ask yourself if you will be able to sell, or even distribute gratis, successfully your software), you might take the following steps:

• compile a recent GCC compiler (e.g. 4.8) by enabling a static stdc++ & a static libgcc (most distribution provided GCC don't do that).

• link your program perhaps statically (but you might not be able to do this, e.g. for /etc/nsswitch.conf related functions, including getaddrinfo and related DNS services).

Even by linking all C++ related stuff statically you still depend on libc.so.6 and then you may have some Gnu Libc versioning issues (hence a binary compiled for a libc version 2.17 won't always run with a libc 2.16 or vice versa). Also notice that GNU libc are usually tied to some kernel version (you cannot use a recent libc on some quite old kernel). You could consider some alternative libc like MUSL libc

BTW, you usually can use some chroot-ed target environment (in which you might install some other distribution, eg. with debootstrap)

Answer 3

If anyone is curious we did solved the problem by building a GCC 4.8 toolchain on top of a RHEL 4.8 dist that continued to be our build agent.

The crux of it is outlined here. The problem was a little bit simpler since we don't need a fully functional cross compiler. Just a GCC 4.8 toolchain on the target host.

Interoperability was still a pain because this old version of Linux had virtually no support for SMB and/or other tech. We ended up with a Bash script that put build outputs in a FTP server and this worked reasonably well. It solved major pains.