Can 32-bit LINUX 2.6 executables run reliably on LINUX 3.2 machines?

Question

Assuming 32-bit support has been loaded on to a given 64-bit LINUX installation:

• Can 32-bit LINUX 2.6 executable run reliably on LINUX 3.2 machines? (yes, restating title) Apparently not!

• What are the limitations placed on 32-bit programs (in terms of what kind of programs, not in terms of 4GB limitations, etc)?

• Is there a specific marker file, executable, syscall that can be checked to determine this ahead of time, so that a script could inform the user that the system is not configured correctly? Then I could write a script, say "canirunhere" that could make it obvious to the user, as opposed to just getting some bizarre FLOATING POINT EXCEPTION crash.

  • Trying to run the program (within the script) and see if it crashes is not an option, as the program could just be doing a divide-by-zero!

Backstory:

I have a 32-bit binary build for 2.6 LINUX that is exhibiting the classic incompatibility stack trace (seen below). It works on SOME machines, such as x86_64 Linux 3.2 machines (in this case, 3.2.0-8) , but not others, such as AMD64 Linux 3.2 machines (in this case, 3.2.44-3).

Necessary information:

The program dies in the dynamic loader itself. Here are the "file" and "uname -a" bits of info:

The file:
ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.6.9, dynamically linked (uses shared libs), not stripped

The NON-functional system:
3.2.44-3.2.0.3.1-amd64-10846333 #1 SMP Wed May 29 13:08:01 UTC 2013 x86_64 GNU/Linux

The FUNCTIONAL system, an x86_64 VM running Ubuntu:
3.2.0-58-generic #88-Ubuntu SMP Tue Dec 3 17:37:58 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

Note that the Ubuntu system did not initially support 32-bit programs, and thus I had to follow the instructions on the (now sadly dead) link in this previous question. (click here) I cannot perform the same task on the non-functional machine, although the lack of a "file not found" error means that 32-bit support is theoretically there.

Stack trace:
Program received signal SIGFPE, Arithmetic exception. 0xf7feb876 in do_lookup_x () from /lib/ld-linux.so.2 (gdb) where #0 0xf7feb876 in do_lookup_x () from /lib/ld-linux.so.2 #1 0xf7febc07 in _dl_lookup_symbol_x () from /lib/ld-linux.so.2 #2 0xf7fed251 in _dl_relocate_object () from /lib/ld-linux.so.2 #3 0xf7fe7108 in dl_main () from /lib/ld-linux.so.2 #4 0xf7ff58f1 in _dl_sysdep_start () from /lib/ld-linux.so.2 #5 0xf7fe3c33 in _dl_start () from /lib/ld-linux.so.2 #6 0xf7fe3817 in _start () from /lib/ld-linux.so.2

Strace output: At a responder's suggestion, here is the output of strace (was not the exact same program, so addresses will be slightly different), which confirms that it is in the dynamic loading, hopefully helping to narrow down the cause.

strace ./porgram execve("./program", ["./program"...], [/* 63 vars */]) = 0 brk(0) = 0x80ea000 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf77c5000 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=47552, ...}) = 0 mmap2(NULL, 47552, PROT_READ, MAP_PRIVATE, 3, 0) = 0xf77b9000 close(3) = 0 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) open("/lib/tls/i686/cmov/libc.so.6", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\222"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=1265332, ...}) = 0 mmap2(NULL, 1275268, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xf7681000 mmap2(0xf77b2000, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x130) = 0xf77b2000 mmap2(0xf77b6000, 9604, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xf77b6000 close(3) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf7680000 set_thread_area({entry_number:-1 -> 12, base_addr:0xf76806b0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 --- SIGFPE (Floating point exception) @ 0 (0) ---

Answer 1

Assuming 32-bit support has been loaded on to a given 64-bit LINUX installation

Basically yes, kernel should support older binaries, but in you case the program was linked dynamically, so it needs the specific version of glibc and all other dynamic libraries.

You can try to unpack the complete older 32bit distributive to some subfolder, then do chroot into it and then run your program inside the chroot.

Also, check ldd output of your binary (it may not work if binary was build with very old glibc, because there is the ld.so aka ld-linux.so.2 program to load dynamic binaries and to implement ldd).

There was one example of SIGFPE in do_lookup_x (you can google it with: SIGFPE, Arithmetic exception. do_lookup_x):

Floating point exception ( SIGFPE ) on 'int main(){ return(0); }' @ stackoverflow.com

Support for the GNU hash section was added to glibc around 2006, and mainline distros began to be GNU-hash-only around 2007 or 2008. Your Centrino's glibc is from 2003, which predates GNU hashing.

If the ld.so doesn't understand GNU hash, it will try to use the old ELF hash section instead, which is empty. In particular, I suspect your crash is occurring at this line in elf/do-lookup.h:

 for (symidx = map->l_buckets[hash % map->l_nbuckets];

Since the linker presumably doesn't understand GNU hashes, l_nbuckets would be 0, resulting in the crash.

So, your FPE may come from the same incompatibility between glibc on older 32-bit linux and 32-bit glibc in some newer distros.

Answer 2

As correctly pointed out in the other answer, your problem has nothing to do with kernel versions.

Rather, the glibc on your target (non-functional) machine is too old.

Usually, the result of building on a newer glibc system and running on an older one results in an dynamic linker error GLIBC_2.x version not found (required by ...), but in this case you crash even before you come to that point.

Now, if you can rebuild your application with -Wl,--hash-style=sysv, you will solve the SIGFPE problem, and end up with either a working application (somewhat unlikely), or with GLIBC_2.x not found error (very likely).

This answer shows some of the ways you could work around that.

Note: most UNIX systems support backward compatibility (old binaries continue to run on newer systems) but not forward compatibility (binaries built on newer machine run on older ones).