How does creating a replacement memset trigger duplicate symbol errors in specific circumstances?

Question

I recently wrote a few replacements for string routines (memcpy, memset, and memmove). It is my understanding that if the library containing these routines is specified on the compile / link line, these will take precedence over system standard library routines of the same name. If I'm wrong already, please let me know!

This works correctly in all testing I did (verified by disassembly that the correct routines are there and glibc routines don't exist), but further testing discovered an odd break caused by this:

1) build another file in the same library with -g (I had been building -O2)

2) this file has an explicit call to memset

3a) if the compile time options work in such a way that this memset is inlined by gcc, everything is OK

3b) if, however, the options disable the inlining of a memset call which would have been inlined otherwise, the library will build but using the library to statically link an application causes a duplicate symbol linker error - the other instance of the symbol is the system library's memset.

Basically I can build two versions of my library (100's of source files), and by changing the make CFLAGS in one directory from -O1 -g to just -g I can trigger the linker error when this library is used.

I can take the working version, run it through nm, and see that it has many undefined references to memset including in routines which are linked into my test case - so I know it should be trying to resolve memset in the working case. When I diff this against the nm output for the broken library, all I see is a few extra undefined memcpy and memset references. If memset resolved in the first case (to my routine), it should in the second.

I have also looked at the verbose compiler output and verified that the link lines are exactly the same in both cases, except for the path to this one library.

There are two super puzzling things here (among a myriad of other issues):

1) Why would a file in a library built -O1 -g link any differently than -g

2) Why would a replacement memset, in a user library, conflict with the system memset

And for the grand prize, how does 1) cause 2)

Answer 1

It took a long time to come up with this solution, but it makes sense now:

1) Higher optimization enabled gcc to inline bzero, which had no other references to it at link time. The memset calls it was inlining / not inlining here were red herrings.

2) bzero is in the same file as memset in libc.a : memset.o. When ld tried to pull in memset.o to satisfy the bzero request it got the duplicate memset symbol.

(1) causes (2).

The solution was to provide my own bzero routine in my library, stopping libc's memset.o from ever being needed.

Answer 2

GCC provides a large number of built-in versions of standard library functions. These are provided for optimization purposes.

Many of these functions are optimized in only certain cases. If they are not optimized in a certain case, a call to the library function is emitted.

Hence a library built -O1 -g would link differently than -g.