You don't show your exact linking line, but the chances are you should (but don't) list the libraries after the object files:
gcc -o program main.o -L/custom/lib/dir -lstuff
If there's more than one object file outside of the libraries, then list them all before any of the libraries. The -L
options may go anywhere before the -llib
option that uses a library from the given directory.