'Unexpected end of file' and 'error importing function definition' error running shellscript using qsub

Question

I have the following shellscript:

#!/bin/sh
cd /sw/local/bin/
export LD_LIBRARY_PATH=/sw/local/lib:/usr/local/Trolltech/Qt-4.7.2/lib:$LD_LIBRARY_PATH
./FeatureFinderRaw -in /homes/JG-C1-18.mzML -out /homes/test_remove_after_use.featureXML -threads 20

It works fine when I run it from my own command line, but when I try to do:

qsub -q ningal.q -cwd -V -o /homes/queue.out -e /queue.err featureFind_C1-18_20t.sh 

I get the following error:

/bin/sh: module: line 1: syntax error: unexpected end of file
/bin/sh: error importing function definition for `module'
./FeatureFinderRaw: error while loading shared libraries: libOpenMS.so: cannot open shared object file: No such file or directory
/bin/sh: module: line 1: syntax error: unexpected end of file
/bin/sh: error importing function definition for `module'
./FeatureFinderRaw: error while loading shared libraries: libQtWebKit.so.4: cannot open shared object file: No such file or directory
/bin/sh: module: line 1: syntax error: unexpected end of file
/bin/sh: error importing function definition for `module'
./FeatureFinderRaw: error while loading shared libraries: libQtWebKit.so.4: cannot open shared object file: No such file or directory
/bin/bash: module: line 1: syntax error: unexpected end of file
/bin/bash: error importing function definition for `module'
./FeatureFinderRaw: error while loading shared libraries: libQtWebKit.so.4: cannot open shared object file: No such file or directory
/bin/sh: module: line 1: syntax error: unexpected end of file
/bin/sh: error importing function definition for `module'
./FeatureFinderRaw: error while loading shared libraries: libQtWebKit.so.4: cannot open shared object file: No such file or directory
/bin/sh: module: line 1: syntax error: unexpected end of file
/bin/sh: error importing function definition for `module'

I don't understand why I get this error when using qsub, but not when running the script directly on the same cluster machine. How can I run a script using qsub?

Answer 1

Also have this problem in a wrapper script that uses

qsub -shell no -b yes -cwd -V somescript.bash arg1 arg2 etc

if you use it to submit another bash shell script. It produces the annonying

/bin/sh: module: line 1: syntax error: unexpected end of file
/bin/sh: error importing function definition for `BASH_FUNC_module'

(this is Sun Grid Engine 211.11 running on CentOS 6.6) Turns out things are solved by simply putting the following on top of the wrapper script (not of the wrapped script):

unset module

That's all.

Answer 2

In /usr/share/Modules/init/bash commented out the 'export -f module' line.

In a normal login shell, modules.sh will be called from profile.d so the module command is available. In a non login shell, like an app wrapper script it just source the above file first.

Generally in applications script after sourcing above file they again give command "module load apps/vendor/app" which means extra sourcing.

Reference::- http://gridengine.org/pipermail/users/2011-November/002019.html

Answer 3

For some reason unknown to me adding semicolons at the end of every line fixed the problem.

Answer 4

Add the command below to ~/.bash_profile or ~/.bashrc file then logout/login again.

unset module

Or simple run quick command:

echo "unset module" >> ~/.bash_profile && source ~/.bash_profile

Answer 5

Most likely cause is that you saved the file with DOS line endings (\r\n) instead of POSIX line endings (\r) if adding semicolons make a difference.