how to specify error log file and output file in qsub

Question

I have a qsub script as

#####----submit_job.sh---#####
    #!/bin/sh
    #$ -N job1
    #$ -t 1-100
    #$ -cwd
    SEEDFILE=/home/user1/data1
    SEED=$(sed -n -e "$SGE_TASK_ID p" $SEEDFILE)
    /home/user1/run.sh $SEED 

The problem is-- it puts all error and output files (job1.eJOBID & job1.oJOBID) to the same directory from where I am running qsub submit_job.sh while I want to save these file (output and error log file in same different place (specified as $SEED_output). I tried to change the line as

/home/user1/run.sh $SEED -o $SEED_output

But it didn't work. Any suggestion ?? How can I specify path and name of default output and error log file ??

Answer 1

Typically error and output files are given as pbs directives in the qsub script or as command line options to the qsub script like so:

#! /bin/bash
#PBS -q queue_name
#PBS -A account_name
#PBS -l nodes=12:ppn=12
#PBS -l walltime=18:00:00
#PBS -e /mypath/error.txt
#PBS -o /mypath/output.txt

or as a command line option to qsub like so:

qsub -o /mypath/output.txt -e /mypath/error.txt submit_job.sh

With the first option I don't think you can use a variable as the shell won't look at lines that are commented. Plus I think PBS deals with the commented lines before the shell would. If you know the path when you're invoking qsub, you could try the second option. Alternatively you could try simply redirecting the output and error in the script itself:

/home/user1/run.sh $SEED > ${SEED}/output.txt 2> ${SEED}/error.txt

The third option is probably the easiest. Output and error files might still be created in the run directory, though they'd likely be empty.

Answer 2

At first glance, you need brackets around your variable in the -o declaration.

/home/user1/run.sh $SEED -o ${SEED}_output

Otherwise bash is looking for a variable called ${SEED_output} which doesn't exist.

Answer 3

I ran into the same problem. And my workaround was to have:

#$ -N script-name
#$ -wd <path-to-project>/log 
cd ..

in the qsub script. Now the error and output file are saved in <path-to-project>/log and the script works in <path-to-project> At least for me that worked