how to pass an argument to a python script when starting with nohup

Question

I need to start a python script with bash using nohup passing an arg that aids in defining a constant in a script I import. There are lots of questions about passing args but I haven't found a successful way using nohup.

a simplified version of my bash script:

#!/bin/bash

BUCKET=$1
echo $BUCKET
script='/home/path/to/script/script.py'
echo "starting $script with nohup"
nohup /usr/bin/python $script $BUCKET &

the relevant part of my config script i'm importing:

FLAG = sys.argv[0]
if FLAG == "b1": 
    AWS_ACCESS_KEY_ID = "key"
    BUCKET = "bucket1" 
    AWS_SECRET_ACCESS_KEY = "secret"
elif FLAG == "b2": 
    AWS_ACCESS_KEY_ID = "key"
    BUCKET = "bucket2"
    AWS_SECRET_ACCESS_KEY = "secret"
else:
    AWS_ACCESS_KEY_ID = "key"
    BUCKET = "bucket3"
    AWS_SECRET_ACCESS_KEY = "secret"

the script thats using it:

from config import BUCKET, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY

#do stuff with the values.  

Frankly, since I'm passing the args to script.py, I'm not confident that they'll be in scope for the import script. That said, when I take a similar approach without using nohup, it works.

Answer 1

In general, the argument vector for any program starts with the program itself, and then all of its arguments and options. Depending on the language, the program may be sys.argv[0], argv[0], $0, or something else, but it's basically always argument #0.

Each program whose job is to run another program—like nohup, and like the Python interpreter itself—generally drops itself and all of its own options, and gives the target program the rest of the command line.

So, nohup takes a COMMAND and zero or more ARGS. Inside that COMMAND, argv[0] will be COMMAND itself (in this case, '/usr/bin/python'), and argv[1] and later will be the additional arguments ('/home/path/to/script/script.py' and whatever $BUCKET resolves to).

Next, Python takes zero or more options, a script, and zero or more args to that script, and exposes the script and its args as sys.argv. So, in your script, sys.argv[0] will be '/home/path/to/script/script.py', and sys.argv[1] will be whatever $BUCKET resolves to.

And bash works similarly to Python; $1 will be the first argument to the bash wrapper script ($0 will be the script itself), and so on. So, sys.argv[1] in the inner Python script will end up getting the first argument passed to the bash wrapper script.

Importing doesn't affect sys.argv at all. So, in both your config module and your top-level script, if you import sys, sys.argv[1] will hold the $1 passed to the bash wrapper script.

(On some platforms, in some circumstances argv[0] may not have the complete path, or may even be empty. But that isn't relevant here. What you care about is the eventual sys.argv[1], and bash, nohup, and python are all guaranteed to pass that through untouched.)

Answer 2

nohup python3 -u ./train.py --dataset dataset_directory/ --model model_output_directory > output.log &

Here Im executing train.py file with python3, Then -u is used to ignore buffering and show the logs on the go without storing, specifying my dataset_directory with argument style and model_output_directory then Greater than symbol(>)

then the logs is stored in output.log and them atlast and(&) symbol is used

To terminate this process

ps ax | grep train

then note the process_ID

sudo kill -9 Process_ID