no error messages with nohup and python?

Question

I run a python script on a linux server with nohup like that:

nohup python3 ./myscript.py > ./mylog.log &

It works and the script writes the log file but the problem is that python error messages / thrown exceptions don't seem to get written into the log file. How could I achieve this?

Has this something to do with stderr? (but it says: "nohup: redirecting stderr to stdout" when I start the script.)

It is a long running script and after a while sometimes the script stops working because of some problem and with missing error messages I have no clue why. The problems always happen after a few days so this is really a pain to debug.

edit: Could it have something to do with flush? Since my own prints use flush but maybe python errors don't so they don't show up in the file once the script aborts?

Answer 1

I have found the reason. It really was the buffering problem (see my edit above). :)

nohup python3 -u ./myscript.py > ./mylog.log &

With the python -u parameter it works. It disables buffering.

Now I can go bug hunting...

Answer 2

You are only redirecting stdout. Error messages are given on stderr. Rerun your script like this:

nohup python3 ./myscript.py &> ./mylog.log &

The &> redirects all output (stdout and stderr) to your log file.

Answer 3

with nohup, the error will not be logged unless you specifically redirect error logs to a second file as shown below

nohup python myscript.py > out.log 2> err.log

you can also redirect both logs and error.log to the same files as

nohup python myscript.py > out.log2>&1

Python cache into the buffer before writing to log. To get realtime log used -u flag for "unbuffered".

nohup python -u myscript.py > out.log2>&1

To run in the background and recapture prompt add & at the end

nohup python -u myscript.py > out.log2>&1 &