Cluster qsub check when job is done. python

Question

I'm building a python pipeline and in that pipeline script I call for another python script which uses qsub to submit a job to a cluster (this job consist op 70 jobs) Now I need the output of those jobs to continue working.

How can I "Pause" the python script until all the 70 jobs are done?

I can't modify the script that submits to the server, so it has to be done in my own script. (using an AMD64 cluster)

edit:

this is how I call the other script:

os.system("python ScriptThatQsub.py arg1 arg2")

I'm still working on this. and searching the interwebz, but I can't help but wonder... is this even possible??

Answer 1

I've just put the Qstat from cluster in a variable and i checked if my jobname was found in that list. If so. sleep for a while. Thanks for the help everyone!

Answer 2

Just use the command line flag -sync y as in qsub -sync y my_job or

from subprocess import Popen
job = Popen(["qsub", "-sync", "y", "my_job"])
job.communicate()