Question

I am trying to run 24 versions of the same code on an 8 core machine. The code takes many many hours to run and I only want to run 8 at a time so I was wondering if it was possible to write a bash script which would run 8 and then when those were complete immediately start the next 8 and so on? I basically dont want all 24 to start and then run incredibly slowly! Thanks, Jack

EDIT 1: (More details on the run) The code runs with the following command: nohup ./MyCode MyInputFile 2> Myoutput

Was it helpful?

Solution

You could use gnu parallel

seq 1 24 | parallel -P 8 ./myscript 

Or with xargs:

seq 1 24 | xargs -l -P 8 ./myscript 

Update:

If you want to run the script with Myinput1 Myinput2 Myinput3 .. as parameters you can do

find . -name 'Myinput*' -print0 | parallel -0 -P 8 ./myscript {1}

or with your command:

find . -name 'Myinput*' -print0 | parallel -0 -P 8 nohup ./myscript {1}  2> Myoutput

OTHER TIPS

yet another way :

for f1 in {1..3};do 
   for f2 in {1..8};do
     echo "$f1,$f2;"
     nohup ./MyCode MyInputFile 2> Myoutput &
   done
   wait
done
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top