Python subprocess.call doesn't wait for command to execute

Question

I'm new to python, which I need to use for an assignment in a course. I developed the solution (an optimization algorithm) in Freemat / octave / matlab .m file and wanted to call it from Python (the python code will be called by a grading python script).

The .m file reads a file called tmp.data and writes the output to output.txt. The python script should then read from that output and convert it to the result that the grading script expects.

All runs fine, except I haven't been able to make Python wait for the call to Matlab to complete and therefore generates an error on the following lines.

Here's the code:

#!/usr/bin/python
# -*- coding: utf-8 -*-

from collections import namedtuple
Item = namedtuple("Item", ['index', 'value', 'weight'])

import subprocess
import os
from subprocess import Popen, PIPE

def solve_it(input_data):
    # Modify this code to run your optimization algorithm

    # Write the inputData to a temporay file
    tmp_file_name = 'tmp.data'
    tmp_file = open(tmp_file_name, 'w')
    tmp_file.write(input_data)
    tmp_file.close()

    # call matlab (or any other solver)
    # subprocess.call('matlab -r gp(\'tmp.data\')', shell=1)
    # run=os.system
    # a=run('matlab -r gp(\'tmp.data\')')
    # process = Popen('matlab -r gp(\'tmp.data\')', stdout=PIPE)
    # Popen.wait()
    # (stdout, stderr) = process.communicate()
    subprocess.call('matlab -r gp(\'tmp.data\')',shell=0)

    # Read result from file
    with open('output.txt') as f:
        result = f.read()

    # remove the temporay file
    os.remove(tmp_file_name)
    os.remove('output.txt')

    return result




    # return stdout.strip()



    # prepare the solution in the specified output format
    # output_data = str(value) + ' ' + str(0) + '\n'
    # output_data += ' '.join(map(str, taken))
    # return output_data


import sys

if __name__ == '__main__':
    if len(sys.argv) > 1:
        file_location = sys.argv[1].strip()
        input_data_file = open(file_location, 'r')
        input_data = ''.join(input_data_file.readlines())
        input_data_file.close()
        print solve_it(input_data)
    else:
        print 'This test requires an input file.  Please select one from the data directory. (i.e. python solver.py ./data/ks_4_0)'

As you see, I've tried with subprocess.call, popen, os.system... to no avail. All of them give me similar errors:

C:\Users\gp\Documents\Documents\personal\educacion\Discrete Optimization\knapsack>python2 solver.py data/ks_19_0
Traceback (most recent call last):
  File "solver.py", line 60, in <module>
    print solve_it(input_data)
  File "solver.py", line 30, in solve_it
    with open('output.txt') as f:
IOError: [Errno 2] No such file or directory: 'output.txt'

Of course! The error comes while matlab is still in the process of opening. It thus is trying to access a file that hasn't been created yet.

What should I do to get Python to wait for Matlab to complete??

I appreciate your kind help, thanks.

Answer 1

Your code seems to irgnore the fact that matlab uses a launcher (matlab_root/bin/matlab.exe) and a main application (matlab_root/bin/xxx/matlab.exe). To keep the launcher open until the main application closes, you have to use the -wait option.

Answer 2

[for the record]

As Daniel pointed out, it was solved by introducing a couple options into the matlab call:

subprocess.call('matlab -nosplash -wait -r "gp(\'tmp.data\')"',shell=0)

After that, it ran beautifully.

Thanks