Provide feedback from Python script to PHP

Question

Suppose a python script that performs a large number of operations, and outputs to the user a progress report while it is running:

$ script.py
Doing 1 of 100 operations...
Operation 1 succeeded with output "023948"
Doing 2 of 100 operations...
Operation 2 succeeded with output "893232"
Doing 3 of 100 operations...
Operation 3 succeeded with output "580217"
Doing 4 of 100 operations...
Operation 4 succeeded with output "228906"

Each line of output is shown about 2-3 seconds after the previous line, so the entire script run may take upwards 300 of seconds. I would like to run this script from a PHP-generated web page, and display the output to the user while it is running. I know that I can poll the PHP script from AJAX and get the latest message to update to the screen, but how can I get the actual output from the Python script while it is running? Functions such as exec() and shell_exec() only return the output after the script has finished running

In the worst case I could modify the Python script to output to a file, and to have an AJAX script continuously ping a PHP script which will read the file and diff it to the last read (also stored on the file system). But I would rather not modify the Python script for various reasons, and additionally I don't particularly like the idea of maintaining two additional files on the filesystem.

Answer 1

Use unbuffered stdout, passing the argument -u to the python script at the opening of the daemon (something as python -u (your python script))

And, in PHP, use something such as proc_open to read the content printed by the Python Script in real time.

Edit

As specified in comments, i can suggest something as:

Python:

import sys, atexit
sys.stdout = open(sys.argv.pop(), "w+") #Replaces stdout with a file returned from sys.argv (command line arguments)
def saveClose():
    sys.stdout.write("--%s--"%sys.stdout.name) #Just to indicate if the script closed
atexit.register(saveClose) #Register with atexit to execute the function at...exit

PHP: (named as daemon.php)

<?php
function execInBackground($cmd) {  // Put the program in background in Windows and *nix
    if (substr(php_uname(), 0, 7) == "Windows"){ // Detect if Windows 
        pclose(popen("start /B ". $cmd, "r")); // Use start /B (windows only) to open a background program in Windows
    } 
    else { 
        exec($cmd . " > /dev/null &");  // Open program as a daemon using & in *nix.
    } 
} 
if(isset($_GET["verify_id"])){ // We have ID?
  $content = file_get_contents($_GET["verify_id"]); // If yes, just load the file here (this is a security problem, but you can fix easily)
  echo $content; // Simply echoes the content of the file
}
else if(isset($_GET["daemon"])){
  $id = md5(uniqid(rand(), true)); // Create a unique hash
  execInBackground($_GET["daemon"]." ".$id); // Execute in the background passing the hash as a argument
  echo $id; // Echoes the hash
}
?>

Javascript: (named as daemon.js and with use of jQuery)

var cmds = {}
function receiveResult(cmd, id, callback){ // This function effectively receives the result from the execution of the program.
   var reg = new RegExp("--"+id+"--$");
   cmds_interval[id] = setInterval(function(){
       $.ajax({
         url:"daemon.php",
         dataType: "text",
         data: {"verify_id":id},
         success: function(msg){
           if(reg.test(msg)){ // Is the script closed?
              msg = msg.replace(reg, ""); // If yes, removes it from the end of the string
              clearInterval(cmds_interval[id]); // And clear the interval
           }
           callback(msg, id, cmd); // Callback with the message from the stdout 
         }
      });
   }, 1000); // refreshes with a interval of 1 second
   return cmds_interval[id];
}

function exec(cmd, callback){
  $.ajax({
    url:"daemon.php",
    dataType: "text",
    data: {"daemon":cmd},
    success: function(id){
       receiveResult(cmd, id, callback);
    }
  });
}

Example of use:

In HTML:

<pre id="console"></pre>
<script language="javascript" type="text/javascript" src="path/to/jquery.js"></script>
<script language="javascript" type="text/javascript" src="path/to/daemon.js"></script>
<script language="javascript" type="text/javascript" src="path/to/demo.js"></script>

In demo.js: (Javascript, and uses jQuery too):

exec("python script.py", function(msg){ 
    $("#console").html(msg);
});

This should work. If it not works, await to tomorrow as i'm exiting now. Good luck.

PS: If the code not work, you can see the code as a example of a algorithm to what you want.

PS 2: The execInBackground function is from here: http://www.php.net/manual/en/function.exec.php#86329

Answer 2

You could use proc_open to do this.

But if you want to go with the file alternative, there's no need to modify your python script, you can simply output to a file.

$execution = shell_exec("script.py > /ouput/file/path &");

and then read the file through ajax.