Unable to kill Flask server launched by Popen on Ubuntu

Question

Here is the reduced requirement of what I am trying to achieve.

# run.py
import requests
import time

from subprocess import Popen, PIPE

server = Popen("./app.py", stdout=PIPE, stderr=PIPE, shell=True)
time.sleep(1)
res = requests.get("http://localhost:1234/")
assert res.status_code == 200
server.kill()
server.terminate()
res = requests.get("http://localhost:1234/")
print res

And the actual server script.

#!/usr/bin/env python
from flask import Flask, make_response, request

app = Flask(__name__)
@app.route('/')
def view():
    return make_response("")

if __name__ == "__main__":
    app.run(host="localhost", port=1234)

On the command line I run python run.py. From shell:

(t)yeukhon@fubini:/tmp$ ps aux|grep app
yeukhon  21452  0.6  0.4  16416  9992 pts/2    S    03:50   0:00 python ./app.py
yeukhon  21471  0.0  0.0   4384   804 pts/2    S+   03:51   0:00 grep --color=auto app

So app.py is still hanging there. I have to kill it from the command line. In fact, the last line of run.py tells us the server was still alive (200 was returned).

I tried to kill with os.kill(server.pid, signal.SIGTERM) and os.kill(server.pid, signal.SIGKILL) but none works.

Normally kill will work, but I am really not sure why it can't receive the signal. I am sure somehow Flask is refusing to stop.

What options do I have?

strangely, my script above works perfectly fine on Mac OSX (I am on 10.8.5, Mountain Lion). So far I have tested on two Ubuntu 12.04 machines and they have the same behavior. I am running Python 2.7.3 on both Ubuntu machines and Python 2.7.2 on my Mac OSX.

correction: The only option I have is to use http://flask.pocoo.org/snippets/67/. But I prefer not to. And yes, I have to launch one using Popen.

Answer 1

By specifying shell=True, the command is run by subshell.

server = Popen("./app.py", stdout=PIPE, stderr=PIPE, shell=True)

server.terminate will kill the subshell, but web server will not be killed.

---

How to verify this? Try print server.pid after the Popen call, and compare that with ps output.

Answer 2

Remove shell=True from your Popen. This will make the first request. Kill the process. And then throw an exception for the second attempt.

Answer 3

Problem quite old but I met similar when I was running flask server with subprocess.Popen in conftest.py with pytest in docker container on Mac OS. This piece of code is not working for me (in conftest.py)

flask_server_proc = subprocess.Popen([
  'python', 'code/flask_server/main.py',
  '--local'
])

flask_server_proc.kill()

Parent process was killed but server still was running and answering requests. So I did this:

import psutil
import subprocess

server = subprocess.Popen([
    'python', 'code/flask_server/main.py',
    '--mysql-user', 'root',
    '--mysql-password', 'pass',
    '--mysql-host', 'mysql',
    '--local-artifact-store'
])

# Here I check if server is up. For example with some health check
# url. Sleep mimics it.
import time
time.sleep(10)
print(server.pid)

# Somewhere here you will call 'yield server'. This is just snippet
# with potential solution of problem so no yielding it.
# The next part should be run after 'yield server'.

# This should kill server.
# server.kill()

# Actually it should be one such process like flask.
flask_to_kill: List[psutil.Process] = []
for process in psutil.process_iter():
    if 'code/flask_server/main.py' in process.cmdline():
        flask_to_kill.append(process)

def on_terminate(proc: psutil.Process):
    print("process {} terminated with exit code {}".format(proc, proc.returncode))

for f in flask_to_kill:
    f.terminate()

gone, alive = psutil.wait_procs(flask_to_kill, timeout=3, callback=on_terminate)

for to_stab in alive:
    to_stab.kill()

so looks like remedy is to use psutil package to kill server after tests. Happy hacking!