Windows 上的 Python - 如何等待多个子进程?
-
01-07-2019 - |
题
如何在 Windows 上的 Python 中等待多个子进程,而不需要主动等待(轮询)?像这样的东西 几乎 对我有用:
proc1 = subprocess.Popen(['python','mytest.py'])
proc2 = subprocess.Popen(['python','mytest.py'])
proc1.wait()
print "1 finished"
proc2.wait()
print "2 finished"
问题是当 proc2
之前完成 proc1
, ,父进程仍然会等待 proc1
. 。在 Unix 上会使用 waitpid(0)
在循环中获取子进程完成时的返回码 - 如何在 Windows 上的 Python 中实现类似的功能?
解决方案
这可能看起来有点矫枉过正,但是,事情是这样的:
import Queue, thread, subprocess
results= Queue.Queue()
def process_waiter(popen, description, que):
try: popen.wait()
finally: que.put( (description, popen.returncode) )
process_count= 0
proc1= subprocess.Popen( ['python', 'mytest.py'] )
thread.start_new_thread(process_waiter,
(proc1, "1 finished", results))
process_count+= 1
proc2= subprocess.Popen( ['python', 'mytest.py'] )
thread.start_new_thread(process_waiter,
(proc2, "2 finished", results))
process_count+= 1
# etc
while process_count > 0:
description, rc= results.get()
print "job", description, "ended with rc =", rc
process_count-= 1
其他提示
扭曲有一个 异步进程生成 API 它适用于Windows。实际上有几种不同的实现,其中许多都不是很好,但是您可以在不更改代码的情况下在它们之间进行切换。
基于 zseil 的答案,您可以通过混合使用子进程和 win32 API 调用来实现此目的。我直接使用 ctypes,因为我的 Python 碰巧没有安装 win32api。我只是在这里从 MSYS 生成 sleep.exe 作为示例,但显然您可以生成您喜欢的任何进程。我使用 OpenProcess() 从进程的 PID 获取句柄,然后使用 WaitForMultipleObjects 等待任何进程完成。
import ctypes, subprocess
from random import randint
SYNCHRONIZE=0x00100000
INFINITE = -1
numprocs = 5
handles = {}
for i in xrange(numprocs):
sleeptime = randint(5,10)
p = subprocess.Popen([r"c:\msys\1.0\bin\sleep.exe", str(sleeptime)], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False)
h = ctypes.windll.kernel32.OpenProcess(SYNCHRONIZE, False, p.pid)
handles[h] = p.pid
print "Spawned Process %d" % p.pid
while len(handles) > 0:
print "Waiting for %d children..." % len(handles)
arrtype = ctypes.c_long * len(handles)
handle_array = arrtype(*handles.keys())
ret = ctypes.windll.kernel32.WaitForMultipleObjects(len(handle_array), handle_array, False, INFINITE)
h = handle_array[ret]
ctypes.windll.kernel32.CloseHandle(h)
print "Process %d done" % handles[h]
del handles[h]
print "All done!"
Windows 上的 Twisted 将在幕后执行主动等待。如果您不想使用线程,则必须使用 win32 API 来避免轮询。像这样的东西:
import win32process
import win32event
# Note: CreateProcess() args are somewhat cryptic, look them up on MSDN
proc1, thread1, pid1, tid1 = win32process.CreateProcess(...)
proc2, thread2, pid2, tid2 = win32process.CreateProcess(...)
thread1.close()
thread2.close()
processes = {proc1: "proc1", proc2: "proc2"}
while processes:
handles = processes.keys()
# Note: WaitForMultipleObjects() supports at most 64 processes at a time
index = win32event.WaitForMultipleObjects(handles, False, win32event.INFINITE)
finished = handles[index]
exitcode = win32process.GetExitCodeProcess(finished)
procname = processes.pop(finished)
finished.close()
print "Subprocess %s finished with exit code %d" % (procname, exitcode)
您可以使用 普苏蒂尔:
>>> import subprocess
>>> import psutil
>>>
>>> proc1 = subprocess.Popen(['python','mytest.py'])
>>> proc2 = subprocess.Popen(['python','mytest.py'])
>>> ls = [psutil.Process(proc1.pid), psutil.Process(proc2.pid)]
>>>
>>> gone, alive = psutil.wait_procs(ls, timeout=3)
“gone”和“alive”是指示哪些进程已消失以及哪些进程仍处于活动状态的列表。
您可以选择指定一个回调,每次监视的进程之一终止时都会调用该回调:
>>> def on_terminate(proc):
... print "%s terminated" % proc
...
>>> gone, alive = psutil.wait_procs(ls, timeout=3, callback=on_terminate)
不隶属于 StackOverflow