Why does it take a long time to import a module in a different thread when the main thread is running an infinite loop?

StackOverflow https://stackoverflow.com/questions/22628294

Question

I have the following Python code:

import threading
from datetime import datetime
import time

def f():
    print('---- {:%H:%M:%S}'.format(datetime.now()))
    import http.server
    print('---- {:%H:%M:%S}'.format(datetime.now()))

threading.Thread(target=f).start()

while True:
    pass

When I execute it, I see a great deal of time being consumed to import http.server. As you can see from the following output it took 23 seconds for the import to occur.

C:\>python foo.py
---- 10:12:03
---- 10:12:26

However, if I put a little sleep in the infinite while loop, the import happens faster.

import threading
from datetime import datetime
import time

def f():
    print('---- {:%H:%M:%S}'.format(datetime.now()))
    import http.server
    print('---- {:%H:%M:%S}'.format(datetime.now()))

threading.Thread(target=f).start()

while True:
    time.sleep(1)

Output:

C:\>python foo.py
---- 10:15:58
---- 10:15:58

I know the usage of join() method but I want to know precisely why does it take so long to import http.server when the infinite while loop doesn't have a sleep statement in it.

Was it helpful?

Solution

CPython uses Global Interpreter Lock to protect interpreter context. This prevents threads from running in the same time. Actually they all run on the single processor core. In CPython you can benefit from threads when they do idle-like operations i.e. waiting for I.O. or listening on the socket.
You gave a lot of work for the main thread. While pass does nothing interesting it consumes CPU cycle and also the interpreter believes it's important to give CPU time to this thread.
With sleep you say don't waste anything for this thread until time expires.

OTHER TIPS

I'm no expert on the topic, but as far as I know multiple threads run on same core one after another, so it's a question of CPU time sharing. Then adding sleep to infinite loop would give your import thread more resources. To run in parallel use multiprocessing. Then your code will use more than one core. You can check out this simple example

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top