Why does my Python unittest test case with threads hang?

Question

I have the following test case. Note that the following test case is not trying to test anything but only trying to demonstrate the hanging issue I am encountering.

import http.server
import urllib.request
import threading
import unittest

class FooTest(unittest.TestCase):
    def setUp(self):
        print('---- setup start')
        self.httpd = http.server.HTTPServer(('', 8080), http.server.SimpleHTTPRequestHandler)
        thread = threading.Thread(target=self.httpd.serve_forever)
        thread.start()
        print('---- setup complete')

    def tearDown(self):
        print('---- teardown start')
        self.httpd.shutdown()
        print('---- teardown complete')

    def test1(self):
        print('---- test1 start')
        print(threading.current_thread())
        urllib.request.urlopen('http://127.0.0.1:8080/foo')
        print('---- test1 complete')

    def test2(self):
        print('---- test2 start')
        print(threading.current_thread())
        urllib.request.urlopen('http://127.0.0.1:8080/foo')
        print('---- test2 complete')

if __name__ == '__main__':
    unittest.main()

I expect 2 errors when I try to execute this test case. Instead, the test case hangs after the following output.

C:\lab>python foo.py -v
test1 (__main__.FooTest) ... ---- setup start
---- setup complete
---- test1 start
<_MainThread(MainThread, started 12980)>
127.0.0.1 - - [24/Mar/2014 21:53:57] code 404, message File not found
127.0.0.1 - - [24/Mar/2014 21:53:57] "GET /foo HTTP/1.1" 404 -
---- teardown start
---- teardown complete
ERROR
test2 (__main__.FooTest) ... ---- setup start
---- setup complete
---- test2 start
<_MainThread(MainThread, started 12980)>

If I remove test2 from the code, then I expect only 1 error and sure enough I see it.

C:\lab>python foo.py -v
test1 (__main__.FooTest) ... ---- setup start
---- setup complete
---- test1 start
<_MainThread(MainThread, started 15720)>
127.0.0.1 - - [24/Mar/2014 21:55:12] code 404, message File not found
127.0.0.1 - - [24/Mar/2014 21:55:12] "GET /foo HTTP/1.1" 404 -
---- teardown start
---- teardown complete
ERROR

======================================================================
ERROR: test1 (__main__.FooTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "foo.py", line 22, in test1
    urllib.request.urlopen('http://127.0.0.1:8080/foo')
  File "C:\Python34\lib\urllib\request.py", line 153, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python34\lib\urllib\request.py", line 461, in open
    response = meth(req, response)
  File "C:\Python34\lib\urllib\request.py", line 574, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Python34\lib\urllib\request.py", line 499, in error
    return self._call_chain(*args)
  File "C:\Python34\lib\urllib\request.py", line 433, in _call_chain
    result = func(*args)
  File "C:\Python34\lib\urllib\request.py", line 582, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: File not found

----------------------------------------------------------------------
Ran 1 test in 0.032s

FAILED (errors=1)

Why does the test case hang if there are 2 tests with errors?

Answer 1

Add a call to server_close to close the socket:

def setUp(self):
    print('---- setup start')
    handler = http.server.SimpleHTTPRequestHandler
    self.httpd = http.server.HTTPServer(('', 8080), handler)
    threading.Thread(target=self.serve).start()
    print('---- setup complete')

def serve(self):
    try:
        self.httpd.serve_forever()
    finally:
        self.httpd.server_close()

Answer 2

I found another way to run httpd in TestCase

import unittest
import threading
from http.server import HTTPServer, BaseHTTPRequestHandler

class TestHttpRequests(unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        cls.httpd = HTTPServer(handler=BaseHTTPRequestHandler)
        threading.Thread(target=cls.httpd.serve_forever).start()

    @classmethod
    def tearDownClass(cls):
        cls.httpd.shutdown()