python threading issue with asynchronous http request

Question

I'm building an asynchronous HTTP request class with urllib2. The code looks like this:

import urllib2, threading, datetime

class ExportHandler(urllib2.HTTPHandler):
            def http_response(self, link, actualdate, nowdate ):
                link += "&export=csv"
                export = urllib2.urlopen( link )

for link, actualdate in commissionSummaryLinks:
            o = urllib2.build_opener(ExportHandler())
            t = threading.Thread(target=o.open, args=(link, actualdate, datetime.datetime.now().strftime("%Y%m%d%H%M%S")))
            t.start()

            print "I'm asynchronous!"
            t.join()

            print "ending threading"

Suffice it to say commissionSummaryLinks IS populated and actualdate is a date time.datetime.strptime() object.

Anyway, I'm receiving an error from all the issued threads that looks like this:

Exception in thread Thread-9:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 808, in __bootstrap_inner
   self.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 761, in run
   self.__target(*self.__args, **self.__kwargs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 402, in open
   req = meth(req)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1123, in do_request_
   'Content-length', '%d' % len(data))
TypeError: object of type 'datetime.date' has no len()

I'm running this on OS X (if it matters). Can anyone tell me what the problem here is?

Answer 1

When you instantiate your thread, you need to provide the arguments to o.open, not arguments to the http_response method of ExportHandler.

In this case, o.open has the following method signature:

open(self, fullurl, data=None, timeout=<object object>) method of urllib2.OpenerDirector instance

My guess is that you should only need to set args=(link,).

If you still need to use those other arguments, you'll probably want to modify the constructor of ExportHandler to take the other arguments you need, and then use them as appropriate in the http_response method. Take a look at the Python Class tutorial for more information on defining a class constructor.