문제

i am using gevent to preform concurrent download.
based on this example this is the code:

import gevent
from gevent import monkey

urls = ['https://www.djangoproject.com/','http://www.nytimes.com/','http://www.microsoft.com']
monkey.patch_all()

import urllib2
from datetime import datetime

def print_head(url):
    print ('Starting %s' % url)
    data = urllib2.urlopen(url).read()   
    print ('%s: %s bytes: %r' % (url, len(data), data[:50]))

startTime = datetime.now()
jobs = [gevent.spawn(print_head, url) for url in urls]
gevent.joinall(jobs)
totalTime = datetime.now() - startTime
print "Total time: %s" % totalTime

my problem is that the above code takes much longer than the serial version and in most cases it is timed out. here is the serial version which is much faster:

import urllib2
from datetime import datetime

urls = ['https://www.djangoproject.com/','http://www.nytimes.com/','http://www.microsoft.com']

def print_head(url):
    print ('Starting %s' % url)
    data = urllib2.urlopen(url).read()
    print ('%s: %s bytes: %r' % (url, len(data), data[:50]))

startTime = datetime.now()    
for url in urls:
    try:
        print_head(url)
    except:
        print 'ops EXCEPTION :('

totalTime = datetime.now() - startTime
print "Total time: %s" % totalTime
도움이 되었습니까?

해결책

ok
the problem was an old gevent package.
i just uninstalled the old one and installed the new one from here as @reclosedev pointed out.
and it is now working fine.

라이센스 : CC-BY-SA ~와 함께 속성
제휴하지 않습니다 StackOverflow
scroll top