scrapy Import Error: scrapy.core.downloader.handlers.s3.S3DownloadHandler

StackOverflow https://stackoverflow.com/questions/23529348

  •  17-07-2023
  •  | 
  •  

Question

i have installed scrapy on my windows 7 machine in a virtualenv called scrapy. scrapy startproject works and i made the tutorial from scrapy docs.

if i run

scrapy crawl dmoz

i get following error message:

File "C:\Users\mac\pystuff\scrapy\lib\site-packages\scrapy\utils\misc.py", line 42,in load_object
raise ImportError("Error loading object '%s': %s" % (path, e))
ImportError: Error loading object'scrapy.core.downloader.handlers.s3.S3DownloadHandler': DLL load failed: Could not find the specified module.

TESTS:

python console:

>>> import scrapy
>>> scrapy
>>> <module 'scrapy' from 'C:\Users\mac\pystuff\scrapy\lib\site-packages\scrapy\__init__.pyc'>

scrapy is available

powershell:

PS C: python -m pydoc scrapy.core.downloader.handlers

output:

NAME
scrapy.core.downloader.handlers - Download handlers for different schemes

FILE
c:\users\mac\pystuff\scrapy\lib\site-  packages\scrapy\core\downloader\handlers\__init__.py

PACKAGE CONTENTS
    file
    ftp
    http
    http10
    http11
    s3

CLASSES
    .
    .
    .

all other modules except s3 show the correct pydoc. for example: python -m pydoc scrapy.core.downloader.handlers.file

only s3 gives the import error:

PS C: python -m pydoc scrapy.core.downloader.handlers.s3
PS C: python -m pydoc scrapy.core.downloader.handlers.s3.S3DownloadHandler

output:

problem in scrapy.core.downloader.handlers.s3 - <type 'exceptions.ImportError'>: DLL load failed: Could not find the specified module.
Was it helpful?

Solution

pywin32 was problem package. i ve tried to import it in python console and it could not be found. i copy these 3 packages from C:(yourpythonpath)\Lib\site-packages\pywin32_system32\ to C:(yourpythonpath)\Lib\site-packages\win32\

  1. pythoncom27.dll
  2. pythoncomloader27.dll
  3. pywintype.dll

that solved the problem!

OTHER TIPS

You can download it as an installation from here: http://sourceforge.net/projects/pywin32/

I met the same problem and after a little bit research I tried almost all the suggestions and finally solve the problem. Here is my sequence:

Reinstall OPENSSL

Reinstall pywin32 amd64

Reinstall piwin32 32-bit

and magically the problem solved. My win server is 64-bit but the problem is solved by installing a 32-bit package. I guess this is because I previously install some other package that is 32-bit and didn't realize that. I'll keep you update if I have any progress.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top