selecting the proper Python interpreter when using Scrapy

Question

I have Scrapy successfully installed. I have recently successfully worked through the tutorial at http://doc.scrapy.org/en/latest/intro/tutorial.html .

However, now when I run the spider from the correct directory as instructed:

C:\My Projects\tutorial > scrapy crawl dmoz.

I am getting the error:

C:\Anaconda\python.exe: No module named scrapy.

This makes sense, because the interpreter that has access to the Scrapy module lives at

C:\python27\python.exe

My question is then, how do I point Scrapy to the correct Python interpreter? Or is this an incorrect way of conceptualizing what I'm doing?

Yes, I am using Windows 7

Thanks in advance.

Answer 1

You need to make sure that the path to the proper Python executable is before the path to any other Python executable. You should add C:\Python27 to the beginning of your system PATH variable. Beware: this may break things you use Anaconda for!

For future Googlers - In Windows 7, the PATH variable can be edited by pressing Win+Pause to open the System Control Panel. In the left hand bar, there is an option "Advanced System Settings". In the window that opens after clicking that, choose the "Advanced" tab, then at the bottom is a button that says "Environment Variables". Click that, then choose the variable named PATH from the bottom box for system variables. Click "Edit" and add the path as described above.