Issues with the installation of scrapyd on Windows

Question

I am having issues with the installation of scrapyd on Wndows 7 I have installed the package using easy_install, but still the command scrapyd comes up with nothing. Here is the output of my install:

C:\Python27\Lib\site-packages\scrapy>easy_install scrapyd
Searching for scrapyd
Reading https://pypi.python.org/simple/scrapyd/
Best match: scrapyd 1.0.1
Downloading https://pypi.python.org/packages/source/s/scrapyd/scrapyd-1.0.1.tar.gz#md5=bc002742a938c40fa7e80f6743320c95
Processing scrapyd-1.0.1.tar.gz
Writing c:\users\uname\appdata\local\temp\easy_install-q7nfgz\scrapyd-1.0.1\setup.cfg
Running scrapyd-1.0.1\setup.py -q bdist_egg --dist-dir c:\users\uname\appdata\local\temp\easy_install-q7nfgz\scrapyd-1.0.1\egg-dist-tmp-ss3ual
no previously-included directories found matching 'docs\_build'
zip_safe flag not set; analyzing archive contents...
Adding scrapyd 1.0.1 to easy-install.pth file
Installing scrapyd script to C:\Python27\Scripts
Installing scrapyd-deploy script to C:\Python27\Scripts

Installed c:\python27\lib\site-packages\scrapyd-1.0.1-py2.7.egg
Processing dependencies for scrapyd
Finished processing dependencies for scrapyd

C:\Python27\Lib\site-packages\scrapy>scrapyd
'scrapyd' is not recognized as an internal or external command,
operable program or batch file.

Any ideas, what the issue is?

Answer 1

I am new here and I cannot comment. So perhaps it is not an answer but rather workaround.

Firstly, i am not sure if scrapyd works on windows at all. Please see this part of Scrapy documentation.

Personally, I develop on the Windows but deploy on Debian (where couldn't find the scrapyd package at all at the time I was working on it) and noticed that performance of scrapy is considerably worse than on linux box, so you might reconsider your target. But I digress.

However, you don't need scrapyd at all. Scrapyd is just a wrapper that keeps running scrapy server in background. To get same functionality you can run from commandline (from the scrapy project folder):

scrapy server

The scrapy server would be then running as long as the commandline is running but you can workaround this with bat file and some software to running tasks in background. Hope it helps.

Answer 2

Solution: Starting scrapyd on Windows 7 [after installation] from console:

cd to Python_install_dir/Scripts and run scrapyd from python.

In my case:

1.) cd C:\Python27\Scripts

2.) C:\Python27\Scripts> python scrapyd

now you can connect to scrapyd: http://127.0.0.1:6800/ & via console

If you wanna running the server in the background, just run this command as a windows background process.