how to start scrapyd server on EC2 instance

Question

I have setup an instance on aws. Now I want to start scrapyd on a particular port. according to documentation

aptitude install scrapyd-X.YY

but aptitude is not found. I have tried to installing aptitude using yum but there is no match found (may be it only works with apt-get, but I have yum ap-get is also missing)

can any one please help me that is there any other way to do this ??

Answer 1

You are using an yum based OS, not an apt based OS. Forget any commands that involve apt or a variation thereof.

Skip the steps you've already done:

yum install python
yum install python-pip
yum install libxml2-python
pip install Scrapy

As for libxml2-python, keep in mind that "versions prior to 2.6.28 are known to have problems parsing certain malformed HTML, and have also been reported to contain leaks, so 2.6.28 or above is highly recommended"

Answer 2

If you first install pip:

sudo yum install python-pip

you can use pip to install scrapyd like so

pip install scrapyd

source: http://scrapyd.readthedocs.org/en/latest/install.html