Question

From the Scrapy tutorial:

domain_name: identifies the Spider. It must be unique, that is, you can’t set the same domain name for different Spiders.

Does this mean that domain_name must be a valid domain name, like

domain_name = 'example.com'

Or can I name

domain_name = 'ex1'

The problem is I had a spider that worked with domain name

domain_name = 'whitecase.com'

Now I created a new spider as an instance of CrawlSpider and named it

domain_name = 'wc2'

but I am getting the error "could not find spider for domain "wc2""

Was it helpful?

Solution

To answer my own question:

domain_name = 'wc2'

works.

OTHER TIPS

As you have found out yourself, domain_name shouldn't be real. It's used when calling the spider to crawl.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top