•  23-07-2023
  •  | 
  •  

Question

I am crawling a page and got a list of links from the hrefs

['www.fundsupermart.co.in/main/fundinfo/viewFund.svdo?sedolnumber=SDM0347','www.fundsupermart.co.in/main/fundinfo/viewFund.svdo?sedolnumber=SDM0346', 'www.fundsupermart.co.in/main/fundinfo/viewFund.svdo?sedolnumber=AXS0229' ....]

Is it possible to crawl these links from the same spider? its like crawling another link while a crawler is already crawling a link. I don't need the code just a little guideline that how it can be done.

Was it helpful?

Solution

Use a CrawlSpider and define appropriate rules for the list of links?

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top