spider_closed function is not fired

StackOverflow https://stackoverflow.com/questions/23200162

  •  07-07-2023
  •  | 
  •  

سؤال

I want to log some data when the spider finishes working.

I make this function inside my spider.

def spider_closed(self, spider):
        log.msg('The spider {0} finished working on'.format(self.name), level=log.INFO)

But this function is not getting fired.

I am using scrapy 0.20 with python 2.7

Please note that I can log to my log anything and actually I am logging a lot of data that I can see in my log file. so the logging is working with me.

Edit

I guess that I have to dispatch a signal signals.spider_closed but I don't know how to do that.

هل كانت مفيدة؟

المحلول

You should attach your method as a receiver to the SignalManager signal dispatcher:

from scrapy.xlib.pydispatch import dispatcher
from scrapy import signals

class MySpider(Spider):
    def __init__(self, *args, **kwargs):
        dispatcher.connect(self.spider_closed, signal=signals.spider_closed)
        super(MySpider, self).__init__(*args, **kwargs)

    def spider_closed(self, spider):
        log.msg('The spider {0} finished working on'.format(self.name), level=log.INFO)

Also see relevant threads:

مرخصة بموجب: CC-BY-SA مع الإسناد
لا تنتمي إلى StackOverflow
scroll top