Kingfisher Process API (v1)#

class kingfisher_scrapy.extensions.kingfisher_process_api.KingfisherProcessAPI(url, key, directory=None)[source]#

If the KINGFISHER_API_URI and KINGFISHER_API_KEY environment variables or configuration settings are set, then messages are sent to a Kingfisher Process API for the item_scraped and spider_closed signals.

classmethod from_crawler(crawler)[source]#
spider_closed(spider, reason)[source]#

Sends an API request to end the collection’s store step.

spider_error(failure, response, spider)[source]#

Sends an API request to store a file error in Kingfisher Process when a spider callback generates an error.

item_error(item, response, spider, failure)[source]#

Sends an API request to store a file error in Kingfisher Process when a item pipeline generates an error.

item_scraped(item, spider)[source]#

Sends an API request to store the file, file item or file error in Kingfisher Process.