Kingfisher Process API (v2)#

class kingfisher_scrapy.extensions.kingfisher_process_api2.KingfisherProcessAPI2(url, stats, rabbit_url=None, rabbit_exchange_name=None, rabbit_routing_key=None)[source]#

If the KINGFISHER_API2_URL environment variable or configuration setting is set, then messages are sent to a Kingfisher Process API for the item_scraped and spider_closed signals.

ITEMS_SENT_POST = 'kingfisher_process_items_sent_post'#
ITEMS_FAILED_POST = 'kingfisher_process_items_failed_post'#
ITEMS_SENT_RABBIT = 'kingfisher_process_items_sent_rabbit'#
ITEMS_FAILED_RABBIT = 'kingfisher_process_items_failed_rabbit'#
open_connection_and_channel()[source]#
classmethod from_crawler(crawler)[source]#
spider_opened(spider)[source]#

Sends an API request to create a collection in Kingfisher Process.

spider_closed(spider, reason)[source]#

Sends an API request to close the collection in Kingfisher Process.

item_scraped(item, spider)[source]#

Sends either a RabbitMQ or API request to store the file, file item or file error in Kingfisher Process.