Kingfisher Process API (v2)

class kingfisher_scrapy.extensions.kingfisher_process_api2.KingfisherProcessAPI2(url, stats, rabbit_url, rabbit_exchange_name, rabbit_routing_key)

If the KINGFISHER_API2_URL environment variable or configuration setting is set, then messages are sent to a Kingfisher Process API for the item_scraped and spider_closed signals.

ITEMS_SENT_POST = 'kingfisher_process_items_sent_post'

ITEMS_FAILED_POST = 'kingfisher_process_items_failed_post'

ITEMS_SENT_RABBIT = 'kingfisher_process_items_sent_rabbit'

ITEMS_FAILED_RABBIT = 'kingfisher_process_items_failed_rabbit'

open_connection_and_channel()

classmethod from_crawler(crawler)

spider_opened(spider)

Sends an API request to create a collection in Kingfisher Process.

spider_closed(spider, reason)

Sends an API request to close the collection in Kingfisher Process.

item_scraped(item, spider)

Sends either a RabbitMQ or API request to store the file, file item or file error in Kingfisher Process.