Simple Spider#

class kingfisher_scrapy.base_spiders.simple_spider.SimpleSpider(*args, **kwargs)[source]#

Most spiders can inherit from this class. It assumes all responses have the same data type.

  1. Inherit from SimpleSpider

  2. Set a data_type class attribute to the data type of the responses

  3. Write a start_requests() method (and any intermediate callbacks) to send requests

import scrapy

from kingfisher_scrapy.base_spiders import SimpleSpider

class MySpider(SimpleSpider):
    name = 'my_spider'

    # SimpleSpider
    data_type = 'release_package'

    def start_requests(self):
        yield scrapy.Request('', meta={'file_name': 'all.json'})