Simple Spider

class kingfisher_scrapy.base_spiders.simple_spider.SimpleSpider(*args, **kwargs)

Most spiders can inherit from this class. It assumes all responses have the same data type.

1. Inherit from SimpleSpider

2. Set a data_type class attribute to the data type of the responses

3. Write a start_requests() method (and any intermediate callbacks) to send requests

import scrapy

from kingfisher_scrapy.base_spiders import SimpleSpider

class MySpider(SimpleSpider):
    name = 'my_spider'

    # SimpleSpider
    data_type = 'release_package'

    def start_requests(self):
        yield scrapy.Request('https://example.com/api/package.json', meta={'file_name': 'all.json'})

parse(response)