Spiders¶

In Kingfisher Collect, there is one spider per data source. Each data source exposes its data in different ways, contains data from different years, and covers different stages of the contracting process.

Spider metadata¶

Below, we provide some of this information (metadata) for each spider. The meaning of the metadata is:

Metadata Description

Domain

The name of the IT system, jurisdiction, agency or program that is publishing the data. Some spiders have the same domain, in which case the difference between them is indicated in the suffix to the spider’s name:

Suffix	Description
`_releases`	Downloads individual releases or release packages
`_records`	Downloads individual records or record packages
`_bulk`	The data source is bulk data
`_historical`	The data source is historical data

Caveats Any noteworthy API or dataset limitations.

Spider arguments The Spider arguments the spider supports.

Environment variables

The environment variables required to run the spider. If a data source requires account registration, a link is provided. You can set environment variables as, for example:

env NAME1=VALUE1 NAME2=VALUE2 scrapy crawl spider_name

API documentation The URL of the API’s documentation.

Bulk download documentation The URL of any bulk download documentation.

Swagger API documentation The URL of the API’s documentation, using Swagger.

API endpoints If official API documentation is unavailable, we provide some brief documentation.

Spider arguments¶

A spider’s behavior can be changed by setting spider arguments:

scrapy crawl colombia -a NAME=VALUE

You can set multiple spider arguments:

scrapy crawl colombia -a from_date=2015-01-01 -a until_date=2019-12-31

All spiders support these arguments:

Name	Description	Format
`sample`	Set the number of files and file items to download.	integer

Some spiders support these arguments:

Name	Description	Format
`from_date`	Download data from this date onward. The spider might describe specific semantics (e.g. the relevant date is the date the contract was signed). Otherwise, the relevant date is the release date.	See spider
`until_date`	Download data until this date. The spider might describe specific semantics (e.g. the relevant date is the date the contract was signed). Otherwise, the relevant date is the release date.	See spider
`start_page`	Download data from this page onward.	integer
`publisher`	Download data by this publisher.	See spider
`system`	Download data from this system.	See spider

Afghanistan¶

class kingfisher_scrapy.spiders.afghanistan_record_packages.AfghanistanRecordPackages(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Afghanistan Government Electronic & Open Procurement System (AGEOPS)

Spider arguments

from_date: Download only data from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2018-12-15’.
until_date: Download only data until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.

API documentation

https://ocds.ageops.net/

scrapy crawl afghanistan_record_packages

class kingfisher_scrapy.spiders.afghanistan_records.AfghanistanRecords(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Afghanistan Government Electronic & Open Procurement System (AGEOPS)
API documentation: https://ocds.ageops.net/

scrapy crawl afghanistan_records

class kingfisher_scrapy.spiders.afghanistan_release_packages.AfghanistanReleasePackages(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Afghanistan Government Electronic & Open Procurement System (AGEOPS)

Spider arguments

from_date: Download only data from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2018-12-15’.
until_date: Download only data until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.

API documentation

https://ocds.ageops.net/

scrapy crawl afghanistan_release_packages

class kingfisher_scrapy.spiders.afghanistan_releases.AfghanistanReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Afghanistan Government Electronic & Open Procurement System (AGEOPS)

Spider arguments

from_date: Download only data from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2018-12-12’.
until_date: Download only data until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.

API documentation

https://ocds.ageops.net/

scrapy crawl afghanistan_releases

Argentina¶

class kingfisher_scrapy.spiders.argentina_buenos_aires.ArgentinaBuenosAires(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Ciudad de Buenos Aires
API documentation: https://data.buenosaires.gob.ar/acerca/ckan
Bulk download documentation: https://data.buenosaires.gob.ar/dataset/buenos-aires-compras/archivo/2a3d077c-71b6-4ba7-8924-f3e38cf1b8fc

scrapy crawl argentina_buenos_aires

class kingfisher_scrapy.spiders.argentina_vialidad.ArgentinaVialidad(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Vialidad Nacional of Ministerio de Obras Públicas
API documentation: https://datosabiertos.vialidad.gob.ar/ui/index.html#!/datos_abiertos

scrapy crawl argentina_vialidad

Armenia¶

class kingfisher_scrapy.spiders.armenia.Armenia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Armenian e-Procurement System (ARMEPS)
Caveats: The API paginates results using an offset query string parameter, which is a timestamp. If a timestamp causes an error, the spider will try to find the nearest timestamp within the following 1024 days that succeeds.

scrapy crawl armenia

Australia¶

class kingfisher_scrapy.spiders.australia.Australia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

AusTender

Spider arguments

from_date: Download only data from this time onward (YYYY-MM-DDThh:mm:ss format). Defaults to ‘2004-01-01T00:00:00’.
until_date: Download only data until this time (YYYY-MM-DDThh:mm:ss format). Defaults to now.

API documentation

https://github.com/austender/austender-ocds-api

Swagger API documentation

https://app.swaggerhub.com/apis/austender/ocds-api/1.1

scrapy crawl australia

class kingfisher_scrapy.spiders.australia_nsw.AustraliaNSW(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: New South Wales (NSW)
API documentation: https://github.com/NSW-eTendering/NSW-eTendering-API/blob/master/README.md

scrapy crawl australia_nsw

Bolivia¶

class kingfisher_scrapy.spiders.bolivia_agetic.BoliviaAgetic(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Agencia de Gobierno Electrónico y Tecnologías de Información y Comunicación (AGETIC)
Bulk download documentation: https://datos.gob.bo/id/dataset/contrataciones-agetic-2019-estandar-ocp

scrapy crawl bolivia_agetic

Canada¶

class kingfisher_scrapy.spiders.canada_buyandsell.CanadaBuyAndSell(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Public Works and Government Services Canada
Caveats: The dataset is a pilot.
API documentation: https://buyandsell.gc.ca/procurement-data/open-contracting-data-standard-pilot/download-ocds-pilot-data

scrapy crawl canada_buyandsell

class kingfisher_scrapy.spiders.canada_montreal.CanadaMontreal(*args, **kwargs)[source]¶

Domain: Montréal, Québec
API documentation: http://donnees.ville.montreal.qc.ca/dataset/contrats-et-subventions-api

scrapy crawl canada_montreal

class kingfisher_scrapy.spiders.canada_quebec.CanadaQuebec(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Secrétariat du Conseil du trésor
Bulk download documentation: https://www.donneesquebec.ca/recherche/dataset/systeme-electronique-dappel-doffres-seao

scrapy crawl canada_quebec

Chile¶

class kingfisher_scrapy.spiders.chile_compra_bulk.ChileCompraBulk(*args, **kwargs)[source]¶

Domain

ChileCompra

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). Defaults to ‘2009-01’.
until_date: Download only data until this month (YYYY-MM format). Defaults to the current month.

Bulk download documentation

https://desarrolladores.mercadopublico.cl/OCDS/DescargaMasiva

scrapy crawl chile_compra_bulk

class kingfisher_scrapy.spiders.chile_compra_records.ChileCompraRecords(*args, **kwargs)[source]¶

Domain

ChileCompra

Spider arguments

from_date

Download only data from this month onward (YYYY-MM format). Defaults to ‘2009-01’.

until_date

Download only data until this month (YYYY-MM format). Defaults to the current month.

system

Filter by system, if not set gets all:

convenio: Framework agreements only
trato-directo: Direct tenders only
licitacion: Traditional tenders

Swagger API documentation

https://desarrolladores.mercadopublico.cl/docs/services/5a9ed02f718ed712f4403e75/operations/data-listaa-omes-agno-mes

scrapy crawl chile_compra_records

class kingfisher_scrapy.spiders.chile_compra_releases.ChileCompraReleases(*args, **kwargs)[source]¶

Domain

ChileCompra

Spider arguments

from_date

Download only data from this month onward (YYYY-MM format). Defaults to ‘2009-01’.

until_date

Download only data until this month (YYYY-MM format). Defaults to the current month.

system

Filter by system, if not set gets all:

convenio: Framework agreements only
trato-directo: Direct tenders only
licitacion: Traditional tenders

Swagger API documentation

https://desarrolladores.mercadopublico.cl/docs/services/5a9ed02f718ed712f4403e75/operations/data-listaa-omes-agno-mes

scrapy crawl chile_compra_releases

Colombia¶

class kingfisher_scrapy.spiders.colombia.Colombia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Colombia Compra Eficiente (CCE)

Spider arguments

from_date: Download only data from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2011-01-01’.
until_date: Download only data until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.
start_page: The page number from which to start crawling.

API documentation

https://www.colombiacompra.gov.co/transparencia/api

Swagger API documentation

https://apiocds.colombiacompra.gov.co:8443/apiCCE2.0/

scrapy crawl colombia

class kingfisher_scrapy.spiders.colombia_bulk.ColombiaBulk(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Colombia Compra Eficiente (CCE)

Spider arguments

from_date

Download only data from this year onward (YYYY format). Defaults to ‘2011’. Only supported for ‘SECOP1’ system.

until_date

Download only data until this year (YYYY format). Defaults to the current year. Only supported for ‘SECOP1’ system.

system

Filter by system:

SECOP1

Sistema Electrónico para la Contratación Pública (todos los procesos de contratación)

SECOP2

Sistema Electrónico para la Contratación Pública (todos los procesos de contratación)

TVEC

Tienda Virtual del Estado Colombiano (acuerdos marco, agregación de demanda, mínima cuantía)

Bulk download documentation

https://www.colombiacompra.gov.co/transparencia/datos-json

scrapy crawl colombia_bulk

Costa Rica¶

class kingfisher_scrapy.spiders.costa_rica_poder_judicial_records.CostaRicaPoderJudicialRecords(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Poder Judicial de Costa Rica

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2018’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

Bulk download documentation

http://datosabiertospj.eastus.cloudapp.azure.com/dataset/estandar-de-datos-de-contrataciones-abiertas-ocds

scrapy crawl costa_rica_poder_judicial_records

class kingfisher_scrapy.spiders.costa_rica_poder_judicial_releases.CostaRicaPoderJudicialReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Poder Judicial de Costa Rica
Bulk download documentation: http://datosabiertospj.eastus.cloudapp.azure.com/dataset/estandar-de-datos-de-contrataciones-abiertas-ocds

scrapy crawl costa_rica_poder_judicial_releases

Croatia¶

class kingfisher_scrapy.spiders.croatia.Croatia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Electronic Public Procurement Classifieds of the Republic of Croatia
Bulk download documentation: https://eojn.nn.hr/SPIN/application/ipn/Oglasnik/PreuzimanjeUgovoraOCD.aspx

scrapy crawl croatia

Digiwhist¶

class kingfisher_scrapy.spiders.digiwhist_austria.DigiwhistAustria(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_austria

class kingfisher_scrapy.spiders.digiwhist_belgium.DigiwhistBelgium(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_belgium

class kingfisher_scrapy.spiders.digiwhist_bulgaria.DigiwhistBulgaria(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_bulgaria

class kingfisher_scrapy.spiders.digiwhist_croatia.DigiwhistCroatia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_croatia

class kingfisher_scrapy.spiders.digiwhist_cyprus.DigiwhistCyprus(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_cyprus

class kingfisher_scrapy.spiders.digiwhist_czech_republic.DigiwhistCzechRepublic(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_czech_republic

class kingfisher_scrapy.spiders.digiwhist_denmark.DigiwhistDenmark(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_denmark

class kingfisher_scrapy.spiders.digiwhist_estonia.DigiwhistEstonia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_estonia

class kingfisher_scrapy.spiders.digiwhist_finland.DigiwhistFinland(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_finland

class kingfisher_scrapy.spiders.digiwhist_france.DigiwhistFrance(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_france

class kingfisher_scrapy.spiders.digiwhist_georgia.DigiwhistGeorgia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_georgia

class kingfisher_scrapy.spiders.digiwhist_germany.DigiwhistGermany(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_germany

class kingfisher_scrapy.spiders.digiwhist_greece.DigiwhistGreece(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_greece

class kingfisher_scrapy.spiders.digiwhist_hungary.DigiwhistHungary(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_hungary

class kingfisher_scrapy.spiders.digiwhist_iceland.DigiwhistIceland(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_iceland

class kingfisher_scrapy.spiders.digiwhist_ireland.DigiwhistIreland(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_ireland

class kingfisher_scrapy.spiders.digiwhist_italy.DigiwhistItaly(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_italy

class kingfisher_scrapy.spiders.digiwhist_latvia.DigiwhistLatvia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_latvia

class kingfisher_scrapy.spiders.digiwhist_lithuania.DigiwhistLithuania(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_lithuania

class kingfisher_scrapy.spiders.digiwhist_luxembourg.DigiwhistLuxembourg(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_luxembourg

class kingfisher_scrapy.spiders.digiwhist_malta.DigiwhistMalta(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_malta

class kingfisher_scrapy.spiders.digiwhist_netherlands.DigiwhistNetherlands(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_netherlands

class kingfisher_scrapy.spiders.digiwhist_norway.DigiwhistNorway(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_norway

class kingfisher_scrapy.spiders.digiwhist_poland.DigiwhistPoland(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_poland

class kingfisher_scrapy.spiders.digiwhist_portugal.DigiwhistPortugal(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_portugal

class kingfisher_scrapy.spiders.digiwhist_romania.DigiwhistRomania(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_romania

class kingfisher_scrapy.spiders.digiwhist_slovakia.DigiwhistSlovakia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_slovakia

class kingfisher_scrapy.spiders.digiwhist_slovenia.DigiwhistSlovenia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_slovenia

class kingfisher_scrapy.spiders.digiwhist_spain.DigiwhistSpain(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_spain

class kingfisher_scrapy.spiders.digiwhist_sweden.DigiwhistSweden(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_sweden

class kingfisher_scrapy.spiders.digiwhist_switzerland.DigiwhistSwitzerland(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_switzerland

class kingfisher_scrapy.spiders.digiwhist_ted.DigiwhistTED(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_ted

class kingfisher_scrapy.spiders.digiwhist_united_kingdom.DigiwhistUnitedKingdom(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

scrapy crawl digiwhist_united_kingdom

Dominican Republic¶

class kingfisher_scrapy.spiders.dominican_republic.DominicanRepublic(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Dirección General de Contrataciones Públicas (DGCP)

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2018’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

Bulk download documentation

https://www.dgcp.gob.do/estandar-mundial-ocds/

scrapy crawl dominican_republic

class kingfisher_scrapy.spiders.dominican_republic_api.DominicanRepublicPortal(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Dirección General de Contrataciones Públicas (DGCP)

Spider arguments

from_date: Download only data from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2018-01-01’.
until_date: Download only data until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.

API documentation

http://148.101.176.123:48080/ocdsdr/docs

scrapy crawl dominican_republic_api

Ecuador¶

class kingfisher_scrapy.spiders.ecuador_emergency.EcuadorEmergency(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Servicio Nacional de Contratación Pública
Bulk download documentation: https://portal.compraspublicas.gob.ec/sercop/data-estandar-ocds/

scrapy crawl ecuador_emergency

Europe¶

class kingfisher_scrapy.spiders.europe_ted_tender_base.EuropeTedTenderBase(*args, **kwargs)[source]¶

Domain

TenderBase

API endpoints

Get releases by page

Link

http://www.tenderbase.eu/api/releases/?page={page}

Parameters

page: page number

scrapy crawl europe_ted_tender_base

France¶

class kingfisher_scrapy.spiders.france.France(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: France
Swagger API documentation: https://doc.data.gouv.fr/api/reference/

scrapy crawl france

Georgia¶

class kingfisher_scrapy.spiders.georgia_opendata.GeorgiaOpenData(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: State Procurement Agency (SPA)

scrapy crawl georgia_opendata

class kingfisher_scrapy.spiders.georgia_records.GeorgiaRecords(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: State Procurement Agency (SPA)
Bulk download documentation: https://odapi.spa.ge/publication-policy
Swagger API documentation: https://odapi.spa.ge/api/swagger.ui

scrapy crawl georgia_records

class kingfisher_scrapy.spiders.georgia_releases.GeorgiaReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: State Procurement Agency (SPA)
Bulk download documentation: https://odapi.spa.ge/publication-policy
Swagger API documentation: https://odapi.spa.ge/api/swagger.ui

scrapy crawl georgia_releases

Honduras¶

class kingfisher_scrapy.spiders.honduras_cost.HondurasCoST(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Systema de Información y Seguimiento de Obras y Contratos (SISOCS)

scrapy crawl honduras_cost

class kingfisher_scrapy.spiders.honduras_oncae.HondurasONCAE(*args, **kwargs)[source]¶

Domain

Oficina Normativa de Contratación y Adquisiciones del Estado (ONCAE)

Spider arguments

from_date

Download only data from this year onward (YYYY format). Defaults to ‘2005’.

until_date

Download only data until this year (YYYY format). Defaults to the current year.

system

Filter by system:

CE: Catálogo Electrónico
DDC: Módulo de Difusión Directa de Contratos
HC1: HonduCompras 1.0 (Módulo de Difusión de Compras y Contrataciones)

Bulk download documentation

http://oncae.gob.hn/datosabiertos

scrapy crawl honduras_oncae

class kingfisher_scrapy.spiders.honduras_portal_bulk.HondurasPortalBulk(*args, **kwargs)[source]¶

Domain

Oficina Normativa de Contratación y Adquisiciones del Estado (ONCAE) / Secretaria de Finanzas de Honduras (SEFIN)

Spider arguments

from_date

Download only data from this month onward (YYYY-MM format). Defaults to ‘2005-11’.

until_date

Download only data until this month (YYYY-MM format). Defaults to the current month.

publisher

Filter by publisher:

oncae: Oficina Normativa de Contratación y Adquisiciones del Estado
sefin: Secretaria de Finanzas de Honduras

system

Filter by oncae system:

CE: Catálogo Electrónico
DDC: Módulo de Difusión Directa de Contratos
HC1: HonduCompras 1.0 (Módulo de Difusión de Compras y Contrataciones)

Bulk download documentation

http://www.contratacionesabiertas.gob.hn/descargas/

scrapy crawl honduras_portal_bulk

class kingfisher_scrapy.spiders.honduras_portal_records.HondurasPortalRecords(*args, **kwargs)[source]¶

Domain

Oficina Normativa de Contratación y Adquisiciones del Estado (ONCAE) / Secretaria de Finanzas de Honduras (SEFIN)

Spider arguments

publisher

Filter by publisher:

oncae: Oficina Normativa de Contratación y Adquisiciones del Estado
sefin: Secretaria de Finanzas de Honduras

API documentation

http://www.contratacionesabiertas.gob.hn/manual_api/

Swagger API documentation

http://www.contratacionesabiertas.gob.hn/servicio/

scrapy crawl honduras_portal_records

class kingfisher_scrapy.spiders.honduras_portal_releases.HondurasPortalReleases(*args, **kwargs)[source]¶

Domain

Oficina Normativa de Contratación y Adquisiciones del Estado (ONCAE) / Secretaria de Finanzas de Honduras (SEFIN)

Spider arguments

publisher

Filter by publisher:

oncae: Oficina Normativa de Contratación y Adquisiciones del Estado
sefin: Secretaria de Finanzas de Honduras

API documentation

http://www.contratacionesabiertas.gob.hn/manual_api/

Swagger API documentation

http://www.contratacionesabiertas.gob.hn/servicio/

scrapy crawl honduras_portal_releases

India¶

class kingfisher_scrapy.spiders.india_civic_data_lab.IndiaCivicDataLab(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Civic Data Lab India
Bulk download documentation: https://github.com/CivicDataLab/himachal-pradesh-health-procurement-OCDS/

scrapy crawl india_civic_data_lab

Indonesia¶

class kingfisher_scrapy.spiders.indonesia_bandung.IndonesiaBandung(*args, **kwargs)[source]¶

Domain

Bandung Integrated Resource Management System (BIRMS)

Spider arguments

from_date: Download only data from this year onward (YYYY format). Defaults to ‘2013’.
until_date: Download only data until this year (YYYY format). Defaults to the current year.

API endpoints

Get contract list by year

Link

https://birms.bandung.go.id/api/contracts/year/{year}?page={page}

Parameters

year: contract year number
page: page number

Get contract detail by OCID

Link

https://birms.bandung.go.id/api/newcontract/ocds-afzrfb-{source-database}-{year}-{uniqid}

Parameters

source-database: (b)irms or (s)pse
year: contract year number
uniqid: release id

scrapy crawl indonesia_bandung

class kingfisher_scrapy.spiders.indonesia_opentender.IndonesiaOpentender(*args, **kwargs)[source]¶

Domain

Open Tender

Spider arguments

from_date: Download only data from this year onward (YYYY format). Defaults to ‘2008’.
until_date: Download only data until this year (YYYY format). Defaults to the current year.

Bulk download documentation

https://v3.opentender.net/#/ocds

scrapy crawl indonesia_opentender

Italy¶

class kingfisher_scrapy.spiders.italy.Italy(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: AppaltiPOP
Bulk download documentation: https://www.appaltipop.it/it/download

scrapy crawl italy

Kenya¶

class kingfisher_scrapy.spiders.kenya_makueni.KenyaMakueni(*args, **kwargs)[source]¶

Domain: Makueni County
Swagger API documentation: https://opencontracting.makueni.go.ke/swagger-ui.html#/ocds-controller

scrapy crawl kenya_makueni

Kosovo¶

class kingfisher_scrapy.spiders.kosovo.Kosovo(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Public Procurement Regulatory Commission

Spider arguments

from_date: Download only data from this time onward (YYYY-MM-DDThh:mm:ss format). If until_date is provided, defaults to ‘2000-01-01T00:00:00’.
until_date: Download only data until this time (YYYY-MM-DDThh:mm:ss format). If from_date is provided, defaults to now.

API documentation

https://ocdskrpp-test.rks-gov.net/Help

scrapy crawl kosovo

Kyrgyzstan¶

class kingfisher_scrapy.spiders.kyrgyzstan.Kyrgyzstan(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Ministry of Finance
Caveats: The planning endpoint is not implemented because is not completely in OCDS.
Bulk download documentation: https://analytics.zakupki.gov.kg/dashboard/weekly and then to ‘API & EXPORT’. Direct access to https://analytics.zakupki.gov.kg/api-export doesn’t work.
Swagger API documentation: https://app.swaggerhub.com/apis/DPAteam/export-api-documentation/1.0

scrapy crawl kyrgyzstan

Malta¶

class kingfisher_scrapy.spiders.malta.Malta(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Malta

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). If until_date is provided, defaults to ‘2019-10’.
until_date: Download only data until this month (YYYY-MM format). If from_date is provided, defaults to the current month.

API documentation

https://docs.google.com/document/d/1VnCEywKkkQ7BcVbT7HlW2s_N_QI8W0KE/edit

scrapy crawl malta

Mexico¶

class kingfisher_scrapy.spiders.mexico_administracion_publica_federal_api.MexicoAdministracionPublicaFederalAPI(*args, **kwargs)[source]¶

Domain: Administración Pública Federal (APF): Secretaria de la Función Pública (SFP) - Secretaría de Hacienda y Crédito Público (SHCP)
API documentation: https://www.datos.gob.mx/busca/dataset/api-de-contrataciones-abiertas-de-la-apf

scrapy crawl mexico_administracion_publica_federal_api

class kingfisher_scrapy.spiders.mexico_administracion_publica_federal_bulk.MexicoAdministracionPublicaFederalBulk(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Administración Pública Federal (APF): Secretaría de Hacienda y Crédito Público (SHCP)
Bulk download documentation: https://datos.gob.mx/busca/dataset/concentrado-de-contrataciones-abiertas-de-la-apf

scrapy crawl mexico_administracion_publica_federal_bulk

class kingfisher_scrapy.spiders.mexico_grupo_aeroporto.MexicoGrupoAeroporto(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Grupo Aeroportuario de la Ciudad de México (CDMX)

scrapy crawl mexico_grupo_aeroporto

class kingfisher_scrapy.spiders.mexico_idaip.MexicoIDAIP(*args, **kwargs)[source]¶

Domain

Instituto Duranguense de Acceso a la Información y de Protección de Datos Personales (IDAIP)

Spider arguments

from_date: Download only data from this year onward (YYYY format). Defaults to ‘2020’.
until_date: Download only data until this year (YYYY format). Defaults to ‘2021’.

API documentation

http://contratacionesabiertas.idaip.org.mx/contratacionesabiertas/datosabiertos

scrapy crawl mexico_idaip

class kingfisher_scrapy.spiders.mexico_inai_api.MexicoINAIAPI(*args, **kwargs)[source]¶

Domain

Instituto Nacional de Transparencia, Acceso a la Información y Protección de Datos Personales (INAI)

Spider arguments

from_date: Download only data from this year onward (YYYY format). Defaults to ‘2015’.
until_date: Download only data until this year (YYYY format). Defaults to ‘2020’.

API documentation

http://contratacionesabiertas.inai.org.mx/contratacionesabiertas/datosabiertos

scrapy crawl mexico_inai_api

class kingfisher_scrapy.spiders.mexico_nuevo_leon_records.MexicoNuevoLeonRecords(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Secretaría de Infraestructura del Gobierno del Estado de Nuevo León
Bulk download documentation: http://si.nl.gob.mx/transparencia/publicaciones

scrapy crawl mexico_nuevo_leon_records

class kingfisher_scrapy.spiders.mexico_nuevo_leon_releases.MexicoNuevoLeonReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Secretaría de Infraestructura del Gobierno del Estado de Nuevo León
Bulk download documentation: http://si.nl.gob.mx/transparencia/acerca-del-proyecto

scrapy crawl mexico_nuevo_leon_releases

class kingfisher_scrapy.spiders.mexico_plataforma_digital_nacional.MexicoPlataformaDigitalNacional(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Plataforma Digital Nacional
Bulk download documentation: https://plataformadigitalnacional.org/contrataciones

scrapy crawl mexico_plataforma_digital_nacional

class kingfisher_scrapy.spiders.mexico_quien_es_quien.MexicoQuienEsQuien(*args, **kwargs)[source]¶

Domain: QuiénEsQuién.Wiki
API documentation: https://qqwapi-elastic.readthedocs.io/es/latest/
Swagger API documentation: https://api.quienesquien.wiki/v3/docs/

scrapy crawl mexico_quien_es_quien

class kingfisher_scrapy.spiders.mexico_yucatan_inaip.MexicoYucatan(*args, **kwargs)[source]¶

Domain

Instituto Estatal de Transparencia, Acceso a la Información Pública y Protección de Datos Personales (INAIP) - Yucatán

Spider arguments

from_date: Download only data from this year onward (YYYY format). Defaults to ‘2020’.
until_date: Download only data until this year (YYYY format). Defaults to ‘2020’.

API documentation

https://contratacionesabiertas.inaipyucatan.org.mx/contratacionesabiertas/datosabiertos#

scrapy crawl mexico_yucatan_inaip

Moldova¶

class kingfisher_scrapy.spiders.moldova.Moldova(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

MTender

Spider arguments

from_date: Download only data from this time onward (YYYY-MM-DDThh:mm:ss format).

scrapy crawl moldova

class kingfisher_scrapy.spiders.moldova_old.MoldovaOld(*args, **kwargs)[source]¶

Domain

Public Procurement Agency (PPA)

Spider arguments

from_date: Download only data from this year onward (YYYY format). Defaults to ‘2012’.
until_date: Download only data until this year (YYYY format). Defaults to ‘2017’.

Bulk download documentation

http://opencontracting.date.gov.md/downloads

scrapy crawl moldova_old

class kingfisher_scrapy.spiders.moldova_positive_initiative.MoldovaPositiveInitiative(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Positive Initiative - Moldova
Bulk download documentation: https://www.tender.health/ocdsrelease

scrapy crawl moldova_positive_initiative

Nepal¶

class kingfisher_scrapy.spiders.nepal_dhangadhi.NepalDhangadhi(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Dhangadhi Infrastructure Management System (IMS)
Caveats: Some URLs listed in https://admin.ims.susasan.org/api/static-data/dhangadhi require login and cannot be downloaded
Bulk download documentation: https://ims.susasan.org/dhangadhi/about

scrapy crawl nepal_dhangadhi

class kingfisher_scrapy.spiders.nepal_portal.NepalPortal(*args, **kwargs)[source]¶

Domain

Public Procurement Monitoring Office (PPMO)

Spider arguments

from_date: Download only data from this year onward (YYYY format). Defaults to ‘2012’.
until_date: Download only data until this year (YYYY format). Defaults to ‘2018’.

Bulk download documentation

http://ppip.gov.np/downloads

scrapy crawl nepal_portal

Nicaragua¶

class kingfisher_scrapy.spiders.nicaragua_solid_waste.NicaraguaSolidWaste(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Solid Waste Mitigation Platform (SWMP)

Spider arguments

from_date: Download only data from this date onward (YYYY-MM-DD format). Defaults to ‘2000-01-01’.
until_date: Download only data until this date (YYYY-MM-DD format). Defaults to today.

scrapy crawl nicaragua_solid_waste

Nigeria¶

class kingfisher_scrapy.spiders.nigeria_budeshi_records.NigeriaBudeshiRecords(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Budeshi Nigeria
API documentation: https://budeshi.ng/Api

scrapy crawl nigeria_budeshi_records

class kingfisher_scrapy.spiders.nigeria_budeshi_releases.NigeriaBudeshiReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Budeshi Nigeria
API documentation: https://budeshi.ng/Api

scrapy crawl nigeria_budeshi_releases

class kingfisher_scrapy.spiders.nigeria_portal.NigeriaPortal(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Nigeria Open Contracting Portal (NOCOPO) of Bureau of Public Procurement (BPP)

scrapy crawl nigeria_portal

Openopps¶

class kingfisher_scrapy.spiders.openopps.OpenOpps(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

OpenOpps

Spider arguments

from_date: Download only data from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2011-01-01’.
until_date: Download only data until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.

Environment variables

KINGFISHER_OPENOPPS_USERNAME: To get an API account, contact contact@openopps.com.
KINGFISHER_OPENOPPS_PASSWORD: Your API account password.

API documentation

https://docs.google.com/document/d/1u0da3BTU7fBFjX6i7j_tKXa1YwdXL7hY4Kw9GdsaAr0/edit

Swagger API documentation

https://api.openopps.com/api/schema/

env KINGFISHER_OPENOPPS_USERNAME=... KINGFISHER_OPENOPPS_PASSWORD=... scrapy crawl openopps

Pakistan¶

class kingfisher_scrapy.spiders.pakistan_ppra_bulk.PakistanPPRABulk(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Pakistan Public Procurement Regulatory Authority (PPRA)
API documentation: https://www.ppra.org.pk/api/

scrapy crawl pakistan_ppra_bulk

class kingfisher_scrapy.spiders.pakistan_ppra_releases.PakistanPPRAReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Pakistan Public Procurement Regulatory Authority (PPRA)
API documentation: https://www.ppra.org.pk/api/

scrapy crawl pakistan_ppra_releases

Paraguay¶

class kingfisher_scrapy.spiders.paraguay_dncp_records.ParaguayDNCPRecords(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Dirección Nacional de Contrataciones Públicas (DNCP)

Spider arguments

from_date: Download only data from this time onward (YYYY-MM-DDThh:mm:ss format). Defaults to ‘2010-01-01T00:00:00’.
until_date: Download only data until this time (YYYY-MM-DDThh:mm:ss format). Defaults to now.

Environment variables

KINGFISHER_PARAGUAY_DNCP_REQUEST_TOKEN: To get an API account and request token go to https://contrataciones.gov.py/datos/adm/login.

Swagger API documentation

https://contrataciones.gov.py/datos/api/v3/doc

env KINGFISHER_PARAGUAY_DNCP_REQUEST_TOKEN=... scrapy crawl paraguay_dncp_records

class kingfisher_scrapy.spiders.paraguay_dncp_releases.ParaguayDNCPReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Dirección Nacional de Contrataciones Públicas (DNCP)

Spider arguments

from_date: Download only data from this time onward (YYYY-MM-DDThh:mm:ss format). Defaults to ‘2010-01-01T00:00:00’.
until_date: Download only data until this time (YYYY-MM-DDThh:mm:ss format). Defaults to now.

Environment variables

KINGFISHER_PARAGUAY_DNCP_REQUEST_TOKEN: To get an API account and request token go to https://contrataciones.gov.py/datos/adm/login.

Swagger API documentation

https://contrataciones.gov.py/datos/api/v3/doc

env KINGFISHER_PARAGUAY_DNCP_REQUEST_TOKEN=... scrapy crawl paraguay_dncp_releases

class kingfisher_scrapy.spiders.paraguay_hacienda.ParaguayHacienda(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Ministerio de Hacienda

Environment variables

KINGFISHER_PARAGUAY_HACIENDA_REQUEST_TOKEN: To get an API account and request token go to https://datos.hacienda.gov.py/aplicaciones/new.
KINGFISHER_PARAGUAY_HACIENDA_CLIENT_SECRET: Your client secret generated.

Swagger API documentation

https://datos.hacienda.gov.py/odmh-api-v1/api-docs/

env KINGFISHER_PARAGUAY_HACIENDA_REQUEST_TOKEN=... KINGFISHER_PARAGUAY_HACIENDA_CLIENT_SECRET=... scrapy crawl paraguay_hacienda

Peru¶

class kingfisher_scrapy.spiders.peru_compras.PeruCompras(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Peru Compras - Contracts from Framework Agreements

Spider arguments

from_date: Download only data from this date onward (YYYY-MM-DD format). Defaults to ‘2017-01-01’.
until_date: Download only data until this date (YYYY-MM-DD format). Defaults to today.

scrapy crawl peru_compras

Portugal¶

class kingfisher_scrapy.spiders.portugal.Portugal(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Instituto dos Mercados Públicos, do Imobiliário e da Construção (IMPIC)
API documentation: https://dados.gov.pt/pt/apidoc/
Bulk download documentation: https://dados.gov.pt/pt/datasets/ocds-portal-base-www-base-gov-pt/
Swagger API documentation: https://dados.gov.pt/pt/apidoc/#/datasets

scrapy crawl portugal

class kingfisher_scrapy.spiders.portugal_records.PortugalRecords(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Instituto dos Mercados Públicos, do Imobiliário e da Construção (IMPIC)

Spider arguments

from_date: Download only data with a contract signing date from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2010-01-01’.
until_date: Download only data with a contract signing date until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.

Swagger API documentation

http://www.base.gov.pt/swagger/index.html

scrapy crawl portugal_records

class kingfisher_scrapy.spiders.portugal_releases.PortugalReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Instituto dos Mercados Públicos, do Imobiliário e da Construção (IMPIC)

Spider arguments

from_date: Download only data with a contract signing date from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2010-01-01’.
until_date: Download only data with a contract signing date until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.

Swagger API documentation

http://www.base.gov.pt/swagger/index.html

scrapy crawl portugal_releases

Scotland¶

class kingfisher_scrapy.spiders.scotland_public_contracts.ScotlandPublicContracts(*args, **kwargs)[source]¶

Domain

Public Contracts Scotland

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). Defaults to a year ago.
until_date: Download only data until this month (YYYY-MM format). Defaults to the current month.

API documentation

https://api.publiccontractsscotland.gov.uk/v1

scrapy crawl scotland_public_contracts

Slovenia¶

class kingfisher_scrapy.spiders.slovenia.Slovenia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Ministry of Public Administration Slovenia
Bulk download documentation: http://tbfy.ijs.si/?m=tenders&a=list_data&t=si_ocds

scrapy crawl slovenia

Spain¶

class kingfisher_scrapy.spiders.spain_zaragoza.SpainZaragoza(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Ayuntamiento de Zaragoza
Caveats: The API’s before and after query string parameters have no effect and are therefore not implemented.
Swagger API documentation: https://www.zaragoza.es/docs-api_sede/

scrapy crawl spain_zaragoza

Tanzania¶

class kingfisher_scrapy.spiders.tanzania_zabuni.TanzaniaZabuni(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Tanzania Zabuni
API documentation: https://zabuni.co.tz/docs

scrapy crawl tanzania_zabuni

Uganda¶

class kingfisher_scrapy.spiders.uganda_releases.Uganda(*args, **kwargs)[source]¶

Domain

Government Procurement Portal (GPP) of Public Procurement and Disposal of Public Assets Authority (PPDA)

Caveats

The domains described in the API documentation must be replaced by https://gpppapi.com

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2017’. The year refers to the start of the fiscal year range, e.g. if from_date = ‘2017’ then the fiscal year is ‘2017-2018’
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year. The year refers to the start of the fiscal year range, e.g. if until_date = ‘2017’ then the fiscal year is ‘2017-2018’

API documentation

https://docs.google.com/spreadsheets/d/10tVioy-VOQa1FwWoRl5e1pMbGpiymA0iycNcoDFkvks/edit#gid=365266172

scrapy crawl uganda_releases

Uk¶

class kingfisher_scrapy.spiders.uk_contracts_finder.UKContractsFinder(*args, **kwargs)[source]¶

Domain: Contracts Finder

scrapy crawl uk_contracts_finder

class kingfisher_scrapy.spiders.uk_fts.UKFTS(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Find a Tender Service (FTS)

Spider arguments

from_date: Download only data from this time onward (YYYY-MM-DDThh:mm:ss format). If until_date is provided, defaults to ‘2021-01-01T00:00:00’.
until_date: Download only data until this time (YYYY-MM-DDThh:mm:ss format). If from_date is provided, defaults to now.

API documentation

https://www.find-tender.service.gov.uk/apidocumentation/1.0/GET-ocdsReleasePackages

scrapy crawl uk_fts

class kingfisher_scrapy.spiders.uk_fts_test.UKFTSTest(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain: Find a Tender Service (FTS)
Caveats: This spider uses a test service that returns test data. For getting real data see uk_fts instead.

scrapy crawl uk_fts_test

Uruguay¶

class kingfisher_scrapy.spiders.uruguay_historical.UruguayHistorical(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Agencia Reguladora de Compras Estatales (ARCE)

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2002’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

Bulk download documentation

https://www.gub.uy/agencia-compras-contrataciones-estado/datos-y-estadisticas/datos/open-contracting

scrapy crawl uruguay_historical

class kingfisher_scrapy.spiders.uruguay_records.UruguayRecords(*args, **kwargs)[source]¶

Domain

Agencia Reguladora de Compras Estatales (ARCE)

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). Defaults to ‘2017-11’.
until_date: Download only data until this month (YYYY-MM format). Defaults to the current month.

API documentation

https://www.gub.uy/agencia-compras-contrataciones-estado/datos-y-estadisticas/datos/open-contracting

scrapy crawl uruguay_records

class kingfisher_scrapy.spiders.uruguay_releases.UruguayReleases(*args, **kwargs)[source]¶

Domain

Agencia Reguladora de Compras Estatales (ARCE)

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). Defaults to ‘2017-11’.
until_date: Download only data until this month (YYYY-MM format). Defaults to the current month.

API documentation

https://www.gub.uy/agencia-compras-contrataciones-estado/datos-y-estadisticas/datos/open-contracting

scrapy crawl uruguay_releases

Zambia¶

class kingfisher_scrapy.spiders.zambia.Zambia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]¶

Domain

Zambia Public Procurement Authority

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). If until_date is provided, defaults to ‘2016-07’.
until_date: Download only data until this month (YYYY-MM format). If from_date is provided, defaults to the current month.

scrapy crawl zambia