Spiders

In Kingfisher Collect, there is one spider per data source. Each data source exposes its data in different ways, contains data from different years, and covers different stages of the contracting process.

Spider metadata

Below, we provide some of this information (metadata) for each spider. The meaning of the metadata is:

Metadata Description
Domain

The name of the IT system, jurisdiction, agency or program that is publishing the data. Some spiders have the same domain, in which case the difference between them is indicated in the suffix to the spider’s name:

Suffix Description
_releases Downloads individual releases or release packages
_records Downloads individual records or record packages
_bulk The data source is bulk data
_historical The data source is historical data
Caveats Any noteworthy API or dataset limitations.
Spider arguments The Spider arguments the spider supports.
Environment variables

The environment variables required to run the spider. If a data source requires account registration, a link is provided. You can set environment variables as, for example:

env NAME1=VALUE1 NAME2=VALUE2 scrapy crawl spider_name
API documentation The URL of the API’s documentation.
Bulk download documentation The URL of any bulk download documentation.
Swagger API documentation The URL of the API’s documentation, using Swagger.
API endpoints If official API documentation is unavailable, we provide some brief documentation.

Spider arguments

A spider’s behavior can be changed by setting spider arguments:

scrapy crawl colombia -a NAME=VALUE

You can set multiple spider arguments:

scrapy crawl colombia -a from_date=2015-01-01 -a until_date=2019-12-31

All spiders support these arguments:

Name Description Format
sample Set the number of files and file items to download. integer

Some spiders support these arguments:

Name Description Format
from_date Download data from this date onward. The spider might describe specific semantics (e.g. the relevant date is the date the contract was signed). Otherwise, the relevant date is the release date. See spider
until_date Download data until this date. The spider might describe specific semantics (e.g. the relevant date is the date the contract was signed). Otherwise, the relevant date is the release date. See spider
start_page Download data from this page onward. integer
publisher Download data by this publisher. See spider
system Download data from this system. See spider

Afghanistan

class kingfisher_scrapy.spiders.afghanistan_record_packages.AfghanistanRecordPackages(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Afghanistan Government Electronic & Open Procurement System (AGEOPS)
Spider arguments
from_date
Download only data from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2018-12-15’.
until_date
Download only data until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.
API documentation
https://ocds.ageops.net/
scrapy crawl afghanistan_record_packages
class kingfisher_scrapy.spiders.afghanistan_records.AfghanistanRecords(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Afghanistan Government Electronic & Open Procurement System (AGEOPS)
API documentation
https://ocds.ageops.net/
scrapy crawl afghanistan_records
class kingfisher_scrapy.spiders.afghanistan_release_packages.AfghanistanReleasePackages(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Afghanistan Government Electronic & Open Procurement System (AGEOPS)
Spider arguments
from_date
Download only data from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2018-12-15’.
until_date
Download only data until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.
API documentation
https://ocds.ageops.net/
scrapy crawl afghanistan_release_packages
class kingfisher_scrapy.spiders.afghanistan_releases.AfghanistanReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Afghanistan Government Electronic & Open Procurement System (AGEOPS)
Spider arguments
from_date
Download only data from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2018-12-12’.
until_date
Download only data until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.
API documentation
https://ocds.ageops.net/
scrapy crawl afghanistan_releases

Argentina

class kingfisher_scrapy.spiders.argentina_buenos_aires.ArgentinaBuenosAires(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Ciudad de Buenos Aires
API documentation
https://data.buenosaires.gob.ar/acerca/ckan
Bulk download documentation
https://data.buenosaires.gob.ar/dataset/buenos-aires-compras/archivo/2a3d077c-71b6-4ba7-8924-f3e38cf1b8fc
scrapy crawl argentina_buenos_aires
class kingfisher_scrapy.spiders.argentina_vialidad.ArgentinaVialidad(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Vialidad Nacional of Ministerio de Obras Públicas
API documentation
https://datosabiertos.vialidad.gob.ar/ui/index.html#!/datos_abiertos
scrapy crawl argentina_vialidad

Armenia

class kingfisher_scrapy.spiders.armenia.Armenia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Armenian e-Procurement System (ARMEPS)
Caveats
The API paginates results using an offset query string parameter, which is a timestamp. If a timestamp causes an error, the spider will try to find the nearest timestamp within the following 1024 days that succeeds.
scrapy crawl armenia

Australia

class kingfisher_scrapy.spiders.australia.Australia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
AusTender
Spider arguments
from_date
Download only data from this time onward (YYYY-MM-DDThh:mm:ss format). Defaults to ‘2004-01-01T00:00:00’.
until_date
Download only data until this time (YYYY-MM-DDThh:mm:ss format). Defaults to now.
API documentation
https://github.com/austender/austender-ocds-api
Swagger API documentation
https://app.swaggerhub.com/apis/austender/ocds-api/1.1
scrapy crawl australia
class kingfisher_scrapy.spiders.australia_nsw.AustraliaNSW(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
New South Wales (NSW)
API documentation
https://github.com/NSW-eTendering/NSW-eTendering-API/blob/master/README.md
scrapy crawl australia_nsw

Bolivia

class kingfisher_scrapy.spiders.bolivia_agetic.BoliviaAgetic(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Agencia de Gobierno Electrónico y Tecnologías de Información y Comunicación (AGETIC)
Bulk download documentation
https://datos.gob.bo/id/dataset/contrataciones-agetic-2019-estandar-ocp
scrapy crawl bolivia_agetic

Canada

class kingfisher_scrapy.spiders.canada_buyandsell.CanadaBuyAndSell(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Public Works and Government Services Canada
Caveats
The dataset is a pilot.
API documentation
https://buyandsell.gc.ca/procurement-data/open-contracting-data-standard-pilot/download-ocds-pilot-data
scrapy crawl canada_buyandsell
class kingfisher_scrapy.spiders.canada_montreal.CanadaMontreal(*args, **kwargs)[source]
Domain
Montréal, Québec
API documentation
http://donnees.ville.montreal.qc.ca/dataset/contrats-et-subventions-api
scrapy crawl canada_montreal
class kingfisher_scrapy.spiders.canada_quebec.CanadaQuebec(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Secrétariat du Conseil du trésor
Bulk download documentation
https://www.donneesquebec.ca/recherche/dataset/systeme-electronique-dappel-doffres-seao
scrapy crawl canada_quebec

Chile

class kingfisher_scrapy.spiders.chile_compra_bulk.ChileCompraBulk(*args, **kwargs)[source]
Domain
ChileCompra
Spider arguments
from_date
Download only data from this month onward (YYYY-MM format). Defaults to ‘2009-01’.
until_date
Download only data until this month (YYYY-MM format). Defaults to the current month.
Bulk download documentation
https://desarrolladores.mercadopublico.cl/OCDS/DescargaMasiva
scrapy crawl chile_compra_bulk
class kingfisher_scrapy.spiders.chile_compra_records.ChileCompraRecords(*args, **kwargs)[source]
Domain
ChileCompra
Spider arguments
from_date
Download only data from this month onward (YYYY-MM format). Defaults to ‘2009-01’.
until_date
Download only data until this month (YYYY-MM format). Defaults to the current month.
system
Filter by system, if not set gets all:
convenio
Framework agreements only
trato-directo
Direct tenders only
licitacion
Traditional tenders
Swagger API documentation
https://desarrolladores.mercadopublico.cl/docs/services/5a9ed02f718ed712f4403e75/operations/data-listaa-omes-agno-mes
scrapy crawl chile_compra_records
class kingfisher_scrapy.spiders.chile_compra_releases.ChileCompraReleases(*args, **kwargs)[source]
Domain
ChileCompra
Spider arguments
from_date
Download only data from this month onward (YYYY-MM format). Defaults to ‘2009-01’.
until_date
Download only data until this month (YYYY-MM format). Defaults to the current month.
system
Filter by system, if not set gets all:
convenio
Framework agreements only
trato-directo
Direct tenders only
licitacion
Traditional tenders
Swagger API documentation
https://desarrolladores.mercadopublico.cl/docs/services/5a9ed02f718ed712f4403e75/operations/data-listaa-omes-agno-mes
scrapy crawl chile_compra_releases

Colombia

class kingfisher_scrapy.spiders.colombia.Colombia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Colombia Compra Eficiente (CCE)
Spider arguments
from_date
Download only data from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2011-01-01’.
until_date
Download only data until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.
start_page
The page number from which to start crawling.
API documentation
https://www.colombiacompra.gov.co/transparencia/api
Swagger API documentation
https://apiocds.colombiacompra.gov.co:8443/apiCCE2.0/
scrapy crawl colombia
class kingfisher_scrapy.spiders.colombia_bulk.ColombiaBulk(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Colombia Compra Eficiente (CCE)
Spider arguments
from_date
Download only data from this year onward (YYYY format). Defaults to ‘2011’. Only supported for ‘SECOP1’ system.
until_date
Download only data until this year (YYYY format). Defaults to the current year. Only supported for ‘SECOP1’ system.
system

Filter by system:

SECOP1
Sistema Electrónico para la Contratación Pública (todos los procesos de contratación)
SECOP2
Sistema Electrónico para la Contratación Pública (todos los procesos de contratación)
TVEC
Tienda Virtual del Estado Colombiano (acuerdos marco, agregación de demanda, mínima cuantía)
Bulk download documentation
https://www.colombiacompra.gov.co/transparencia/datos-json
scrapy crawl colombia_bulk

Costa Rica

class kingfisher_scrapy.spiders.costa_rica_poder_judicial_records.CostaRicaPoderJudicialRecords(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Poder Judicial de Costa Rica
Spider arguments
from_date
Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2018’.
until_date
Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.
Bulk download documentation
http://datosabiertospj.eastus.cloudapp.azure.com/dataset/estandar-de-datos-de-contrataciones-abiertas-ocds
scrapy crawl costa_rica_poder_judicial_records
class kingfisher_scrapy.spiders.costa_rica_poder_judicial_releases.CostaRicaPoderJudicialReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Poder Judicial de Costa Rica
Bulk download documentation
http://datosabiertospj.eastus.cloudapp.azure.com/dataset/estandar-de-datos-de-contrataciones-abiertas-ocds
scrapy crawl costa_rica_poder_judicial_releases

Croatia

class kingfisher_scrapy.spiders.croatia.Croatia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Electronic Public Procurement Classifieds of the Republic of Croatia
Bulk download documentation
https://eojn.nn.hr/SPIN/application/ipn/Oglasnik/PreuzimanjeUgovoraOCD.aspx
scrapy crawl croatia

Digiwhist

class kingfisher_scrapy.spiders.digiwhist_austria.DigiwhistAustria(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_austria
class kingfisher_scrapy.spiders.digiwhist_belgium.DigiwhistBelgium(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_belgium
class kingfisher_scrapy.spiders.digiwhist_bulgaria.DigiwhistBulgaria(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_bulgaria
class kingfisher_scrapy.spiders.digiwhist_croatia.DigiwhistCroatia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_croatia
class kingfisher_scrapy.spiders.digiwhist_cyprus.DigiwhistCyprus(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_cyprus
class kingfisher_scrapy.spiders.digiwhist_czech_republic.DigiwhistCzechRepublic(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_czech_republic
class kingfisher_scrapy.spiders.digiwhist_denmark.DigiwhistDenmark(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_denmark
class kingfisher_scrapy.spiders.digiwhist_estonia.DigiwhistEstonia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_estonia
class kingfisher_scrapy.spiders.digiwhist_finland.DigiwhistFinland(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_finland
class kingfisher_scrapy.spiders.digiwhist_france.DigiwhistFrance(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_france
class kingfisher_scrapy.spiders.digiwhist_georgia.DigiwhistGeorgia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_georgia
class kingfisher_scrapy.spiders.digiwhist_germany.DigiwhistGermany(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_germany
class kingfisher_scrapy.spiders.digiwhist_greece.DigiwhistGreece(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_greece
class kingfisher_scrapy.spiders.digiwhist_hungary.DigiwhistHungary(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_hungary
class kingfisher_scrapy.spiders.digiwhist_iceland.DigiwhistIceland(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_iceland
class kingfisher_scrapy.spiders.digiwhist_ireland.DigiwhistIreland(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_ireland
class kingfisher_scrapy.spiders.digiwhist_italy.DigiwhistItaly(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_italy
class kingfisher_scrapy.spiders.digiwhist_latvia.DigiwhistLatvia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_latvia
class kingfisher_scrapy.spiders.digiwhist_lithuania.DigiwhistLithuania(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_lithuania
class kingfisher_scrapy.spiders.digiwhist_luxembourg.DigiwhistLuxembourg(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_luxembourg
class kingfisher_scrapy.spiders.digiwhist_malta.DigiwhistMalta(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_malta
class kingfisher_scrapy.spiders.digiwhist_netherlands.DigiwhistNetherlands(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_netherlands
class kingfisher_scrapy.spiders.digiwhist_norway.DigiwhistNorway(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_norway
class kingfisher_scrapy.spiders.digiwhist_poland.DigiwhistPoland(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_poland
class kingfisher_scrapy.spiders.digiwhist_portugal.DigiwhistPortugal(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_portugal
class kingfisher_scrapy.spiders.digiwhist_romania.DigiwhistRomania(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_romania
class kingfisher_scrapy.spiders.digiwhist_slovakia.DigiwhistSlovakia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_slovakia
class kingfisher_scrapy.spiders.digiwhist_slovenia.DigiwhistSlovenia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_slovenia
class kingfisher_scrapy.spiders.digiwhist_spain.DigiwhistSpain(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_spain
class kingfisher_scrapy.spiders.digiwhist_sweden.DigiwhistSweden(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_sweden
class kingfisher_scrapy.spiders.digiwhist_switzerland.DigiwhistSwitzerland(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_switzerland
class kingfisher_scrapy.spiders.digiwhist_ted.DigiwhistTED(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_ted
class kingfisher_scrapy.spiders.digiwhist_united_kingdom.DigiwhistUnitedKingdom(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
scrapy crawl digiwhist_united_kingdom

Dominican Republic

class kingfisher_scrapy.spiders.dominican_republic.DominicanRepublic(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Dirección General de Contrataciones Públicas (DGCP)
Spider arguments
from_date
Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2018’.
until_date
Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.
Bulk download documentation
https://www.dgcp.gob.do/estandar-mundial-ocds/
scrapy crawl dominican_republic
class kingfisher_scrapy.spiders.dominican_republic_api.DominicanRepublicPortal(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Dirección General de Contrataciones Públicas (DGCP)
Spider arguments
from_date
Download only data from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2018-01-01’.
until_date
Download only data until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.
API documentation
http://148.101.176.123:48080/ocdsdr/docs
scrapy crawl dominican_republic_api

Ecuador

class kingfisher_scrapy.spiders.ecuador_emergency.EcuadorEmergency(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Servicio Nacional de Contratación Pública
Bulk download documentation
https://portal.compraspublicas.gob.ec/sercop/data-estandar-ocds/
scrapy crawl ecuador_emergency

Europe

class kingfisher_scrapy.spiders.europe_ted_tender_base.EuropeTedTenderBase(*args, **kwargs)[source]
Domain
TenderBase
API endpoints
Get releases by page
Link
http://www.tenderbase.eu/api/releases/?page={page}
Parameters
page
page number
scrapy crawl europe_ted_tender_base

France

class kingfisher_scrapy.spiders.france.France(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
France
Swagger API documentation
https://doc.data.gouv.fr/api/reference/
scrapy crawl france

Georgia

class kingfisher_scrapy.spiders.georgia_opendata.GeorgiaOpenData(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
State Procurement Agency (SPA)
scrapy crawl georgia_opendata
class kingfisher_scrapy.spiders.georgia_records.GeorgiaRecords(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
State Procurement Agency (SPA)
Bulk download documentation
https://odapi.spa.ge/publication-policy
Swagger API documentation
https://odapi.spa.ge/api/swagger.ui
scrapy crawl georgia_records
class kingfisher_scrapy.spiders.georgia_releases.GeorgiaReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
State Procurement Agency (SPA)
Bulk download documentation
https://odapi.spa.ge/publication-policy
Swagger API documentation
https://odapi.spa.ge/api/swagger.ui
scrapy crawl georgia_releases

Honduras

class kingfisher_scrapy.spiders.honduras_cost.HondurasCoST(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Systema de Información y Seguimiento de Obras y Contratos (SISOCS)
scrapy crawl honduras_cost
class kingfisher_scrapy.spiders.honduras_oncae.HondurasONCAE(*args, **kwargs)[source]
Domain
Oficina Normativa de Contratación y Adquisiciones del Estado (ONCAE)
Spider arguments
from_date
Download only data from this year onward (YYYY format). Defaults to ‘2005’.
until_date
Download only data until this year (YYYY format). Defaults to the current year.
system

Filter by system:

CE
Catálogo Electrónico
DDC
Módulo de Difusión Directa de Contratos
HC1
HonduCompras 1.0 (Módulo de Difusión de Compras y Contrataciones)
Bulk download documentation
http://oncae.gob.hn/datosabiertos
scrapy crawl honduras_oncae
class kingfisher_scrapy.spiders.honduras_portal_bulk.HondurasPortalBulk(*args, **kwargs)[source]
Domain
Oficina Normativa de Contratación y Adquisiciones del Estado (ONCAE) / Secretaria de Finanzas de Honduras (SEFIN)
Spider arguments
from_date
Download only data from this month onward (YYYY-MM format). Defaults to ‘2005-11’.
until_date
Download only data until this month (YYYY-MM format). Defaults to the current month.
publisher

Filter by publisher:

oncae
Oficina Normativa de Contratación y Adquisiciones del Estado
sefin
Secretaria de Finanzas de Honduras
system

Filter by oncae system:

CE
Catálogo Electrónico
DDC
Módulo de Difusión Directa de Contratos
HC1
HonduCompras 1.0 (Módulo de Difusión de Compras y Contrataciones)
Bulk download documentation
http://www.contratacionesabiertas.gob.hn/descargas/
scrapy crawl honduras_portal_bulk
class kingfisher_scrapy.spiders.honduras_portal_records.HondurasPortalRecords(*args, **kwargs)[source]
Domain
Oficina Normativa de Contratación y Adquisiciones del Estado (ONCAE) / Secretaria de Finanzas de Honduras (SEFIN)
Spider arguments
publisher

Filter by publisher:

oncae
Oficina Normativa de Contratación y Adquisiciones del Estado
sefin
Secretaria de Finanzas de Honduras
API documentation
http://www.contratacionesabiertas.gob.hn/manual_api/
Swagger API documentation
http://www.contratacionesabiertas.gob.hn/servicio/
scrapy crawl honduras_portal_records
class kingfisher_scrapy.spiders.honduras_portal_releases.HondurasPortalReleases(*args, **kwargs)[source]
Domain
Oficina Normativa de Contratación y Adquisiciones del Estado (ONCAE) / Secretaria de Finanzas de Honduras (SEFIN)
Spider arguments
publisher

Filter by publisher:

oncae
Oficina Normativa de Contratación y Adquisiciones del Estado
sefin
Secretaria de Finanzas de Honduras
API documentation
http://www.contratacionesabiertas.gob.hn/manual_api/
Swagger API documentation
http://www.contratacionesabiertas.gob.hn/servicio/
scrapy crawl honduras_portal_releases

India

class kingfisher_scrapy.spiders.india_civic_data_lab.IndiaCivicDataLab(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Civic Data Lab India
Bulk download documentation
https://github.com/CivicDataLab/himachal-pradesh-health-procurement-OCDS/
scrapy crawl india_civic_data_lab

Indonesia

class kingfisher_scrapy.spiders.indonesia_bandung.IndonesiaBandung(*args, **kwargs)[source]
Domain
Bandung Integrated Resource Management System (BIRMS)
Spider arguments
from_date
Download only data from this year onward (YYYY format). Defaults to ‘2013’.
until_date
Download only data until this year (YYYY format). Defaults to the current year.
API endpoints
Get contract list by year
Link
https://birms.bandung.go.id/api/contracts/year/{year}?page={page}
Parameters
year
contract year number
page
page number
Get contract detail by OCID
Link
https://birms.bandung.go.id/api/newcontract/ocds-afzrfb-{source-database}-{year}-{uniqid}
Parameters
source-database
(b)irms or (s)pse
year
contract year number
uniqid
release id
scrapy crawl indonesia_bandung
class kingfisher_scrapy.spiders.indonesia_opentender.IndonesiaOpentender(*args, **kwargs)[source]
Domain
Open Tender
Spider arguments
from_date
Download only data from this year onward (YYYY format). Defaults to ‘2008’.
until_date
Download only data until this year (YYYY format). Defaults to the current year.
Bulk download documentation
https://v3.opentender.net/#/ocds
scrapy crawl indonesia_opentender

Italy

class kingfisher_scrapy.spiders.italy.Italy(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
AppaltiPOP
Bulk download documentation
https://www.appaltipop.it/it/download
scrapy crawl italy

Kenya

class kingfisher_scrapy.spiders.kenya_makueni.KenyaMakueni(*args, **kwargs)[source]
Domain
Makueni County
Swagger API documentation
https://opencontracting.makueni.go.ke/swagger-ui.html#/ocds-controller
scrapy crawl kenya_makueni

Kosovo

class kingfisher_scrapy.spiders.kosovo.Kosovo(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Public Procurement Regulatory Commission
Spider arguments
from_date
Download only data from this time onward (YYYY-MM-DDThh:mm:ss format). If until_date is provided, defaults to ‘2000-01-01T00:00:00’.
until_date
Download only data until this time (YYYY-MM-DDThh:mm:ss format). If from_date is provided, defaults to now.
API documentation
https://ocdskrpp-test.rks-gov.net/Help
scrapy crawl kosovo

Kyrgyzstan

class kingfisher_scrapy.spiders.kyrgyzstan.Kyrgyzstan(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Ministry of Finance
Caveats
The planning endpoint is not implemented because is not completely in OCDS.
Bulk download documentation
https://analytics.zakupki.gov.kg/dashboard/weekly and then to ‘API & EXPORT’. Direct access to https://analytics.zakupki.gov.kg/api-export doesn’t work.
Swagger API documentation
https://app.swaggerhub.com/apis/DPAteam/export-api-documentation/1.0
scrapy crawl kyrgyzstan

Malta

class kingfisher_scrapy.spiders.malta.Malta(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Malta
Spider arguments
from_date
Download only data from this month onward (YYYY-MM format). If until_date is provided, defaults to ‘2019-10’.
until_date
Download only data until this month (YYYY-MM format). If from_date is provided, defaults to the current month.
API documentation
https://docs.google.com/document/d/1VnCEywKkkQ7BcVbT7HlW2s_N_QI8W0KE/edit
scrapy crawl malta

Mexico

class kingfisher_scrapy.spiders.mexico_administracion_publica_federal_api.MexicoAdministracionPublicaFederalAPI(*args, **kwargs)[source]
Domain
Administración Pública Federal (APF): Secretaria de la Función Pública (SFP) - Secretaría de Hacienda y Crédito Público (SHCP)
API documentation
https://www.datos.gob.mx/busca/dataset/api-de-contrataciones-abiertas-de-la-apf
scrapy crawl mexico_administracion_publica_federal_api
class kingfisher_scrapy.spiders.mexico_administracion_publica_federal_bulk.MexicoAdministracionPublicaFederalBulk(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Administración Pública Federal (APF): Secretaría de Hacienda y Crédito Público (SHCP)
Bulk download documentation
https://datos.gob.mx/busca/dataset/concentrado-de-contrataciones-abiertas-de-la-apf
scrapy crawl mexico_administracion_publica_federal_bulk
class kingfisher_scrapy.spiders.mexico_grupo_aeroporto.MexicoGrupoAeroporto(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Grupo Aeroportuario de la Ciudad de México (CDMX)
scrapy crawl mexico_grupo_aeroporto
class kingfisher_scrapy.spiders.mexico_idaip.MexicoIDAIP(*args, **kwargs)[source]
Domain
Instituto Duranguense de Acceso a la Información y de Protección de Datos Personales (IDAIP)
Spider arguments
from_date
Download only data from this year onward (YYYY format). Defaults to ‘2020’.
until_date
Download only data until this year (YYYY format). Defaults to ‘2021’.
API documentation
http://contratacionesabiertas.idaip.org.mx/contratacionesabiertas/datosabiertos
scrapy crawl mexico_idaip
class kingfisher_scrapy.spiders.mexico_inai_api.MexicoINAIAPI(*args, **kwargs)[source]
Domain
Instituto Nacional de Transparencia, Acceso a la Información y Protección de Datos Personales (INAI)
Spider arguments
from_date
Download only data from this year onward (YYYY format). Defaults to ‘2015’.
until_date
Download only data until this year (YYYY format). Defaults to ‘2020’.
API documentation
http://contratacionesabiertas.inai.org.mx/contratacionesabiertas/datosabiertos
scrapy crawl mexico_inai_api
class kingfisher_scrapy.spiders.mexico_nuevo_leon_records.MexicoNuevoLeonRecords(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Secretaría de Infraestructura del Gobierno del Estado de Nuevo León
Bulk download documentation
http://si.nl.gob.mx/transparencia/publicaciones
scrapy crawl mexico_nuevo_leon_records
class kingfisher_scrapy.spiders.mexico_nuevo_leon_releases.MexicoNuevoLeonReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Secretaría de Infraestructura del Gobierno del Estado de Nuevo León
Bulk download documentation
http://si.nl.gob.mx/transparencia/acerca-del-proyecto
scrapy crawl mexico_nuevo_leon_releases
class kingfisher_scrapy.spiders.mexico_plataforma_digital_nacional.MexicoPlataformaDigitalNacional(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Plataforma Digital Nacional
Bulk download documentation
https://plataformadigitalnacional.org/contrataciones
scrapy crawl mexico_plataforma_digital_nacional
class kingfisher_scrapy.spiders.mexico_quien_es_quien.MexicoQuienEsQuien(*args, **kwargs)[source]
Domain
QuiénEsQuién.Wiki
API documentation
https://qqwapi-elastic.readthedocs.io/es/latest/
Swagger API documentation
https://api.quienesquien.wiki/v3/docs/
scrapy crawl mexico_quien_es_quien
class kingfisher_scrapy.spiders.mexico_yucatan_inaip.MexicoYucatan(*args, **kwargs)[source]
Domain
Instituto Estatal de Transparencia, Acceso a la Información Pública y Protección de Datos Personales (INAIP) - Yucatán
Spider arguments
from_date
Download only data from this year onward (YYYY format). Defaults to ‘2020’.
until_date
Download only data until this year (YYYY format). Defaults to ‘2020’.
API documentation
https://contratacionesabiertas.inaipyucatan.org.mx/contratacionesabiertas/datosabiertos#
scrapy crawl mexico_yucatan_inaip

Moldova

class kingfisher_scrapy.spiders.moldova.Moldova(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
MTender
Spider arguments
from_date
Download only data from this time onward (YYYY-MM-DDThh:mm:ss format).
scrapy crawl moldova
class kingfisher_scrapy.spiders.moldova_old.MoldovaOld(*args, **kwargs)[source]
Domain
Public Procurement Agency (PPA)
Spider arguments
from_date
Download only data from this year onward (YYYY format). Defaults to ‘2012’.
until_date
Download only data until this year (YYYY format). Defaults to ‘2017’.
Bulk download documentation
http://opencontracting.date.gov.md/downloads
scrapy crawl moldova_old
class kingfisher_scrapy.spiders.moldova_positive_initiative.MoldovaPositiveInitiative(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Positive Initiative - Moldova
Bulk download documentation
https://www.tender.health/ocdsrelease
scrapy crawl moldova_positive_initiative

Nepal

class kingfisher_scrapy.spiders.nepal_dhangadhi.NepalDhangadhi(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Dhangadhi Infrastructure Management System (IMS)
Caveats
Some URLs listed in https://admin.ims.susasan.org/api/static-data/dhangadhi require login and cannot be downloaded
Bulk download documentation
https://ims.susasan.org/dhangadhi/about
scrapy crawl nepal_dhangadhi
class kingfisher_scrapy.spiders.nepal_portal.NepalPortal(*args, **kwargs)[source]
Domain
Public Procurement Monitoring Office (PPMO)
Spider arguments
from_date
Download only data from this year onward (YYYY format). Defaults to ‘2012’.
until_date
Download only data until this year (YYYY format). Defaults to ‘2018’.
Bulk download documentation
http://ppip.gov.np/downloads
scrapy crawl nepal_portal

Nicaragua

class kingfisher_scrapy.spiders.nicaragua_solid_waste.NicaraguaSolidWaste(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Solid Waste Mitigation Platform (SWMP)
Spider arguments
from_date
Download only data from this date onward (YYYY-MM-DD format). Defaults to ‘2000-01-01’.
until_date
Download only data until this date (YYYY-MM-DD format). Defaults to today.
scrapy crawl nicaragua_solid_waste

Nigeria

class kingfisher_scrapy.spiders.nigeria_budeshi_records.NigeriaBudeshiRecords(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Budeshi Nigeria
API documentation
https://budeshi.ng/Api
scrapy crawl nigeria_budeshi_records
class kingfisher_scrapy.spiders.nigeria_budeshi_releases.NigeriaBudeshiReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Budeshi Nigeria
API documentation
https://budeshi.ng/Api
scrapy crawl nigeria_budeshi_releases
class kingfisher_scrapy.spiders.nigeria_portal.NigeriaPortal(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Nigeria Open Contracting Portal (NOCOPO) of Bureau of Public Procurement (BPP)
scrapy crawl nigeria_portal

Openopps

class kingfisher_scrapy.spiders.openopps.OpenOpps(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
OpenOpps
Spider arguments
from_date
Download only data from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2011-01-01’.
until_date
Download only data until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.
Environment variables
KINGFISHER_OPENOPPS_USERNAME
To get an API account, contact contact@openopps.com.
KINGFISHER_OPENOPPS_PASSWORD
Your API account password.
API documentation
https://docs.google.com/document/d/1u0da3BTU7fBFjX6i7j_tKXa1YwdXL7hY4Kw9GdsaAr0/edit
Swagger API documentation
https://api.openopps.com/api/schema/
env KINGFISHER_OPENOPPS_USERNAME=... KINGFISHER_OPENOPPS_PASSWORD=... scrapy crawl openopps

Pakistan

class kingfisher_scrapy.spiders.pakistan_ppra_bulk.PakistanPPRABulk(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Pakistan Public Procurement Regulatory Authority (PPRA)
API documentation
https://www.ppra.org.pk/api/
scrapy crawl pakistan_ppra_bulk
class kingfisher_scrapy.spiders.pakistan_ppra_releases.PakistanPPRAReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Pakistan Public Procurement Regulatory Authority (PPRA)
API documentation
https://www.ppra.org.pk/api/
scrapy crawl pakistan_ppra_releases

Paraguay

class kingfisher_scrapy.spiders.paraguay_dncp_records.ParaguayDNCPRecords(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Dirección Nacional de Contrataciones Públicas (DNCP)
Spider arguments
from_date
Download only data from this time onward (YYYY-MM-DDThh:mm:ss format). Defaults to ‘2010-01-01T00:00:00’.
until_date
Download only data until this time (YYYY-MM-DDThh:mm:ss format). Defaults to now.
Environment variables
KINGFISHER_PARAGUAY_DNCP_REQUEST_TOKEN
To get an API account and request token go to https://contrataciones.gov.py/datos/adm/login.
Swagger API documentation
https://contrataciones.gov.py/datos/api/v3/doc
env KINGFISHER_PARAGUAY_DNCP_REQUEST_TOKEN=... scrapy crawl paraguay_dncp_records
class kingfisher_scrapy.spiders.paraguay_dncp_releases.ParaguayDNCPReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Dirección Nacional de Contrataciones Públicas (DNCP)
Spider arguments
from_date
Download only data from this time onward (YYYY-MM-DDThh:mm:ss format). Defaults to ‘2010-01-01T00:00:00’.
until_date
Download only data until this time (YYYY-MM-DDThh:mm:ss format). Defaults to now.
Environment variables
KINGFISHER_PARAGUAY_DNCP_REQUEST_TOKEN
To get an API account and request token go to https://contrataciones.gov.py/datos/adm/login.
Swagger API documentation
https://contrataciones.gov.py/datos/api/v3/doc
env KINGFISHER_PARAGUAY_DNCP_REQUEST_TOKEN=... scrapy crawl paraguay_dncp_releases
class kingfisher_scrapy.spiders.paraguay_hacienda.ParaguayHacienda(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Ministerio de Hacienda
Environment variables
KINGFISHER_PARAGUAY_HACIENDA_REQUEST_TOKEN
To get an API account and request token go to https://datos.hacienda.gov.py/aplicaciones/new.
KINGFISHER_PARAGUAY_HACIENDA_CLIENT_SECRET
Your client secret generated.
Swagger API documentation
https://datos.hacienda.gov.py/odmh-api-v1/api-docs/
env KINGFISHER_PARAGUAY_HACIENDA_REQUEST_TOKEN=... KINGFISHER_PARAGUAY_HACIENDA_CLIENT_SECRET=... scrapy crawl paraguay_hacienda

Peru

class kingfisher_scrapy.spiders.peru_compras.PeruCompras(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Peru Compras - Contracts from Framework Agreements
Spider arguments
from_date
Download only data from this date onward (YYYY-MM-DD format). Defaults to ‘2017-01-01’.
until_date
Download only data until this date (YYYY-MM-DD format). Defaults to today.
scrapy crawl peru_compras

Portugal

class kingfisher_scrapy.spiders.portugal.Portugal(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Instituto dos Mercados Públicos, do Imobiliário e da Construção (IMPIC)
API documentation
https://dados.gov.pt/pt/apidoc/
Bulk download documentation
https://dados.gov.pt/pt/datasets/ocds-portal-base-www-base-gov-pt/
Swagger API documentation
https://dados.gov.pt/pt/apidoc/#/datasets
scrapy crawl portugal
class kingfisher_scrapy.spiders.portugal_records.PortugalRecords(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Instituto dos Mercados Públicos, do Imobiliário e da Construção (IMPIC)
Spider arguments
from_date
Download only data with a contract signing date from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2010-01-01’.
until_date
Download only data with a contract signing date until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.
Swagger API documentation
http://www.base.gov.pt/swagger/index.html
scrapy crawl portugal_records
class kingfisher_scrapy.spiders.portugal_releases.PortugalReleases(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Instituto dos Mercados Públicos, do Imobiliário e da Construção (IMPIC)
Spider arguments
from_date
Download only data with a contract signing date from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2010-01-01’.
until_date
Download only data with a contract signing date until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.
Swagger API documentation
http://www.base.gov.pt/swagger/index.html
scrapy crawl portugal_releases

Scotland

class kingfisher_scrapy.spiders.scotland_public_contracts.ScotlandPublicContracts(*args, **kwargs)[source]
Domain
Public Contracts Scotland
Spider arguments
from_date
Download only data from this month onward (YYYY-MM format). Defaults to a year ago.
until_date
Download only data until this month (YYYY-MM format). Defaults to the current month.
API documentation
https://api.publiccontractsscotland.gov.uk/v1
scrapy crawl scotland_public_contracts

Slovenia

class kingfisher_scrapy.spiders.slovenia.Slovenia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Ministry of Public Administration Slovenia
Bulk download documentation
http://tbfy.ijs.si/?m=tenders&a=list_data&t=si_ocds
scrapy crawl slovenia

Spain

class kingfisher_scrapy.spiders.spain_zaragoza.SpainZaragoza(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Ayuntamiento de Zaragoza
Caveats
The API’s before and after query string parameters have no effect and are therefore not implemented.
Swagger API documentation
https://www.zaragoza.es/docs-api_sede/
scrapy crawl spain_zaragoza

Tanzania

class kingfisher_scrapy.spiders.tanzania_zabuni.TanzaniaZabuni(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Tanzania Zabuni
API documentation
https://zabuni.co.tz/docs
scrapy crawl tanzania_zabuni

Uganda

class kingfisher_scrapy.spiders.uganda_releases.Uganda(*args, **kwargs)[source]
Domain
Government Procurement Portal (GPP) of Public Procurement and Disposal of Public Assets Authority (PPDA)
Caveats
The domains described in the API documentation must be replaced by https://gpppapi.com
Spider arguments
from_date
Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2017’. The year refers to the start of the fiscal year range, e.g. if from_date = ‘2017’ then the fiscal year is ‘2017-2018’
until_date
Download only data until this year (YYYY format). If from_date is provided, defaults to the current year. The year refers to the start of the fiscal year range, e.g. if until_date = ‘2017’ then the fiscal year is ‘2017-2018’
API documentation
https://docs.google.com/spreadsheets/d/10tVioy-VOQa1FwWoRl5e1pMbGpiymA0iycNcoDFkvks/edit#gid=365266172
scrapy crawl uganda_releases

Uk

class kingfisher_scrapy.spiders.uk_contracts_finder.UKContractsFinder(*args, **kwargs)[source]
Domain
Contracts Finder
scrapy crawl uk_contracts_finder
class kingfisher_scrapy.spiders.uk_fts.UKFTS(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Find a Tender Service (FTS)
Spider arguments
from_date
Download only data from this time onward (YYYY-MM-DDThh:mm:ss format). If until_date is provided, defaults to ‘2021-01-01T00:00:00’.
until_date
Download only data until this time (YYYY-MM-DDThh:mm:ss format). If from_date is provided, defaults to now.
API documentation
https://www.find-tender.service.gov.uk/apidocumentation/1.0/GET-ocdsReleasePackages
scrapy crawl uk_fts
class kingfisher_scrapy.spiders.uk_fts_test.UKFTSTest(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Find a Tender Service (FTS)
Caveats
This spider uses a test service that returns test data. For getting real data see uk_fts instead.
scrapy crawl uk_fts_test

Uruguay

class kingfisher_scrapy.spiders.uruguay_historical.UruguayHistorical(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Agencia Reguladora de Compras Estatales (ARCE)
Spider arguments
from_date
Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2002’.
until_date
Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.
Bulk download documentation
https://www.gub.uy/agencia-compras-contrataciones-estado/datos-y-estadisticas/datos/open-contracting
scrapy crawl uruguay_historical
class kingfisher_scrapy.spiders.uruguay_records.UruguayRecords(*args, **kwargs)[source]
Domain
Agencia Reguladora de Compras Estatales (ARCE)
Spider arguments
from_date
Download only data from this month onward (YYYY-MM format). Defaults to ‘2017-11’.
until_date
Download only data until this month (YYYY-MM format). Defaults to the current month.
API documentation
https://www.gub.uy/agencia-compras-contrataciones-estado/datos-y-estadisticas/datos/open-contracting
scrapy crawl uruguay_records
class kingfisher_scrapy.spiders.uruguay_releases.UruguayReleases(*args, **kwargs)[source]
Domain
Agencia Reguladora de Compras Estatales (ARCE)
Spider arguments
from_date
Download only data from this month onward (YYYY-MM format). Defaults to ‘2017-11’.
until_date
Download only data until this month (YYYY-MM format). Defaults to the current month.
API documentation
https://www.gub.uy/agencia-compras-contrataciones-estado/datos-y-estadisticas/datos/open-contracting
scrapy crawl uruguay_releases

Zambia

class kingfisher_scrapy.spiders.zambia.Zambia(sample=None, note=None, from_date=None, until_date=None, crawl_time=None, keep_collection_open=None, package_pointer=None, release_pointer=None, truncate=None, compile_releases=None, *args, **kwargs)[source]
Domain
Zambia Public Procurement Authority
Spider arguments
from_date
Download only data from this month onward (YYYY-MM format). If until_date is provided, defaults to ‘2016-07’.
until_date
Download only data until this month (YYYY-MM format). If from_date is provided, defaults to the current month.
scrapy crawl zambia