Spiders#

In Kingfisher Collect, there is one spider per data source. Each data source exposes its data in different ways, contains data from different years, and covers different stages of the contracting process.

Spider metadata#

Below, we provide some of this information (metadata) for each spider. The meaning of the metadata is:

Metadata

Description

Domain

The name of the IT system, jurisdiction, agency or program that is publishing the data. Some spiders have the same domain, in which case the difference between them is indicated in the suffix to the spider’s name:

Suffix	Description
`_releases`	Downloads individual releases or release packages
`_records`	Downloads individual records or record packages
`_bulk`	The data source is bulk data
`_historical`	The data source is historical data

Caveats

Any noteworthy API or dataset limitations.

Spider arguments

The Spider arguments the spider supports.

Environment variables

The environment variables required to run the spider. If a data source requires account registration, a link is provided. You can set environment variables as, for example:

env NAME1=VALUE1 NAME2=VALUE2 scrapy crawl spider_name

API documentation

The URL of the API’s documentation.

Bulk download documentation

The URL of any bulk download documentation.

Swagger API documentation

The URL of the API’s documentation, using Swagger.

API endpoints

If official API documentation is unavailable, we provide some brief documentation.

Spider arguments#

A spider’s behavior can be changed by setting spider arguments:

scrapy crawl colombia -a NAME=VALUE

You can set multiple spider arguments:

scrapy crawl colombia -a from_date=2015-01-01 -a until_date=2019-12-31

All spiders support these arguments:

Name	Description	Format
`sample`	Set the number of files and file items to download.	integer
`path`	Path components to append to each URL, if filters are implemented within the path. See also: Filter the data.	string
`qs:*`	Query string parameters to append to each URL, for example: `qs:parameter1=value1`. See also: Filter the data.	string

Note

If a source publishes release packages or record packages, the sample argument refers to the number of packages to download.

However, if a source publishes files that either:

Contain multiple release packages or record packages
Contain multiple releases or records that are not within a package

Each file will be split into multiple file items. In this case, the sample argument refers to the number of file items (packages, releases or records) to download.

Some spiders support these arguments:

Name	Description	Format
`from_date`	Download data from this date onward. The spider might describe specific semantics (e.g. the relevant date is the date the contract was signed). Otherwise, the relevant date is the release date.	See spider
`until_date`	Download data until this date. The spider might describe specific semantics (e.g. the relevant date is the date the contract was signed). Otherwise, the relevant date is the release date.	See spider
`portal`	Download data from this portal.	See spider
`publisher`	Download data by this publisher.	See spider
`system`	Download data from this system.	See spider

Available data sources by country#

Argentina#

class kingfisher_scrapy.spiders.argentina_buenos_aires.ArgentinaBuenosAires(*args, **kwargs)[source]#

Domain: Ciudad de Buenos Aires
API documentation: https://data.buenosaires.gob.ar/acerca/ckan
Bulk download documentation: https://data.buenosaires.gob.ar/dataset/buenos-aires-compras/archivo/2a3d077c-71b6-4ba7-8924-f3e38cf1b8fc

scrapy crawl argentina_buenos_aires

class kingfisher_scrapy.spiders.argentina_vialidad.ArgentinaVialidad(*args, **kwargs)[source]#

Domain: Vialidad Nacional del Ministerio de Obras Públicas
API documentation: https://datosabiertos.vialidad.gob.ar/ui/index.html#!/datos_abiertos

scrapy crawl argentina_vialidad

Armenia#

class kingfisher_scrapy.spiders.armenia.Armenia(*args, **kwargs)[source]#

Domain: Armenian e-Procurement System (ARMEPS)
Caveats: The API paginates results using an offset query string parameter, which is a timestamp. If a timestamp causes an error, the spider will try to find the nearest timestamp within the following 1024 days that succeeds.

scrapy crawl armenia

Australia#

class kingfisher_scrapy.spiders.australia.Australia(*args, **kwargs)[source]#

Domain

AusTender

Spider arguments

from_date: Download only data from this time onward (YYYY-MM-DDThh:mm:ss format). Defaults to ‘2004-01-01T00:00:00’.
until_date: Download only data until this time (YYYY-MM-DDThh:mm:ss format). Defaults to now.

API documentation

https://github.com/austender/austender-ocds-api

Swagger API documentation

https://app.swaggerhub.com/apis/austender/ocds-api/1.1

scrapy crawl australia

class kingfisher_scrapy.spiders.australia_new_south_wales.AustraliaNewSouthWales(*args, **kwargs)[source]#

Domain: New South Wales (NSW)
API documentation: https://github.com/NSW-eTendering/NSW-eTendering-API/blob/master/README.md

scrapy crawl australia_new_south_wales

Austria#

class kingfisher_scrapy.spiders.austria_digiwhist.AustriaDigiwhist(*args, **kwargs)[source]#

scrapy crawl austria_digiwhist

Belgium#

class kingfisher_scrapy.spiders.belgium_digiwhist.BelgiumDigiwhist(*args, **kwargs)[source]#

scrapy crawl belgium_digiwhist

Bolivia#

class kingfisher_scrapy.spiders.bolivia_agetic.BoliviaAgetic(*args, **kwargs)[source]#

Domain: Agencia de Gobierno Electrónico y Tecnologías de Información y Comunicación (AGETIC)
Bulk download documentation: https://datos.gob.bo/id/dataset/contrataciones-agetic-2019-estandar-ocp

scrapy crawl bolivia_agetic

Bulgaria#

class kingfisher_scrapy.spiders.bulgaria_digiwhist.BulgariaDigiwhist(*args, **kwargs)[source]#

scrapy crawl bulgaria_digiwhist

Canada#

class kingfisher_scrapy.spiders.canada_buyandsell.CanadaBuyandsell(*args, **kwargs)[source]#

Domain: Public Works and Government Services Canada
Caveats: The dataset is a pilot.
API documentation: https://buyandsell.gc.ca/procurement-data/open-contracting-data-standard-pilot/download-ocds-pilot-data

scrapy crawl canada_buyandsell

class kingfisher_scrapy.spiders.canada_montreal.CanadaMontreal(*args, **kwargs)[source]#

Domain: Ville de Montréal (City of Montreal)
API documentation: http://donnees.ville.montreal.qc.ca/dataset/contrats-et-subventions-api

scrapy crawl canada_montreal

class kingfisher_scrapy.spiders.canada_quebec.CanadaQuebec(*args, **kwargs)[source]#

Domain: Secrétariat du Conseil du trésor
Bulk download documentation: https://www.donneesquebec.ca/recherche/dataset/systeme-electronique-dappel-doffres-seao

scrapy crawl canada_quebec

Chile#

class kingfisher_scrapy.spiders.chile_compra_api_records.ChileCompraAPIRecords(*args, **kwargs)[source]#

Domain

ChileCompra

Spider arguments

from_date

Download only data from this month onward (YYYY-MM format). Defaults to ‘2009-01’.

until_date

Download only data until this month (YYYY-MM format). Defaults to the current month.

system

Filter by system, if not set gets all:

convenio: Framework agreements only
trato-directo: Direct tenders only
licitacion: Traditional tenders

Swagger API documentation

https://desarrolladores.mercadopublico.cl/docs/services/5a9ed02f718ed712f4403e75/operations/data-listaa-omes-agno-mes

scrapy crawl chile_compra_api_records

class kingfisher_scrapy.spiders.chile_compra_api_releases.ChileCompraAPIReleases(*args, **kwargs)[source]#

Domain

ChileCompra

Spider arguments

from_date

Download only data from this month onward (YYYY-MM format). Defaults to ‘2009-01’.

until_date

Download only data until this month (YYYY-MM format). Defaults to the current month.

system

Filter by system, if not set gets all:

convenio: Framework agreements only
trato-directo: Direct tenders only
licitacion: Traditional tenders

Swagger API documentation

https://desarrolladores.mercadopublico.cl/docs/services/5a9ed02f718ed712f4403e75/operations/data-listaa-omes-agno-mes

scrapy crawl chile_compra_api_releases

class kingfisher_scrapy.spiders.chile_compra_bulk.ChileCompraBulk(*args, **kwargs)[source]#

Domain

ChileCompra

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). Defaults to ‘2009-01’.
until_date: Download only data until this month (YYYY-MM format). Defaults to the current month.

Bulk download documentation

https://desarrolladores.mercadopublico.cl/OCDS/DescargaMasiva

scrapy crawl chile_compra_bulk

Colombia#

class kingfisher_scrapy.spiders.colombia_ani_records.ColombiaANIRecords(*args, **kwargs)[source]#

Domain

Agencia Nacional de Infraestructura (ANI)

Bulk download documentation

https://aniscopio.ani.gov.co/datos-abiertos

API endpoints

Get all records

Link: https://apicost.azurewebsites.net/cost/records

scrapy crawl colombia_ani_records

class kingfisher_scrapy.spiders.colombia_api.ColombiaAPI(*args, **kwargs)[source]#

Domain

Colombia Compra Eficiente (CCE)

Spider arguments

from_date: Download only data from this date onward (YYYY-MM-DD format). Defaults to ‘2011-01-01’.
until_date: Download only data until this date (YYYY-MM-DD format). Defaults to today.

API documentation

https://www.colombiacompra.gov.co/transparencia/api

Swagger API documentation

https://apiocds.colombiacompra.gov.co/apiCCE2.0/

scrapy crawl colombia_api

class kingfisher_scrapy.spiders.colombia_bulk.ColombiaBulk(*args, **kwargs)[source]#

Domain

Colombia Compra Eficiente (CCE)

Spider arguments

from_date

Download only data from this year onward (YYYY format). Defaults to ‘2011’. Only supported for ‘SECOP1’ system.

until_date

Download only data until this year (YYYY format). Defaults to the current year. Only supported for ‘SECOP1’ system.

system

Filter by system:

SECOP1
Sistema Electrónico para la Contratación Pública (todos los procesos de contratación)

SECOP2
Sistema Electrónico para la Contratación Pública (todos los procesos de contratación)

TVEC
Tienda Virtual del Estado Colombiano (acuerdos marco, agregación de demanda, mínima cuantía)

Bulk download documentation

https://www.colombiacompra.gov.co/transparencia/datos-json

scrapy crawl colombia_bulk

Costa Rica#

class kingfisher_scrapy.spiders.costa_rica_poder_judicial_records.CostaRicaPoderJudicialRecords(*args, **kwargs)[source]#

Domain

Poder Judicial de Costa Rica

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2018’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

Bulk download documentation

http://datosabiertospj.eastus.cloudapp.azure.com/dataset/estandar-de-datos-de-contrataciones-abiertas-ocds

scrapy crawl costa_rica_poder_judicial_records

class kingfisher_scrapy.spiders.costa_rica_poder_judicial_releases.CostaRicaPoderJudicialReleases(*args, **kwargs)[source]#

Domain: Poder Judicial de Costa Rica
Bulk download documentation: http://datosabiertospj.eastus.cloudapp.azure.com/dataset/estandar-de-datos-de-contrataciones-abiertas-ocds

scrapy crawl costa_rica_poder_judicial_releases

Croatia#

class kingfisher_scrapy.spiders.croatia.Croatia(*args, **kwargs)[source]#

Domain: Elektronički oglasnik javne nabave (Electronic public procurement notices)
Bulk download documentation: https://eojn.nn.hr/SPIN/application/ipn/Oglasnik/PreuzimanjeUgovoraOCD.aspx

scrapy crawl croatia

class kingfisher_scrapy.spiders.croatia_digiwhist.CroatiaDigiwhist(*args, **kwargs)[source]#

scrapy crawl croatia_digiwhist

Cyprus#

class kingfisher_scrapy.spiders.cyprus_digiwhist.CyprusDigiwhist(*args, **kwargs)[source]#

scrapy crawl cyprus_digiwhist

Czech Republic#

class kingfisher_scrapy.spiders.czech_republic_digiwhist.CzechRepublicDigiwhist(*args, **kwargs)[source]#

scrapy crawl czech_republic_digiwhist

Denmark#

class kingfisher_scrapy.spiders.denmark_digiwhist.DenmarkDigiwhist(*args, **kwargs)[source]#

scrapy crawl denmark_digiwhist

Digiwhist#

Dominican Republic#

class kingfisher_scrapy.spiders.dominican_republic_api.DominicanRepublicAPI(*args, **kwargs)[source]#

Domain

Dirección General de Contrataciones Públicas (DGCP)

Spider arguments

from_date: Download only data from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2018-01-01’.
until_date: Download only data until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.

API documentation

http://api.dgcp.gob.do/ocdsdr/docs

scrapy crawl dominican_republic_api

class kingfisher_scrapy.spiders.dominican_republic_bulk.DominicanRepublicBulk(*args, **kwargs)[source]#

Domain

Dirección General de Contrataciones Públicas (DGCP)

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2018’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

Bulk download documentation

https://www.dgcp.gob.do/estandar-mundial-ocds/

scrapy crawl dominican_republic_bulk

Ecuador#

class kingfisher_scrapy.spiders.ecuador_emergency.EcuadorEmergency(*args, **kwargs)[source]#

Domain: Servicio Nacional de Contratación Pública
Bulk download documentation: https://portal.compraspublicas.gob.ec/sercop/data-estandar-ocds/

scrapy crawl ecuador_emergency

class kingfisher_scrapy.spiders.ecuador_sercop.EcuadorSERCOP(*args, **kwargs)[source]#

Domain

Servicio Nacional de Contratación Pública (SERCOP)

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). Defaults to ‘2015-01’.
until_date: Download only data until this month (YYYY-MM format). Defaults to the current month.

API documentation

https://datosabiertos.compraspublicas.gob.ec/PLATAFORMA/datos-abiertos/api

Bulk download documentation

https://datosabiertos.compraspublicas.gob.ec/PLATAFORMA/datos-abiertos

scrapy crawl ecuador_sercop

Estonia#

class kingfisher_scrapy.spiders.estonia_digiwhist.EstoniaDigiwhist(*args, **kwargs)[source]#

scrapy crawl estonia_digiwhist

Europe#

class kingfisher_scrapy.spiders.europe_ted_digiwhist.EuropeTEDDigiwhist(*args, **kwargs)[source]#

scrapy crawl europe_ted_digiwhist

European#

Finland#

class kingfisher_scrapy.spiders.finland_digiwhist.FinlandDigiwhist(*args, **kwargs)[source]#

scrapy crawl finland_digiwhist

France#

class kingfisher_scrapy.spiders.france.France(*args, **kwargs)[source]#

Domain: France
Swagger API documentation: https://doc.data.gouv.fr/api/reference/

scrapy crawl france

class kingfisher_scrapy.spiders.france_digiwhist.FranceDigiwhist(*args, **kwargs)[source]#

scrapy crawl france_digiwhist

Georgia#

class kingfisher_scrapy.spiders.georgia_digiwhist.GeorgiaDigiwhist(*args, **kwargs)[source]#

scrapy crawl georgia_digiwhist

class kingfisher_scrapy.spiders.georgia_opendata.GeorgiaOpendata(*args, **kwargs)[source]#

Domain: State Procurement Agency (SPA)

scrapy crawl georgia_opendata

class kingfisher_scrapy.spiders.georgia_records.GeorgiaRecords(*args, **kwargs)[source]#

Domain: State Procurement Agency (SPA)
Bulk download documentation: https://odapi.spa.ge/publication-policy
Swagger API documentation: https://odapi.spa.ge/api/swagger.ui

scrapy crawl georgia_records

class kingfisher_scrapy.spiders.georgia_releases.GeorgiaReleases(*args, **kwargs)[source]#

Domain: State Procurement Agency (SPA)
Bulk download documentation: https://odapi.spa.ge/publication-policy
Swagger API documentation: https://odapi.spa.ge/api/swagger.ui

scrapy crawl georgia_releases

Germany#

class kingfisher_scrapy.spiders.germany_digiwhist.GermanyDigiwhist(*args, **kwargs)[source]#

scrapy crawl germany_digiwhist

Ghana#

class kingfisher_scrapy.spiders.ghana.Ghana(*args, **kwargs)[source]#

Domain

Ghana Electronic Procurement System (GHANEPS)

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). If until_date is provided, defaults to ‘2019-07’.
until_date: Download only data until this month (YYYY-MM format). If from_date is provided, defaults to the current month.

scrapy crawl ghana

Greece#

class kingfisher_scrapy.spiders.greece_digiwhist.GreeceDigiwhist(*args, **kwargs)[source]#

scrapy crawl greece_digiwhist

Honduras#

class kingfisher_scrapy.spiders.honduras_cost.HondurasCoST(*args, **kwargs)[source]#

Domain: Systema de Información y Seguimiento de Obras y Contratos (SISOCS)

scrapy crawl honduras_cost

class kingfisher_scrapy.spiders.honduras_iaip.HondurasIAIP(*args, **kwargs)[source]#

Domain

Instituto de Acceso a la Información Publica (IAIP)

Spider arguments

portal

Filter by portal:

covid19
IAIP Emergencia Covid-19

huracanes
Emergencia Huracán ETA

oficio
Portal Único de Transparencia

Bulk download documentation

https://portalunico.iaip.gob.hn/datosabierto/

scrapy crawl honduras_iaip

class kingfisher_scrapy.spiders.honduras_oncae.HondurasONCAE(*args, **kwargs)[source]#

Domain

Oficina Normativa de Contratación y Adquisiciones del Estado (ONCAE)

Spider arguments

from_date

Download only data from this year onward (YYYY format). Defaults to ‘2005’.

until_date

Download only data until this year (YYYY format). Defaults to the current year.

system

Filter by system:

CE: Catálogo Electrónico
DDC: Módulo de Difusión Directa de Contratos
HC1: HonduCompras 1.0 (Módulo de Difusión de Compras y Contrataciones)

Bulk download documentation

http://oncae.gob.hn/datosabiertos

scrapy crawl honduras_oncae

class kingfisher_scrapy.spiders.honduras_portal_api_records.HondurasPortalAPIRecords(*args, **kwargs)[source]#

Domain

Oficina Normativa de Contratación y Adquisiciones del Estado (ONCAE) / Secretaria de Finanzas de Honduras (SEFIN)

Spider arguments

publisher

Filter by publisher:

oncae: Oficina Normativa de Contratación y Adquisiciones del Estado
sefin: Secretaria de Finanzas de Honduras

API documentation

http://www.contratacionesabiertas.gob.hn/manual_api/

Swagger API documentation

http://www.contratacionesabiertas.gob.hn/servicio/

scrapy crawl honduras_portal_api_records

class kingfisher_scrapy.spiders.honduras_portal_api_releases.HondurasPortalAPIReleases(*args, **kwargs)[source]#

Domain

Oficina Normativa de Contratación y Adquisiciones del Estado (ONCAE) / Secretaria de Finanzas de Honduras (SEFIN)

Spider arguments

publisher

Filter by publisher:

oncae: Oficina Normativa de Contratación y Adquisiciones del Estado
sefin: Secretaria de Finanzas de Honduras

API documentation

http://www.contratacionesabiertas.gob.hn/manual_api/

Swagger API documentation

http://www.contratacionesabiertas.gob.hn/servicio/

scrapy crawl honduras_portal_api_releases

class kingfisher_scrapy.spiders.honduras_portal_bulk.HondurasPortalBulk(*args, **kwargs)[source]#

Domain

Oficina Normativa de Contratación y Adquisiciones del Estado (ONCAE) / Secretaria de Finanzas de Honduras (SEFIN)

Spider arguments

from_date

Download only data from this month onward (YYYY-MM format). If until_date is provided, defaults to ‘2005-11’.

until_date

Download only data until this month (YYYY-MM format). If from_date is provided, defaults to the current month.

publisher

Filter by publisher:

oncae: Oficina Normativa de Contratación y Adquisiciones del Estado
sefin: Secretaria de Finanzas de Honduras

system

Filter by oncae system:

CE: Catálogo Electrónico
DDC: Módulo de Difusión Directa de Contratos
HC1: HonduCompras 1.0 (Módulo de Difusión de Compras y Contrataciones)

Bulk download documentation

http://www.contratacionesabiertas.gob.hn/descargas/

scrapy crawl honduras_portal_bulk

Hungary#

class kingfisher_scrapy.spiders.hungary_digiwhist.HungaryDigiwhist(*args, **kwargs)[source]#

scrapy crawl hungary_digiwhist

Iceland#

class kingfisher_scrapy.spiders.iceland_digiwhist.IcelandDigiwhist(*args, **kwargs)[source]#

scrapy crawl iceland_digiwhist

India#

class kingfisher_scrapy.spiders.india_civic_data_lab.IndiaCivicDataLab(*args, **kwargs)[source]#

Domain: CivicDataLab
Bulk download documentation: https://github.com/CivicDataLab/himachal-pradesh-health-procurement-OCDS/

scrapy crawl india_civic_data_lab

Indonesia#

class kingfisher_scrapy.spiders.indonesia_opentender.IndonesiaOpentender(*args, **kwargs)[source]#

Domain

Opentender.net

Spider arguments

from_date: Download only data from this year onward (YYYY format). Defaults to ‘2008’.
until_date: Download only data until this year (YYYY format). Defaults to the current year.

Bulk download documentation

https://v3.opentender.net/#/ocds

scrapy crawl indonesia_opentender

Ireland#

class kingfisher_scrapy.spiders.ireland_digiwhist.IrelandDigiwhist(*args, **kwargs)[source]#

scrapy crawl ireland_digiwhist

Italy#

class kingfisher_scrapy.spiders.italy_anac.ItalyANAC(*args, **kwargs)[source]#

Domain: Autorità Nazionale Anticorruzione (ANAC)
API documentation: https://dati.anticorruzione.it/opendata/about
Bulk download documentation: https://dati.anticorruzione.it/opendata/organization/anticorruzione

scrapy crawl italy_anac

class kingfisher_scrapy.spiders.italy_appalti_pop.ItalyAppaltiPOP(*args, **kwargs)[source]#

Domain: AppaltiPOP
Bulk download documentation: https://www.appaltipop.it/it/download
Swagger API documentation: https://www.appaltipop.it/api/v1/

scrapy crawl italy_appalti_pop

class kingfisher_scrapy.spiders.italy_digiwhist.ItalyDigiwhist(*args, **kwargs)[source]#

scrapy crawl italy_digiwhist

Kenya#

class kingfisher_scrapy.spiders.kenya_makueni.KenyaMakueni(*args, **kwargs)[source]#

Domain: Makueni County
Swagger API documentation: https://opencontracting.makueni.go.ke/swagger-ui/#/ocds-controller

scrapy crawl kenya_makueni

Kosovo#

class kingfisher_scrapy.spiders.kosovo.Kosovo(*args, **kwargs)[source]#

Domain

Public Procurement Regulatory Commission

Spider arguments

from_date: Download only data with a tender period end date from this time onward (YYYY-MM-DD format). Defaults to ‘2000-01-01’.
until_date: Download only data with a tender period end date until this time (YYYY-MM-DD format). Defaults to now. Note: This date is non-inclusive.

API documentation

https://ocdskrpp.rks-gov.net/Help

scrapy crawl kosovo

Kyrgyzstan#

class kingfisher_scrapy.spiders.kyrgyzstan.Kyrgyzstan(*args, **kwargs)[source]#

Domain

Ministry of Finance

Caveats

The planning endpoint is not implemented because is not completely in OCDS.

Spider arguments

from_date: Download only data from this time onward (YYYY-MM-DDThh:mm:ss format).

Bulk download documentation

http://ocds.zakupki.gov.kg/dashboard/weekly and then to ‘API & EXPORT’. Direct access to http://ocds.zakupki.gov.kg/api-export doesn’t work.

Swagger API documentation

https://app.swaggerhub.com/apis/DPAteam/export-api-documentation/1.0

scrapy crawl kyrgyzstan

Latvia#

class kingfisher_scrapy.spiders.latvia_digiwhist.LatviaDigiwhist(*args, **kwargs)[source]#

scrapy crawl latvia_digiwhist

Lithuania#

class kingfisher_scrapy.spiders.lithuania_digiwhist.LithuaniaDigiwhist(*args, **kwargs)[source]#

scrapy crawl lithuania_digiwhist

Luxembourg#

class kingfisher_scrapy.spiders.luxembourg_digiwhist.LuxembourgDigiwhist(*args, **kwargs)[source]#

scrapy crawl luxembourg_digiwhist

Malta#

class kingfisher_scrapy.spiders.malta.Malta(*args, **kwargs)[source]#

Domain

Malta

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). If until_date is provided, defaults to ‘2019-10’.
until_date: Download only data until this month (YYYY-MM format). If from_date is provided, defaults to the current month.

API documentation

https://docs.google.com/document/d/1VnCEywKkkQ7BcVbT7HlW2s_N_QI8W0KE/edit

scrapy crawl malta

class kingfisher_scrapy.spiders.malta_digiwhist.MaltaDigiwhist(*args, **kwargs)[source]#

scrapy crawl malta_digiwhist

Mexico#

class kingfisher_scrapy.spiders.mexico_administracion_publica_federal_api.MexicoAdministracionPublicaFederalAPI(*args, **kwargs)[source]#

Domain: Administración Pública Federal (APF): Secretaria de la Función Pública (SFP) - Secretaría de Hacienda y Crédito Público (SHCP)
API documentation: https://www.datos.gob.mx/busca/dataset/api-de-contrataciones-abiertas-de-la-apf

scrapy crawl mexico_administracion_publica_federal_api

class kingfisher_scrapy.spiders.mexico_administracion_publica_federal_bulk.MexicoAdministracionPublicaFederalBulk(*args, **kwargs)[source]#

Domain: Administración Pública Federal (APF): Secretaría de Hacienda y Crédito Público (SHCP)
Bulk download documentation: https://datos.gob.mx/busca/dataset/concentrado-de-contrataciones-abiertas-de-la-apf

scrapy crawl mexico_administracion_publica_federal_bulk

class kingfisher_scrapy.spiders.mexico_durango_idaip.MexicoDurangoIDAIP(*args, **kwargs)[source]#

Domain

Instituto Duranguense de Acceso a la Información y de Protección de Datos Personales (IDAIP)

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2020’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

API documentation

http://contratacionesabiertas.idaip.org.mx/contratacionesabiertas/datosabiertos

scrapy crawl mexico_durango_idaip

class kingfisher_scrapy.spiders.mexico_grupo_aeroporto.MexicoGrupoAeroporto(*args, **kwargs)[source]#

Domain: Grupo Aeroportuario de la Ciudad de México (CDMX)

scrapy crawl mexico_grupo_aeroporto

class kingfisher_scrapy.spiders.mexico_guanajuato_iacip.MexicoGuanajuatoIACIP(*args, **kwargs)[source]#

Domain

Instituto de Acceso a la Información Pública para el Estado de Guanajuato (IACIP)

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2021’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

API documentation

http://contratacionesabiertas.iacipgto.mx:4000/contratacionesabiertas/datosabiertos

scrapy crawl mexico_guanajuato_iacip

class kingfisher_scrapy.spiders.mexico_inai_api.MexicoINAIAPI(*args, **kwargs)[source]#

Domain

Instituto Nacional de Transparencia, Acceso a la Información y Protección de Datos Personales (INAI)

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2015’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

API documentation

http://contratacionesabiertas.inai.org.mx/contratacionesabiertas/datosabiertos

scrapy crawl mexico_inai_api

class kingfisher_scrapy.spiders.mexico_mexico_city_infocdmx.MexicoMexicoCityINFOCDMX(*args, **kwargs)[source]#

Domain

Instituto de Transparencia, Acceso a la Información Pública, Protección de Datos Personales y Rendición de Cuentas de la Ciudad de México (INFOCDMX)

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2019’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

API documentation

https://dashboard.infocdmx.org.mx/contratacionesabiertas/datosabiertos

scrapy crawl mexico_mexico_city_infocdmx

class kingfisher_scrapy.spiders.mexico_mexico_state_infoem.MexicoMexicoStateINFOEM(*args, **kwargs)[source]#

Domain

Instituto de Transparencia, Acceso a la Información Pública y Protección de Datos Personales del Estado de México y Municipios (INFOEM)

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2021’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

API documentation

http://www.infoem.org.mx:4000/contratacionesabiertas/datosabiertos

scrapy crawl mexico_mexico_state_infoem

class kingfisher_scrapy.spiders.mexico_nuevo_leon_cotai.MexicoNuevoLeonCOTAI(*args, **kwargs)[source]#

Domain

Comisión de Transparencia y Acceso a la Información del Estado de Nuevo León (COTAI)

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2020’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

API documentation

http://201.149.38.218:4000/contratacionesabiertas/datosabiertos

scrapy crawl mexico_nuevo_leon_cotai

class kingfisher_scrapy.spiders.mexico_nuevo_leon_records.MexicoNuevoLeonRecords(*args, **kwargs)[source]#

Domain: Secretaría de Movilidad y Planeación Urbana de Nuevo León
Bulk download documentation: http://si.nl.gob.mx/transparencia/publicaciones

scrapy crawl mexico_nuevo_leon_records

class kingfisher_scrapy.spiders.mexico_nuevo_leon_releases.MexicoNuevoLeonReleases(*args, **kwargs)[source]#

Domain: Secretaría de Movilidad y Planeación Urbana de Nuevo León
Bulk download documentation: http://si.nl.gob.mx/transparencia/acerca-del-proyecto

scrapy crawl mexico_nuevo_leon_releases

class kingfisher_scrapy.spiders.mexico_oaxaca_iaip.MexicoOaxacaIAIP(*args, **kwargs)[source]#

Domain

Instituto de Acceso a la Información Publica y Protección de Datos Personales del Estado de Oaxaca (IAIPOXACA)

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2021’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

API documentation

http://contratacionesabiertas-iaipoaxaca-org.mx:4000/contratacionesabiertas/datosabiertos

scrapy crawl mexico_oaxaca_iaip

class kingfisher_scrapy.spiders.mexico_plataforma_digital_nacional.MexicoPlataformaDigitalNacional(*args, **kwargs)[source]#

Domain: Plataforma Digital Nacional
Bulk download documentation: https://plataformadigitalnacional.org/contrataciones

scrapy crawl mexico_plataforma_digital_nacional

class kingfisher_scrapy.spiders.mexico_puebla_itaipue.MexicoPueblaITAIPUE(*args, **kwargs)[source]#

Domain

Instituto de Transparencia, Acceso a la Información Pública y Protección de Datos Personales del Estado de Puebla

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2021’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

API documentation

http://189.240.12.27:4000/contratacionesabiertas/datosabiertos/

scrapy crawl mexico_puebla_itaipue

class kingfisher_scrapy.spiders.mexico_quien_es_quien_records.MexicoQuienEsQuienRecords(*args, **kwargs)[source]#

Domain: QuiénEsQuién.Wiki
Caveats: The ‘record’ endpoint returns a ‘data’ array with the first entry as a record package and the subsequent ones as records. This spider transform the output to package all the records within a package using the first package metadata. The ‘uri’ and ‘publicationDate’ record package metadata fields are removed.
API documentation: https://qqwapi-elastic.readthedocs.io/es/latest/
Swagger API documentation: https://api.quienesquien.wiki/v3/docs/

scrapy crawl mexico_quien_es_quien_records

class kingfisher_scrapy.spiders.mexico_quien_es_quien_releases.MexicoQuienEsQuienReleases(*args, **kwargs)[source]#

Domain: QuiénEsQuién.Wiki
API documentation: https://qqwapi-elastic.readthedocs.io/es/latest/
Swagger API documentation: https://api.quienesquien.wiki/v3/docs/

scrapy crawl mexico_quien_es_quien_releases

class kingfisher_scrapy.spiders.mexico_quintana_roo_idaip.MexicoQuintanaRooIDAIP(*args, **kwargs)[source]#

Domain

Instituto de Acceso a la Información y Protección de Datos Personales de Quintana Roo (IDAIPQROO)

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2020’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

API documentation

http://conab.idaipqroo.org.mx:4000/contratacionesabiertas/datosabiertos

scrapy crawl mexico_quintana_roo_idaip

class kingfisher_scrapy.spiders.mexico_sinaloa_ceaip.MexicoSinaloaCEAIP(*args, **kwargs)[source]#

Domain

Comisión Estatal para el Acceso a la Información Pública (CEAIP)

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2021’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

API documentation

http://ceaipsinaloa.ddns.net:4000/contratacionesabiertas/datosabiertos

scrapy crawl mexico_sinaloa_ceaip

class kingfisher_scrapy.spiders.mexico_veracruz_ivai.MexicoVeracruzIVAI(*args, **kwargs)[source]#

Domain

Instituto Veracruzano de Acceso a la Información y Protección de Datos Personales

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2020’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

API documentation

http://www.ivai.org.mx/contrataciones-abiertas/

scrapy crawl mexico_veracruz_ivai

class kingfisher_scrapy.spiders.mexico_yucatan_inaip.MexicoYucatanINAIP(*args, **kwargs)[source]#

Domain

Instituto Estatal de Transparencia, Acceso a la Información Pública y Protección de Datos Personales (INAIP) - Yucatán

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2020’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

API documentation

https://contratacionesabiertas.inaipyucatan.org.mx/contratacionesabiertas/datosabiertos#

scrapy crawl mexico_yucatan_inaip

class kingfisher_scrapy.spiders.mexico_zacatecas_izai.MexicoZacatecasIZAI(*args, **kwargs)[source]#

Domain

Instituto Zacatecano de Transparencia y Acceso a la Información (IZAI)

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2016’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

API documentation

https://contratacionesabiertas.izai.org.mx/contratacionesabiertas/datosabiertos

scrapy crawl mexico_zacatecas_izai

Moldova#

class kingfisher_scrapy.spiders.moldova.Moldova(*args, **kwargs)[source]#

Domain

MTender

Spider arguments

from_date: Download only data from this time onward (YYYY-MM-DDThh:mm:ss format).

scrapy crawl moldova

class kingfisher_scrapy.spiders.moldova_old.MoldovaOld(*args, **kwargs)[source]#

Domain

Public Procurement Agency (PPA)

Spider arguments

from_date: Download only data from this year onward (YYYY format). Defaults to ‘2012’.
until_date: Download only data until this year (YYYY format). Defaults to ‘2017’.

Bulk download documentation

http://opencontracting.date.gov.md/downloads

scrapy crawl moldova_old

Nepal#

class kingfisher_scrapy.spiders.nepal.Nepal(*args, **kwargs)[source]#

Domain

Public Procurement Monitoring Office (PPMO)

Spider arguments

from_date: Download only data from this year onward (YYYY format). Defaults to ‘2012’.
until_date: Download only data until this year (YYYY format). Defaults to ‘2018’.

Bulk download documentation

http://ppip.gov.np/downloads

scrapy crawl nepal

class kingfisher_scrapy.spiders.nepal_dhangadhi.NepalDhangadhi(*args, **kwargs)[source]#

Domain: Dhangadhi Infrastructure Management System (IMS)
Caveats: Some URLs listed in https://admin.ims.susasan.org/api/static-data/dhangadhi require login and cannot be downloaded
Bulk download documentation: https://ims.susasan.org/dhangadhi/about

scrapy crawl nepal_dhangadhi

Netherlands#

class kingfisher_scrapy.spiders.netherlands_digiwhist.NetherlandsDigiwhist(*args, **kwargs)[source]#

scrapy crawl netherlands_digiwhist

Nicaragua#

class kingfisher_scrapy.spiders.nicaragua_solid_waste.NicaraguaSolidWaste(*args, **kwargs)[source]#

Domain

Solid Waste Mitigation Platform (SWMP)

Spider arguments

from_date: Download only data from this date onward (YYYY-MM-DD format). Defaults to ‘2000-01-01’.
until_date: Download only data until this date (YYYY-MM-DD format). Defaults to today.

scrapy crawl nicaragua_solid_waste

Nigeria#

class kingfisher_scrapy.spiders.nigeria_abia_state.NigeriaAbiaState(*args, **kwargs)[source]#

Domain

Abia E-PROCUREMENT

Spider arguments

from_date: Download only data from this year onward (YYYY format). Defaults to ‘2018’.
until_date: Download only data until this year (YYYY format). Defaults to ‘2021’.

Bulk download documentation

https://abiaeprocurement.ab.gov.ng/ocds_report.php

scrapy crawl nigeria_abia_state

class kingfisher_scrapy.spiders.nigeria_anambra_state_records.NigeriaAnambraStateRecords(*args, **kwargs)[source]#

Domain: Anambra State
API documentation: http://www.mepbdp.org.ng/ocds/api

scrapy crawl nigeria_anambra_state_records

class kingfisher_scrapy.spiders.nigeria_budeshi_records.NigeriaBudeshiRecords(*args, **kwargs)[source]#

Domain: Budeshi Nigeria
API documentation: https://budeshi.ng/Api

scrapy crawl nigeria_budeshi_records

class kingfisher_scrapy.spiders.nigeria_budeshi_releases.NigeriaBudeshiReleases(*args, **kwargs)[source]#

Domain: Budeshi Nigeria
API documentation: https://budeshi.ng/Api

scrapy crawl nigeria_budeshi_releases

class kingfisher_scrapy.spiders.nigeria_cross_river_state_records.NigeriaCrossRiverStateRecords(*args, **kwargs)[source]#

Domain

Cross River State

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). If until_date is provided, defaults to ‘2019-08’.
until_date: Download only data until this month (YYYY-MM format). If from_date is provided, defaults to the current month.

API documentation

http://ocdsapi.dppib-crsgov.org/Help

scrapy crawl nigeria_cross_river_state_records

class kingfisher_scrapy.spiders.nigeria_cross_river_state_releases.NigeriaCrossRiverStateReleases(*args, **kwargs)[source]#

Domain

Cross River State

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). If until_date is provided, defaults to ‘2019-08’.
until_date: Download only data until this month (YYYY-MM format). If from_date is provided, defaults to the current month.

API documentation

http://ocdsapi.dppib-crsgov.org/Help

scrapy crawl nigeria_cross_river_state_releases

class kingfisher_scrapy.spiders.nigeria_ebonyi_state.NigeriaEbonyiState(*args, **kwargs)[source]#

Domain

Ebonyi E-PROCUREMENT

Caveats

The JSON data sometimes contains unescaped tab characters within strings.

Spider arguments

from_date: Download only data from this year onward (YYYY format). Defaults to ‘2018’.
until_date: Download only data until this year (YYYY format). Defaults to the current year.

Bulk download documentation

https://ebonyieprocure.eb.gov.ng/ocds_report.php

scrapy crawl nigeria_ebonyi_state

class kingfisher_scrapy.spiders.nigeria_edo_state.NigeriaEdoState(*args, **kwargs)[source]#

Domain: Edo State Open Contracting Data Standards Portal
Caveats: The release list has additional numbered keys with tender, award and contract releases as values. This spider removes the numbered keys and append the releases into the release list.
Bulk download documentation: http://edpms.edostate.gov.ng/ocds/

scrapy crawl nigeria_edo_state

class kingfisher_scrapy.spiders.nigeria_ekiti_state.NigeriaEkitiState(*args, **kwargs)[source]#

Domain: Nigeria Ekiti State Open Contracting Portal

scrapy crawl nigeria_ekiti_state

class kingfisher_scrapy.spiders.nigeria_enugu_state.NigeriaEnuguState(*args, **kwargs)[source]#

Domain: Nigeria Enugu State Open Contracting Portal
Bulk download documentation: https://dueprocess.en.gov.ng/ocds_report.php

scrapy crawl nigeria_enugu_state

class kingfisher_scrapy.spiders.nigeria_gombe_state.NigeriaGombeState(*args, **kwargs)[source]#

Domain: Nigeria Gombe State Open Contracting Portal

scrapy crawl nigeria_gombe_state

class kingfisher_scrapy.spiders.nigeria_kaduna_state_budeshi_records.NigeriaKadunaStateBudeshiRecords(*args, **kwargs)[source]#

Domain: Budeshi Nigeria - Kaduna State
API documentation: https://www.budeshi.ng/kadppa/api

scrapy crawl nigeria_kaduna_state_budeshi_records

class kingfisher_scrapy.spiders.nigeria_kaduna_state_budeshi_releases.NigeriaKadunaStateBudeshiReleases(*args, **kwargs)[source]#

Domain: Budeshi Nigeria - Kaduna State
API documentation: https://www.budeshi.ng/kadppa/api

scrapy crawl nigeria_kaduna_state_budeshi_releases

class kingfisher_scrapy.spiders.nigeria_osun_state.NigeriaOsunState(*args, **kwargs)[source]#

Domain: Nigeria Osun State Open Contracting Portal
Bulk download documentation: https://egp.osunstate.gov.ng/awarded_contracts.php

scrapy crawl nigeria_osun_state

class kingfisher_scrapy.spiders.nigeria_oyo_state.NigeriaOyoState(*args, **kwargs)[source]#

Domain: Nigeria Oyo State Open Contracting Portal

scrapy crawl nigeria_oyo_state

class kingfisher_scrapy.spiders.nigeria_portal.NigeriaPortal(*args, **kwargs)[source]#

Domain: Nigeria Open Contracting Portal (NOCOPO) of Bureau of Public Procurement (BPP)

scrapy crawl nigeria_portal

Norway#

class kingfisher_scrapy.spiders.norway_digiwhist.NorwayDigiwhist(*args, **kwargs)[source]#

scrapy crawl norway_digiwhist

Openopps#

class kingfisher_scrapy.spiders.openopps.Openopps(*args, **kwargs)[source]#

Domain

OpenOpps

Spider arguments

from_date: Download only data from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2011-01-01’.
until_date: Download only data until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.

Environment variables

KINGFISHER_OPENOPPS_USERNAME: To get an API account, contact contact@openopps.com.
KINGFISHER_OPENOPPS_PASSWORD: Your API account password.

API documentation

https://docs.google.com/document/d/1u0da3BTU7fBFjX6i7j_tKXa1YwdXL7hY4Kw9GdsaAr0/edit

Swagger API documentation

https://api.openopps.com/api/schema/

env KINGFISHER_OPENOPPS_USERNAME=... KINGFISHER_OPENOPPS_PASSWORD=... scrapy crawl openopps

Pakistan#

class kingfisher_scrapy.spiders.pakistan_ppra_api.PakistanPPRAAPI(*args, **kwargs)[source]#

Domain: Pakistan Public Procurement Regulatory Authority (PPRA)
API documentation: https://www.ppra.org.pk/api/

scrapy crawl pakistan_ppra_api

class kingfisher_scrapy.spiders.pakistan_ppra_bulk.PakistanPPRABulk(*args, **kwargs)[source]#

Domain: Pakistan Public Procurement Regulatory Authority (PPRA)
API documentation: https://www.ppra.org.pk/api/

scrapy crawl pakistan_ppra_bulk

Paraguay#

class kingfisher_scrapy.spiders.paraguay_dncp_records.ParaguayDNCPRecords(*args, **kwargs)[source]#

Domain

Dirección Nacional de Contrataciones Públicas (DNCP)

Spider arguments

from_date: Download only data from this time onward (YYYY-MM-DDThh:mm:ss format). Defaults to ‘2010-01-01T00:00:00’.
until_date: Download only data until this time (YYYY-MM-DDThh:mm:ss format). Defaults to now.

Environment variables

KINGFISHER_PARAGUAY_DNCP_REQUEST_TOKEN: To get an API account and request token go to https://contrataciones.gov.py/datos/adm/login.

Swagger API documentation

https://contrataciones.gov.py/datos/api/v3/doc

env KINGFISHER_PARAGUAY_DNCP_REQUEST_TOKEN=... scrapy crawl paraguay_dncp_records

class kingfisher_scrapy.spiders.paraguay_dncp_releases.ParaguayDNCPReleases(*args, **kwargs)[source]#

Domain

Dirección Nacional de Contrataciones Públicas (DNCP)

Spider arguments

from_date: Download only data from this time onward (YYYY-MM-DDThh:mm:ss format). Defaults to ‘2010-01-01T00:00:00’.
until_date: Download only data until this time (YYYY-MM-DDThh:mm:ss format). Defaults to now.

Environment variables

KINGFISHER_PARAGUAY_DNCP_REQUEST_TOKEN: To get an API account and request token go to https://contrataciones.gov.py/datos/adm/login.

Swagger API documentation

https://contrataciones.gov.py/datos/api/v3/doc

env KINGFISHER_PARAGUAY_DNCP_REQUEST_TOKEN=... scrapy crawl paraguay_dncp_releases

class kingfisher_scrapy.spiders.paraguay_hacienda.ParaguayHacienda(*args, **kwargs)[source]#

Domain

Ministerio de Hacienda

Environment variables

KINGFISHER_PARAGUAY_HACIENDA_REQUEST_TOKEN: To get an API account and request token go to https://datos.hacienda.gov.py/aplicaciones/new.
KINGFISHER_PARAGUAY_HACIENDA_CLIENT_SECRET: Your client secret generated.

Swagger API documentation

https://datos.hacienda.gov.py/odmh-api-v1/api-docs/

env KINGFISHER_PARAGUAY_HACIENDA_REQUEST_TOKEN=... KINGFISHER_PARAGUAY_HACIENDA_CLIENT_SECRET=... scrapy crawl paraguay_hacienda

Peru#

class kingfisher_scrapy.spiders.peru_compras.PeruCompras(*args, **kwargs)[source]#

Domain

Peru Compras (contracts within framework agreements)

Caveats

The JSON data sometimes contains unescaped newline characters within strings.

Spider arguments

from_date: Download only data from this date onward (YYYY-MM-DD format). Defaults to ‘2017-01-01’.
until_date: Download only data until this date (YYYY-MM-DD format). Defaults to today.

scrapy crawl peru_compras

Poland#

class kingfisher_scrapy.spiders.poland_digiwhist.PolandDigiwhist(*args, **kwargs)[source]#

scrapy crawl poland_digiwhist

Portugal#

class kingfisher_scrapy.spiders.portugal_bulk.PortugalBulk(*args, **kwargs)[source]#

Domain: Instituto dos Mercados Públicos, do Imobiliário e da Construção (IMPIC)
API documentation: https://dados.gov.pt/pt/apidoc/
Bulk download documentation: https://dados.gov.pt/pt/datasets/ocds-portal-base-www-base-gov-pt/
Swagger API documentation: https://dados.gov.pt/pt/apidoc/#/datasets

scrapy crawl portugal_bulk

class kingfisher_scrapy.spiders.portugal_digiwhist.PortugalDigiwhist(*args, **kwargs)[source]#

scrapy crawl portugal_digiwhist

class kingfisher_scrapy.spiders.portugal_records.PortugalRecords(*args, **kwargs)[source]#

Domain

Instituto dos Mercados Públicos, do Imobiliário e da Construção (IMPIC)

Spider arguments

from_date: Download only data with a contract signing date from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2010-01-01’.
until_date: Download only data with a contract signing date until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.

Swagger API documentation

http://www.base.gov.pt/swagger/index.html

scrapy crawl portugal_records

class kingfisher_scrapy.spiders.portugal_releases.PortugalReleases(*args, **kwargs)[source]#

Domain

Instituto dos Mercados Públicos, do Imobiliário e da Construção (IMPIC)

Spider arguments

from_date: Download only data with a contract signing date from this date onward (YYYY-MM-DD format). If until_date is provided, defaults to ‘2010-01-01’.
until_date: Download only data with a contract signing date until this date (YYYY-MM-DD format). If from_date is provided, defaults to today.

Swagger API documentation

http://www.base.gov.pt/swagger/index.html

scrapy crawl portugal_releases

Romania#

class kingfisher_scrapy.spiders.romania_digiwhist.RomaniaDigiwhist(*args, **kwargs)[source]#

scrapy crawl romania_digiwhist

Scotland#

class kingfisher_scrapy.spiders.scotland_public_contracts.ScotlandPublicContracts(*args, **kwargs)[source]#

Domain

Public Contracts Scotland

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). Defaults to a year ago.
until_date: Download only data until this month (YYYY-MM format). Defaults to the current month.

API documentation

https://api.publiccontractsscotland.gov.uk/v1

scrapy crawl scotland_public_contracts

Slovakia#

class kingfisher_scrapy.spiders.slovakia_digiwhist.SlovakiaDigiwhist(*args, **kwargs)[source]#

scrapy crawl slovakia_digiwhist

Slovenia#

class kingfisher_scrapy.spiders.slovenia.Slovenia(*args, **kwargs)[source]#

Domain: Ministry of Public Administration Slovenia
Bulk download documentation: http://tbfy.ijs.si/?m=tenders&a=list_data&t=si_ocds

scrapy crawl slovenia

class kingfisher_scrapy.spiders.slovenia_digiwhist.SloveniaDigiwhist(*args, **kwargs)[source]#

scrapy crawl slovenia_digiwhist

Spain#

class kingfisher_scrapy.spiders.spain_digiwhist.SpainDigiwhist(*args, **kwargs)[source]#

scrapy crawl spain_digiwhist

class kingfisher_scrapy.spiders.spain_zaragoza.SpainZaragoza(*args, **kwargs)[source]#

Domain: Ayuntamiento de Zaragoza
Caveats: The API’s before and after query string parameters have no effect and are therefore not implemented.
Swagger API documentation: https://www.zaragoza.es/docs-api_sede/

scrapy crawl spain_zaragoza

Sweden#

class kingfisher_scrapy.spiders.sweden_digiwhist.SwedenDigiwhist(*args, **kwargs)[source]#

scrapy crawl sweden_digiwhist

Switzerland#

class kingfisher_scrapy.spiders.switzerland_digiwhist.SwitzerlandDigiwhist(*args, **kwargs)[source]#

scrapy crawl switzerland_digiwhist

Tanzania#

class kingfisher_scrapy.spiders.tanzania_zabuni.TanzaniaZabuni(*args, **kwargs)[source]#

Domain: Tanzania Zabuni
API documentation: https://zabuni.co.tz/docs

scrapy crawl tanzania_zabuni

Uganda#

class kingfisher_scrapy.spiders.uganda_releases.UgandaReleases(*args, **kwargs)[source]#

Domain

Government Procurement Portal (GPP) of Public Procurement and Disposal of Public Assets Authority (PPDA)

Caveats

The domains described in the API documentation must be replaced by https://gpppapi.com

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2017’. The year refers to the start of the fiscal year range, e.g. if from_date = ‘2017’ then the fiscal year is ‘2017-2018’
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year. The year refers to the start of the fiscal year range, e.g. if until_date = ‘2017’ then the fiscal year is ‘2017-2018’

API documentation

https://docs.google.com/spreadsheets/d/10tVioy-VOQa1FwWoRl5e1pMbGpiymA0iycNcoDFkvks/edit#gid=365266172

scrapy crawl uganda_releases

Ukraine#

class kingfisher_scrapy.spiders.ukraine.Ukraine(*args, **kwargs)[source]#

Domain

ProZorro OpenProcurement API

Caveats

The API returns OCDS-like contracting processes data, however an ocid is not set. Therefore, as part of this spider, the data.tenderID is used as the ocid and the data.id + data.dateModified fields are used and release.id

Spider arguments

from_date: Download only data from this time onward (YYYY-MM-DDThh:mm:ss format).

API documentation

https://prozorro-api-docs.readthedocs.io/uk/latest/tendering/index.html

scrapy crawl ukraine

United Kingdom#

class kingfisher_scrapy.spiders.united_kingdom_contracts_finder_records.UnitedKingdomContractsFinderRecords(*args, **kwargs)[source]#

Domain: Contracts Finder
API documentation: https://www.contractsfinder.service.gov.uk/apidocumentation/home

scrapy crawl united_kingdom_contracts_finder_records

class kingfisher_scrapy.spiders.united_kingdom_contracts_finder_releases.UnitedKingdomContractsFinderReleases(*args, **kwargs)[source]#

Domain: Contracts Finder
API documentation: https://www.contractsfinder.service.gov.uk/apidocumentation/home
Caveats: The records endpoint is used to get the releases URLs for each ocid.

scrapy crawl united_kingdom_contracts_finder_releases

class kingfisher_scrapy.spiders.united_kingdom_digiwhist.UnitedKingdomDigiwhist(*args, **kwargs)[source]#

scrapy crawl united_kingdom_digiwhist

class kingfisher_scrapy.spiders.united_kingdom_fts.UnitedKingdomFTS(*args, **kwargs)[source]#

Domain

Find a Tender Service (FTS)

Spider arguments

from_date: Download only data from this time onward (YYYY-MM-DDThh:mm:ss format). If until_date is provided, defaults to ‘2021-01-01T00:00:00’.
until_date: Download only data until this time (YYYY-MM-DDThh:mm:ss format). If from_date is provided, defaults to now.

API documentation

https://www.find-tender.service.gov.uk/apidocumentation/1.0/GET-ocdsReleasePackages

scrapy crawl united_kingdom_fts

class kingfisher_scrapy.spiders.united_kingdom_fts_test.UnitedKingdomFTSTest(*args, **kwargs)[source]#

Domain: Find a Tender Service (FTS)
Caveats: This spider uses a test service that returns test data. For getting real data see uk_fts instead.

scrapy crawl united_kingdom_fts_test

Uruguay#

class kingfisher_scrapy.spiders.uruguay_historical.UruguayHistorical(*args, **kwargs)[source]#

Domain

Agencia Reguladora de Compras Estatales (ARCE)

Spider arguments

from_date: Download only data from this year onward (YYYY format). If until_date is provided, defaults to ‘2002’.
until_date: Download only data until this year (YYYY format). If from_date is provided, defaults to the current year.

Bulk download documentation

https://www.gub.uy/agencia-compras-contrataciones-estado/datos-y-estadisticas/datos/open-contracting

scrapy crawl uruguay_historical

class kingfisher_scrapy.spiders.uruguay_records.UruguayRecords(*args, **kwargs)[source]#

Domain

Agencia Reguladora de Compras Estatales (ARCE)

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). Defaults to ‘2017-11’.
until_date: Download only data until this month (YYYY-MM format). Defaults to the current month.

API documentation

https://www.gub.uy/agencia-compras-contrataciones-estado/datos-y-estadisticas/datos/open-contracting

scrapy crawl uruguay_records

class kingfisher_scrapy.spiders.uruguay_releases.UruguayReleases(*args, **kwargs)[source]#

Domain

Agencia Reguladora de Compras Estatales (ARCE)

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). Defaults to ‘2017-11’.
until_date: Download only data until this month (YYYY-MM format). Defaults to the current month.

API documentation

https://www.gub.uy/agencia-compras-contrataciones-estado/datos-y-estadisticas/datos/open-contracting

scrapy crawl uruguay_releases

Zambia#

class kingfisher_scrapy.spiders.zambia.Zambia(*args, **kwargs)[source]#

Domain

Zambia Public Procurement Authority (ZPPA)

Spider arguments

from_date: Download only data from this month onward (YYYY-MM format). If until_date is provided, defaults to ‘2016-07’.
until_date: Download only data until this month (YYYY-MM format). If from_date is provided, defaults to the current month.

scrapy crawl zambia