mardi_importer.integrator module

mardi_importer.integrator.MardiIntegrator module

class mardi_importer.integrator.MardiIntegrator.MardiIntegrator(languages=['en', 'de'])[source]

Bases: MardiClient

config()[source]

Sets up initial configuration for the integrator

Returns:

Clientlogin object

convert_claim_ids(entity)[source]

Function for in-place conversion of wikidata ids found in claims into local ids

Parameters:

entity

Returns:

None

Function for in-place conversion of unit for quantity and globe for globecoordinate to a link to the local entity instead of a link to the wikidata entity.

Parameters:

snak – a wikibaseintegrator snak

Returns:

None

create_db_table()[source]

Check if db table for id mapping is there; if not, create.

Parameters:

None

Returns:

None

create_engine()[source]

Creates SQLalchemy engine

Returns:

SQLalchemy engine

create_id_list_from_file(file)[source]

Function for creating a list of ids from a while where each id is in a new line

Parameters:

file – path to file

Returns: list of ids

get_local_id_by_label(entity_str, entity_type)[source]

Check if entity with a given label or wikidata PID/QID exists in the local wikibase instance.

Parameters:

  • entity_str (str) – It can be a string label or a wikidata ID, specified with the prefix wdt: for properties and wd: for items.

  • entity_type (str) – Either ‘property’ or ‘item’ to specify which type of entity to look for.

Returns:

Local ID of the entity, if found.

Return type:

str

get_qualifiers(claim)[source]

Function for creating qualifiers from wikidata qualifiers and in place adding them to the claim

Parameters:

claim – a wikibaseintegrator claim

Returns:

Qualifiers object, can also be an empty object

get_references(claim)[source]

Function for creating references from wikidata references and in place adding them to the claim

Parameters:

claim – a wikibaseintegrator claim

Returns:

List with references, can also be an empty list

get_wikidata_information(wikidata_id, recurse=False)[source]

Function for pulling wikidata information

Parameters:

  • wikidata_id (str) – wikidata id of the desired entity

  • recurse (Bool) – if claims should also be imported

Returns: wikibase integrator entity or None, if the entity has no labels

import_claim_entities(wikidata_id)[source]

Function for importing entities that are mentioned in claims from wikidata to the local wikibase instance

Parameters:

wikidata_id (str) – id of the entity to be imported

Returns:

local id or None, if the entity had no labels

import_entities(id_list=None, filename='', recurse=True)[source]

Function for importing entities from wikidata into the local instance.

It can accept a single id, a list of ids or a file containing a the ids to be imported.

Parameters:

  • id_list – Single string or list of strings of wikidata entity ids. Lexemes not supported.

  • filename – Filename containing list of entities to be imported.

  • recurse – Whether to import claims for the entities in id_list

Returns:

Dictionary containing the local ids of all the imported entities.

Return type:

Imported entities (Dict)

import_from_label(label)[source]

Imports an entity from Wikidata just from a label

Parameters:

label (str) – label to be imported from wikidata

Returns:

local id for the imported entity

Return type:

local_id (str)

init_wikidata_PID()[source]

Searches the wikidata PID property ID to link properties to its ID in wikidata. When not found, it creates the property.

Returns

wikidata_PID (str): wikidata PID property ID

init_wikidata_QID()[source]

Searches the wikidata QID property ID to link items to its ID in wikidata. When not found, it creates the property.

Returns

wikidata_QID (str): wikidata QID property ID

insert_id_in_db(wikidata_id, local_id, has_all_claims)[source]

Insert wikidata_id, local_id and has_all_claims into mapping table.

Parameters:

  • wikidata_id – Wikidata id

  • local_id – local Wikibase id

  • has_all_claims – Boolean indicating whether the entity has been imported with all claims or no claims (i.e. no recurse)

Returns:

None

overwrite_entity(wikidata_id, local_id)[source]

Function for completing an already existing local entity with its statements from wikidata.

Parameters:

  • wikidata_id – Wikidata entity ID to be imported.

  • local_id – Local id of the existing entity that needs to be completed with further statements.

Returns:

Local entity ID

Return type:

local_id

query(parameter, wikidata_id)[source]

Query the wb_id_mapping db table for a given parameter.

The two important parameters are the local_id and whether the entity has already been imported with all claims

Parameters:

  • parameter (str) – Either local_id or has_all_claims

  • wikidata_id (str) – Wikidata ID

Returns:

for local_id returns the local ID if it exists,

otherwise None. For has_all_claims, a boolean is returned.

Return type:

str or boolean

query_with_local_id(parameter, local_id)[source]

Query the wb_id_mapping db table for a given parameter.

The two important parameters are the wikidata_id and whether the entity has already been imported with all claims

Parameters:

  • parameter (str) – Either wikidata_id or has_all_claims

  • local_id (str) – local ID

Returns:

for wikidata_id returns the wikidata ID if it exists,

otherwise None. For has_all_claims, a boolean is returned.

Return type:

str or boolean

update_has_all_claims(wikidata_id)[source]

Set the has_all_claims property in the wb_id_mapping table to True for the given wikidata_id.

Parameters:

wikidata_id – Wikidata id to be updated.

Returns:

None

mardi_importer.integrator.MardiEntities module

class mardi_importer.integrator.MardiEntities.MardiItemEntity(labels: Labels | None = None, descriptions: Descriptions | None = None, aliases: Aliases | None = None, sitelinks: Sitelinks | None = None, **kwargs: Any)[source]

Bases: MardiItem

get(entity_id, **kwargs)[source]

Request the MediaWiki API to get data for the entity specified in argument.

Parameters:

  • entity_id – The entity_id of the Item entity you want. Must start with a ‘Q’.

  • kwargs

Returns:

an ItemEntity instance

get_QID(alias=False)[source]

Creates a list of QID of all items in the local wikibase with the same label

Returns:

List of QID

Return type:

QIDs (list)

new(**kwargs)[source]
class mardi_importer.integrator.MardiEntities.MardiPropertyEntity(datatype: str | WikibaseDatatype | None = None, labels: Labels | None = None, descriptions: Descriptions | None = None, aliases: Aliases | None = None, **kwargs: Any)[source]

Bases: MardiProperty

get(entity_id, **kwargs)[source]
get_PID()[source]

Returns the PID of the property with the same label

new(**kwargs)[source]