emailconfirmed
952
edits
Main Page: Difference between revisions
Jump to navigation
Jump to search
→Our Wikibase.cloud unboxing experience and first findings
OlafJanssen (talk | contribs) |
OlafJanssen (talk | contribs) |
||
| (20 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
= | =Unofficial Wikibase sandbox for the national library of The Netherlands.= | ||
==Disclaimer== | |||
''<span style="color:maroon">This Wikibase sandbox is for playing, experimenting, learning about the functionalities, possibilities and limitations etc. of Wikibase, and knowledge sharing activities resulting from these learnings. It is maintained by [https://www.wikidata.org/wiki/User:OlafJanssen Olaf Janssen], the Wikimedia coordinator of the KB, mainly as a '''tool for experimentation, learning, skill-building, knowledge sharing and consultancy about Wikibase''', both towards KB staff, (inter)national Wiki(base) communities, GLAMs, KB network partners, as well as other 3rd parties. It does not contain any offical data(sets) of the KB, and the data can be changed/deleted at any time. This instance is not supported by the KB, nor is it part of the official KB IT infrastructure. If you are interested in the official Wikibase instance of the KB, please visit xxx.</span>'' | |||
== Quick links == | == Quick links == | ||
| Line 135: | Line 137: | ||
=== Nog verder bekijken, waarschijnlijk relevant=== | === Nog verder bekijken, waarschijnlijk relevant=== | ||
* https://github.com/LeMyst/WikibaseIntegrator | * https://github.com/LeMyst/WikibaseIntegrator. Zie ook de verdere import opties op https://www.mediawiki.org/wiki/Wikibase/Importing | ||
==Reconciliation service for this Wikibase== | |||
https://github.com/KBNLresearch/OpenRefine-Wikibase | |||
* Recon service o.b.v. Docker: https://openrefine-wikibase.readthedocs.io/en/latest/install.html | |||
* Example config file: https://github.com/KBNLresearch/OpenRefine-Wikibase/blob/main/config.py | |||
* Configuration of OpenRefine: add URLs of recon service and manifest to OpenRefine | |||
* Example manifest: https://github.com/KBNLresearch/OpenRefine-Wikibase/blob/main/kb-test-wikibase-cloud-manifest.json | |||
== Snelheid van bulk data-importeren == | == Snelheid van bulk data-importeren == | ||
| Line 142: | Line 151: | ||
== Wikibase.cloud project updates == | == Wikibase.cloud project updates == | ||
* https://meta.wikimedia.org/wiki/Wikibase/Wikibase.cloud | * https://meta.wikimedia.org/wiki/Wikibase/Wikibase.cloud | ||
== Wikibase.cloud documenation == | |||
* https://www.mediawiki.org/wiki/Wikibase/Wikibase.cloud | |||
== Our Wikibase.cloud unboxing experience and first findings == | == Our Wikibase.cloud unboxing experience and first findings == | ||
Below is an overview of the most important issues we encountered. The numbering refers to the slides in the presentation: | '''<big>The text below is a transcript of the presentation ''[https://commons.wikimedia.org/wiki/File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf KB's Wikibase.cloud Unboxing Experience]''</big>''' | ||
In June-July 2022 the KB created and configured this instance on Wikibase.cloud. Many of the steps we needed to take in the setup process were as to be expected when working with fresh MediaWiki instances in general, and Wikidata in particular. However, some steps were not immediately logical, manifest, clearly documented or simply beyond our (then) knowledge and experience. We created a '''[https://commons.wikimedia.org/wiki/File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf presentation]''' in which we share experiences and first impressions of unboxing, setting up, configuring and tweaking '''our [https://kbtestwikibase.wikibase.cloud/wiki/Main_Page Wikibase.cloud sandbox instance]'''. We also present solutions for most of the issues we encountered. | |||
Below is an overview of the most important issues we encountered. The numbering refers to the corresponding slides in the presentation: | |||
;Setting up our Wikibase.cloud test instance ([https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=2 slide 2] - [https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=13 slide 13]) | ;Setting up our Wikibase.cloud test instance ([https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=2 slide 2] - [https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=13 slide 13]) | ||
* We succesfully created our test instance at https://kbtestwikibase.wikibase.cloud via a personal [https://www.wikibase.cloud/dashboard Wikibase.cloud dashboard] every user gets, [https://www.mediawiki.org/wiki/Wikibase/Wikibase.cloud/Wikis as described here]. This went without any problems, as expected. | * We succesfully created our test instance at https://kbtestwikibase.wikibase.cloud via a personal [https://www.wikibase.cloud/dashboard Wikibase.cloud dashboard] every user gets, [https://www.mediawiki.org/wiki/Wikibase/Wikibase.cloud/Wikis as described here]. This went without any problems, as expected. | ||
* ISSUE: ''We would have liked to be able to [https://www.wikibase.cloud/wikis/create create another Wikibase.cloud instance]'' from this dashboard, but currently there is a hard limit of [https://phabricator.wikimedia.org/T311121 one instance per account]. There is no solution yet. | * ISSUE: ''We would have liked to be able to [https://www.wikibase.cloud/wikis/create create another Wikibase.cloud instance]'' from this dashboard, but currently (15th Aug 2022) there is a hard limit of [https://phabricator.wikimedia.org/T311121 one instance per account]. | ||
: SOLUTION: There is no solution yet. | |||
;Setting up basic content (Ps and Qs) via the GUI ([https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=14 slide 14] - [https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=29 slide 29]) | ;Setting up basic content (Ps and Qs) via the GUI ([https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=14 slide 14] - [https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=29 slide 29]) | ||
* As can be seen from the [https://kbtestwikibase.wikibase.cloud/wiki/Special:ListUsers users list], there is a preset default admin [[User:Ookgezellig]] account for this instance, resulting from the global wikibase.cloud creation account (ookgezellig@gmail.com) | * As can be seen from the [https://kbtestwikibase.wikibase.cloud/wiki/Special:ListUsers users list], there is a preset default admin [[User:Ookgezellig]] account for this instance, resulting from the global wikibase.cloud creation account (ookgezellig@gmail.com) | ||
* [https://kbtestwikibase.wikibase.cloud/w/index.php?title=Special:CreateAccount&returnto=Main+Page Creating additional, regular user accounts] on this Wiki instance was as expected. We created [[User:OlafJanssen]] with normal, restricted privileges. | * [https://kbtestwikibase.wikibase.cloud/w/index.php?title=Special:CreateAccount&returnto=Main+Page Creating additional, regular user accounts] on this Wiki instance was as expected. We created [[User:OlafJanssen]] with normal, restricted privileges. | ||
* ISSUE: ''Regular user accounts with default privileges are not allowed to add URLs to items'' (see Phabricator [https://phabricator.wikimedia.org/T310419 T310419], [https://phabricator.wikimedia.org/T86453 T86453] and [https://phabricator.wikimedia.org/T310421 T310421]). In order to be able to add the Wikidata URL to (for instance) [https://kbtestwikibase.wikibase.cloud/wiki/Property:P3 P3] ("is a unique exmple of"), we used the [[User:Ookgezellig]] admin account to temporarily upgrade the [[User:OlafJanssen]] account to admin privileges as well. | * ISSUE: ''Regular user accounts with default privileges are not allowed to add URLs to items'' (see Phabricator [https://phabricator.wikimedia.org/T310419 T310419], [https://phabricator.wikimedia.org/T86453 T86453] and [https://phabricator.wikimedia.org/T310421 T310421]). | ||
* IMPORTANT: the Wikidata URL in [https://kbtestwikibase.wikibase.cloud/wiki/Property:P3 P3] must be formatted as '''http:'''//www.wikidata.org/'''entity'''/P31, and NOT as http'''s'''://www.wikidata.org/entity/P31, and NOT as <nowiki>https://www.wikidata.org/</nowiki>'''wiki/Property:'''P31, or any other mix of these incorrect | : SOLUTION: In order to be able to add the Wikidata URL to (for instance) [https://kbtestwikibase.wikibase.cloud/wiki/Property:P3 P3] ("is a unique exmple of"), we used the [[User:Ookgezellig]] admin account to temporarily upgrade the [[User:OlafJanssen]] account to admin privileges as well. | ||
* ISSUE: When adding statements to a Q-item such as [[Item:Q29|Theun de Vries]], ''by default the statements are (and remain) displayed in the same order as you originally added them''. This is different from Wikidata, where in the corresponding [https://www.wikidata.org/wiki/Q2143934 Wikidata item about Theun de Vries] the statements are displayed in a custom, fixed order, with "Instance of" and "Image" on top, and a list of external identifiers at the bottom of the page, irrespective of the exacty order these statements were originally added to this Wikidata item. This issue was solved by adding the page [[MediaWiki:Wikibase-SortedProperties]] to the Wikibase, in which the custom order of statements can be specified. This is the same as in Wikidata, see [https://www.wikidata.org/wiki/MediaWiki:Wikibase-SortedProperties MediaWiki:Wikibase-SortedProperties] and [https://www.mediawiki.org/wiki/Manual:Interface/Wikibase-SortedProperties the manual] on Wikidata. Please note that the required page cache purging needed for this functionality to kick in can be actively triggered via [[Special:Purge]]. | * IMPORTANT: the Wikidata URL in [https://kbtestwikibase.wikibase.cloud/wiki/Property:P3 P3] must be formatted as '''http:'''//www.wikidata.org/'''entity'''/P31, and NOT as http'''s'''://www.wikidata.org/entity/P31, and NOT as <nowiki>https://www.wikidata.org/</nowiki>'''wiki/Property:'''P31, or any other mix of these incorrect URL syntaxes. This is important for correctly functioning, abbreviated display of these URLs in the Wikibase query service ("[http://www.wikidata.org/entity/P31 wd:P31]", as can be seen in the query result <nowiki>https://tinyurl.com/2kd5dthd</nowiki>). | ||
* ISSUE: When adding statements to a Q-item such as [[Item:Q29|Theun de Vries]], ''by default the statements are (and remain) displayed in the same order as you originally added them''. This is different from Wikidata, where in the corresponding [https://www.wikidata.org/wiki/Q2143934 Wikidata item about Theun de Vries] the statements are displayed in a custom, fixed order, with "Instance of" and "Image" on top, and a list of external identifiers at the bottom of the page, irrespective of the exacty order these statements were originally added to this Wikidata item. | |||
: SOLUTION: This issue was solved by adding the page [[MediaWiki:Wikibase-SortedProperties]] to the Wikibase, in which the custom order of statements can be specified. This is the same as in Wikidata, see [https://www.wikidata.org/wiki/MediaWiki:Wikibase-SortedProperties MediaWiki:Wikibase-SortedProperties] and [https://www.mediawiki.org/wiki/Manual:Interface/Wikibase-SortedProperties the manual] on Wikidata. Please note that the required page cache purging needed for this functionality to kick in can be actively triggered via [[Special:Purge]]. | |||
;SPARQL and REST APIs ([https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=30 slide 30] - [https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=43 slide 43]) | ;SPARQL and REST APIs ([https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=30 slide 30] - [https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=43 slide 43]) | ||
* We created a small number of [[Project:SPARQL/examples|SPARQL query examples]] on this Wikibase. | * We created a small number of [[Project:SPARQL/examples|SPARQL query examples]] on this Wikibase. | ||
* ISSUE: ''the Wikidata [https://www.wikidata.org/wiki/Template:SPARQL SPARQL template] for query syntax highlighting is not implemented in this Wikibase''. Instead one must use a ''<nowiki><sparql tryit="1">...</sparql></nowiki>'' wrapper, as demonstrated [https://kbtestwikibase.wikibase.cloud/w/index.php?title=Project:SPARQL/examples&action=submit in this example]. | * ISSUE: ''the Wikidata [https://www.wikidata.org/wiki/Template:SPARQL SPARQL template] for query syntax highlighting is not implemented in this Wikibase''. | ||
* | : SOLUTION: Instead one must use a ''<nowiki><sparql tryit="1">...</sparql></nowiki>'' wrapper, as demonstrated [https://kbtestwikibase.wikibase.cloud/w/index.php?title=Project:SPARQL/examples&action=submit in this example]. | ||
* IMPORTANT: The exact URL [https://kbtestwikibase.wikibase.cloud/wiki/Project:SPARQL/examples https://kbtestwikibase.wikibase.cloud/wiki/'''Project:SPARQL/examples'''], as well as the correct formatting of this page, are important to correctly display these query examples in the [https://kbtestwikibase.wikibase.cloud/query/# query services interface] (click the Examples button at the top) | * IMPORTANT: ''in Wikidata prefixes don't need to be specified explicitly in SPARQL queries''. However, in our Wikibase.cloud instance, prefixes must be explicitly stated, as [[Project:SPARQL/examples#Federated_querying:_mixing_data_from_this_Wikibase_with_data_from_Wikidata|this example]] shows. Please note and mind the distinction between PREFIX kbwdt, kbwd, wdt and wd. | ||
* IMPORTANT: The exact URL [https://kbtestwikibase.wikibase.cloud/wiki/Project:SPARQL/examples https://kbtestwikibase.wikibase.cloud/wiki/'''Project:SPARQL/examples'''], as well as the correct formatting of this page, are important to correctly display these query examples in the [https://kbtestwikibase.wikibase.cloud/query/# query services interface] (click the ''Examples'' button at the top) | |||
* Via the regular [https://kbtestwikibase.wikibase.cloud/w/api.php Wikimedia REST API] the full [[Item:Q29|item on Theun de Vries]] can be requested [https://kbtestwikibase.wikibase.cloud/w/api.php?action=wbgetentities&ids=Q29&format=json as JSON] or [https://kbtestwikibase.wikibase.cloud/w/api.php?action=wbgetentities&ids=Q29&format=xml as XML] as well. | |||
* | * An alternative way for requesting full Wikibase items directly from the Qnumber is via the [https://www.mediawiki.org/wiki/Wikibase/EntityData Special:EntityData] URL. The ouput can be obtained in seven different formats: [[Special:EntityData/Q29|HTML]], [[Special:EntityData/Q29.json|JSON]], [[Special:EntityData/Q29.rdf|RDF/XML]], [[Special:EntityData/Q29.nt|NT]], [[Special:EntityData/Q29.ttl|TTL]], [[Special:EntityData/Q29.n3|N3]] and [[Special:EntityData/Q29.php|PHP]]. | ||
: Or equivalently, using a format argument, eg [https://kbtestwikibase.wikibase.cloud/wiki/Special:EntityData?id=Q29&format=json Special:EntityData?id=Q29&format=json] for a JSON respons. | |||
* For simple interaction with the Wikibase API via Python, the [https://github.com/samuelmeuli/wikibase-api wikibase-api] Python library can be used . [https://wikibase-api.readthedocs.io/en/latest/getting_started/installation_and_usage.html Read the docs] for more details. | |||
* | : Examples on how to use this library are available at https://github.com/KBNLresearch/wikibase-api/tree/master/examples-kb + https://github.com/kbnlresearch/wikibase-api/blob/master/examples-kb/wikibase-api.py | ||
* Wikibase | |||
* | ;Three problems we encounterd, yet unsolved ([https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=44 slide 44] - [https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=52 slide 52]) | ||
* ISSUE: ''Regular user accounts with default privileges are not allowed to add URLs to items'' | |||
* | : SOLUTION: See discussion above | ||
* ISSUE: ''Creation of common.js and common.css pages on Wikibase.cloud instances is not allowed.'' On Wikidata, users (eg. [https://www.wikidata.org/wiki/User:OlafJanssen User:OlafJanssen]) can create personal [https://www.wikidata.org/wiki/User:OlafJanssen/common.js common.js] and [https://www.wikidata.org/wiki/User:OlafJanssen/common.css common.css] pages for adding custom functionalities and layout to their Wikidata interfaces. On Wikibase.cloud, this is currently not possible. | |||
: SOLUTION: No solution yet, see Phabricator [https://phabricator.wikimedia.org/T310787 T310787]. | |||
* ISSUE: ''Default data import speed is rather limited''. When importing items into a Wikibase using REST API-based tools such as [https://www.wikidata.org/wiki/Help:QuickStatements QuickStatements], [https://www.mediawiki.org/wiki/Manual:Pywikibot pywikibot] or [https://github.com/SuLab/WikidataIntegrator WikidataIntegrator], the maximum import speed is about 3-10 items per second, as explained in the article ''[https://www.wikibase.consulting/fast-bulk-import-into-wikibase/ Fast Bulk Import Into Wikibase]'' by [https://www.wikibase.consulting/ The Wikibase Consultancy]. This is an OK speed for small datasets, but if you (like the KB) plan to import millions of items, this is obviously too slow. | |||
: SOLUTION: PHP scripts or direct SQL tools such as [https://github.com/UB-Mannheim/RaiseWikibase RaiseWikibase], as further outlined in the article. At the moment these solutions are not considered nor used by the KB. | |||
;Where to get help? ([https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page= | ;Where to get help? ([https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=53 slide 53] - [https://commons.wikimedia.org/w/index.php?title=File:KB_Wikibase.cloud_Unboxing_Experience,_Netherlands_Wikibase_Knowlegde_Group,_22-07-2022.pdf&page=58 slide 58]) | ||
# Wikibase.cloud issues on Phabricator: https://phabricator.wikimedia.org/tag/wikibase.cloud/ | # Wikibase.cloud issues on Phabricator: https://phabricator.wikimedia.org/tag/wikibase.cloud/ | ||
# Wikibase.cloud updates: https://meta.wikimedia.org/wiki/Wikibase/Wikibase.cloud | # Wikibase.cloud updates: https://meta.wikimedia.org/wiki/Wikibase/Wikibase.cloud | ||