datasets/cm-entities/2026-10-12/generation
prov:Activity · http://data.judaicalink.org/datasets/cm-entities/2026-10-12/generation
From cm-entities
- rdf:type
- prov:Activity
- rdfs:comment
Input not published (inputs:compact-memory/fulltext-2026-10-05-v2, 43 files, 2,097,436,345 bytes; named by checksum in the run record): Compact Memory full text for the Solr core cm_meta (bundle v2). 846,500 pages of Compact Memory as documents for the Solr core cm_meta, one JSON…
Input not published (inputs:compact-memory/fulltext-2026-10-05-v2, 43 files, 2,097,436,345 bytes; named by checksum in the run record): Compact Memory full text for the Solr core cm_meta (bundle v2). 846,500 pages of Compact Memory as documents for the Solr core cm_meta, one JSON object per line in 35 gzipped parts (docs-NNNN.jsonl.gz): page VLID, journal, date of issue, page label, volume, OCR text. manifest.json lists the parts with their checksums, the pages left out (no OCR file, no metadata, no text) and the corrections (Hebrew calendar years converted to Gregorian, which can be one year off; '(null)' removed; volumes rebuilt from the MODS records). date_review.tsv lists 513 pages whose year is isolated from the rest of their journal (not changed). The bundle also holds its README.md, the scripts that made and corrected it (extract_cm_fulltext.py, fix_bundle.py with fix_bundle.log), the loader for Solr (load_cm_fulltext.py) and dropped.tsv.gz (the pages left out, with the reason). Origin: Made on 2026-10-05 by extract_cm_fulltext.py (cm_indexer in the repository of the FID portal, handed over to Frankfurt) from the OCR XML files of Compact Memory on the JudaicaLink labs server in Mannheim, the page table (compact-memory-supp) and the MODS records (inputs:compact-memory/mods-2020-06-15); a copy is on that server.- rdfs:comment
Input not published (inputs:web-caches/api-cache-2026-10-06, 1464 files, 387,042,272 bytes; named by checksum in the run record): Answers of the Wikipedia, Wikidata and DNB EntityFacts APIs (kg-complements, 2026-10-06). 1,464 JSON answers, one file per request, named by the sha1 …
Input not published (inputs:web-caches/api-cache-2026-10-06, 1464 files, 387,042,272 bytes; named by checksum in the run record): Answers of the Wikipedia, Wikidata and DNB EntityFacts APIs (kg-complements, 2026-10-06). 1,464 JSON answers, one file per request, named by the sha1 of the request (URL + '?' + its parameters as JSON with sorted keys) under a folder of the first two hex digits: <xx>/<sha1>.json. A file holds the answer as returned (null for no record); it does not repeat the request. 97 Wikidata answers with complete entities make 365 of the 372 MB. Origin: Queried by the kg-complements builders on 2026-10-06 (cm-entities-wikidata: https://de.wikipedia.org/w/api.php and https://www.wikidata.org/w/api.php; gnd-persons-entityfacts: https://hub.culturegraph.org/entityfacts/).- rdfs:comment
- Presence: 5,445 entities.
- rdfs:comment
- Wikidata: 4,859 entities not named by any dataset; {'no Wikidata item': 11, 'other': 91, 'person': 3504, 'place': 1253}; 195 cached API answers used.
- rdfs:seeAlso
- https://github.com/judaicalink/judaicalink-data/blob/main/generators/cm-entities/runs/2026-10-12.json
- prov:endedAtTime
- 2026-10-12T07:59:13Z
- prov:startedAtTime
- 2026-10-12T07:58:18Z
- prov:used
- dumps/cm-tagme/2021-01-18/cm_tagme_references.ttl.gz
- prov:used
- dumps/cm-tagme/2021-01-18/cm_tagme_resources.ttl.gz
- prov:used
- dumps/dbpedia-persons/2018-10-24/Entityfacts-dbpedia-sameas.ttl.gz
- prov:used
- dumps/dbpedia-persons/2018-10-24/generated_persons_DBPedia_enriched_02.ttl.gz
- prov:used
- dumps/dbpedia-persons/2018-10-24/persons-gndid.ttl.gz
- prov:used
- dumps/interlinks/2018-07-18/interlinks-04-enriched-09.ttl.gz
- prov:used
- dumps/interlinks/2018-07-18/interlinks.n3.gz
- prov:used
- https://github.com/judaicalink/judaicalink-data/blob/b8e227c3dc120eeabdda242096eca3683fbc2683/generators/cm-entities/generate.py
- prov:wasAssociatedWith
- Kai Eckert
- prov:wasAssociatedWith
- judaicalink-data: the JudaicaLink data generator framework