datasets/compact-memory/2026-10-12/generation
prov:Activity · http://data.judaicalink.org/datasets/compact-memory/2026-10-12/generation
From compact-memory
- rdf:type
- prov:Activity
- rdfs:comment
- 1 page of the full-text bundle without an OCR file path left out.
- rdfs:comment
Input not published (inputs:compact-memory/fulltext-2026-10-05-v2, 43 files, 2,097,436,345 bytes; named by checksum in the run record): Compact Memory full text for the Solr core cm_meta (bundle v2). 846,500 pages of Compact Memory as documents for the Solr core cm_meta, one JSON…
Input not published (inputs:compact-memory/fulltext-2026-10-05-v2, 43 files, 2,097,436,345 bytes; named by checksum in the run record): Compact Memory full text for the Solr core cm_meta (bundle v2). 846,500 pages of Compact Memory as documents for the Solr core cm_meta, one JSON object per line in 35 gzipped parts (docs-NNNN.jsonl.gz): page VLID, journal, date of issue, page label, volume, OCR text. manifest.json lists the parts with their checksums, the pages left out (no OCR file, no metadata, no text) and the corrections (Hebrew calendar years converted to Gregorian, which can be one year off; '(null)' removed; volumes rebuilt from the MODS records). date_review.tsv lists 513 pages whose year is isolated from the rest of their journal (not changed). The bundle also holds its README.md, the scripts that made and corrected it (extract_cm_fulltext.py, fix_bundle.py with fix_bundle.log), the loader for Solr (load_cm_fulltext.py) and dropped.tsv.gz (the pages left out, with the reason). Origin: Made on 2026-10-05 by extract_cm_fulltext.py (cm_indexer in the repository of the FID portal, handed over to Frankfurt) from the OCR XML files of Compact Memory on the JudaicaLink labs server in Mannheim, the page table (compact-memory-supp) and the MODS records (inputs:compact-memory/mods-2020-06-15); a copy is on that server.- rdfs:comment
Input not published (inputs:compact-memory/mods-2020-06-15, 1 file, 51,520,917 bytes; named by checksum in the run record): Compact Memory metadata export of UB Frankfurt (MODS records and page table). Zip with two files: CM_Knoten_MODS_XML_15.06.2020, the MODS records of all nod…
Input not published (inputs:compact-memory/mods-2020-06-15, 1 file, 51,520,917 bytes; named by checksum in the run record): Compact Memory metadata export of UB Frankfurt (MODS records and page table). Zip with two files: CM_Knoten_MODS_XML_15.06.2020, the MODS records of all nodes of Compact Memory (journals, volumes, issues, articles; 64 MB of XML), and CM_Seiten_Metadaten_15.06.2020.xlsx, the page table (its CSV conversion is the file of compact-memory-supp). Origin: Export of the Visual Library of UB Frankfurt of 2020-06-15, received from UB Frankfurt; kept in the repository of the FID portal (wagtail-common-base, site/portal/judaicalink/sources/, commit ea89ebf3).- rdfs:comment
- Input not published (inputs:compact-memory/ocr-paths-2026-10-05, 1 file, 5,322,820 bytes; named by checksum in the run record): List of the Compact Memory OCR files. Gzipped text, one path per line (856,043 lines), relative to the XML folder: <journal>/<volume>/<issue>/<file prefix>_<page VLID>.xml. The page IRIs of compact-memory are built from these paths: <journal>-<volume>-<issue>--<file prefix>-<page VLID>. Origin: Listing of the OCR XML files of Compact Memory on the JudaicaLink labs server in Mannheim (find -name '*.xml' in the XML folder), 2026-10-05.
- rdfs:seeAlso
- https://github.com/judaicalink/judaicalink-data/blob/main/generators/compact-memory/runs/2026-10-12.json
- prov:endedAtTime
- 2026-10-12T07:55:02Z
- prov:startedAtTime
- 2026-10-12T07:53:59Z
- prov:used
- dumps/compact-memory-supp/2021-01-19/CM_Seiten_Metadaten.csv.gz
- prov:used
- dumps/compact-memory/2021-01-18/cm_tagme_pages.ttl.gz
- prov:used
- https://github.com/judaicalink/judaicalink-data/blob/b8e227c3dc120eeabdda242096eca3683fbc2683/generators/compact-memory/generate.py
- prov:wasAssociatedWith
- Kai Eckert
- prov:wasAssociatedWith
- judaicalink-data: the JudaicaLink data generator framework