datasets/cm-ocr-quality/2026-10-12/generation
prov:Activity · http://data.judaicalink.org/datasets/cm-ocr-quality/2026-10-12/generation
From cm-ocr-quality
- rdf:type
- prov:Activity
- rdfs:comment
Input not published (inputs:compact-memory/fulltext-2026-10-05-v2, 43 files, 2,097,436,345 bytes; named by checksum in the run record): Compact Memory full text for the Solr core cm_meta (bundle v2). 846,500 pages of Compact Memory as documents for the Solr core cm_meta, one JSON…
Input not published (inputs:compact-memory/fulltext-2026-10-05-v2, 43 files, 2,097,436,345 bytes; named by checksum in the run record): Compact Memory full text for the Solr core cm_meta (bundle v2). 846,500 pages of Compact Memory as documents for the Solr core cm_meta, one JSON object per line in 35 gzipped parts (docs-NNNN.jsonl.gz): page VLID, journal, date of issue, page label, volume, OCR text. manifest.json lists the parts with their checksums, the pages left out (no OCR file, no metadata, no text) and the corrections (Hebrew calendar years converted to Gregorian, which can be one year off; '(null)' removed; volumes rebuilt from the MODS records). date_review.tsv lists 513 pages whose year is isolated from the rest of their journal (not changed). The bundle also holds its README.md, the scripts that made and corrected it (extract_cm_fulltext.py, fix_bundle.py with fix_bundle.log), the loader for Solr (load_cm_fulltext.py) and dropped.tsv.gz (the pages left out, with the reason). Origin: Made on 2026-10-05 by extract_cm_fulltext.py (cm_indexer in the repository of the FID portal, handed over to Frankfurt) from the OCR XML files of Compact Memory on the JudaicaLink labs server in Mannheim, the page table (compact-memory-supp) and the MODS records (inputs:compact-memory/mods-2020-06-15); a copy is on that server.- rdfs:comment
- Word list: 116,604 words occurring at least 10 times in every 20th page.
- rdfs:seeAlso
- https://github.com/judaicalink/judaicalink-data/blob/main/generators/cm-ocr-quality/runs/2026-10-12.json
- prov:endedAtTime
- 2026-10-12T07:58:18Z
- prov:startedAtTime
- 2026-10-12T07:55:18Z
- prov:used
- dumps/compact-memory/2026-10-12/compact-memory-pages.ttl.gz
- prov:used
- https://github.com/judaicalink/judaicalink-data/blob/b8e227c3dc120eeabdda242096eca3683fbc2683/generators/cm-ocr-quality/generate.py
- prov:wasAssociatedWith
- Kai Eckert
- prov:wasAssociatedWith
- judaicalink-data: the JudaicaLink data generator framework