renAIssance VLM OCR for 17th-Century Spanish with Targeted LLM Correction

Project Summary: end-to-end OCR for 17th-Century Spanish Text using VLMs and assisted correction by an LLM The project addresses the significant challenge of performing accurate Optical Character Recognition (OCR) on 17th-century Spanish printed and handwritten documents. These historical materials are difficult for standard OCR tools due to archaic vocabulary, orthographic variations, and visual degradation such as ink bleed-through and faded text. Proposed Solution consists of an end-to-end pipeline centered on the Qwen2.5-VL-7B vision-language model (VLM). My approach moves beyond simple off-the-shelf recognition by implementing a multi-stage adaptation strategy: - Language Adaptation: Using LoRA-style masked language modeling (MLM) on the CORDE and CODEA corpora to teach the model 17th-century linguistic patterns and abbreviations. - Supervised Fine-Tuning: Training the adapted model on ground-truth transcriptions from the RenAlssance and RODRIGO datasets. - Targeted LLM Correction: Implementing a confidence-gated correction layer. Unlike generic "wholesale" correction which often introduces hallucinations, this layer uses a smaller, historically-tuned LLM to fix only low-confidence spans identified by the OCR model. Key Deliverables: By the end of the project, I aim to deliver: - A Reproducible OCR Pipeline: A complete system capable of handling both printed and handwritten 17th-century Spanish text. - Targeted Correction Module: A historically-adapted module that improves transcription accuracy by focusing only on uncertain text segments. - Evaluation and Normalization Harness: A documented toolkit for consistent Character Error Rate (CER) and Word Error Rate (WER) assessment. - Ablation Studies: Detailed reports isolating the performance gains from each stage of adaptation (language, visual, and correction) to guide future contributors.