Entropy Heat-Mapping: Localizing GPT-Based OCR Errors with Sliding-Window Shannon Analysis

TL;DR

This work addresses the problem of locating local OCR errors in GPT-based mathematical transcription by converting per-token entropy into a visual uncertainty landscape using a sliding-window mean $A_i$. It introduces entropy heat-mapping and a top-$k$ plus tail entropy framework to produce hotspot regions where errors are likely, complemented by a proof-of-concept evaluation on a small, multi-resolution corpus. The authors formalize the method, provide a mathematical justification that the top-$k$+tail entropy lower-bounds true entropy, and demonstrate qualitative alignment between hotspots and actual errors, supported by open-source resources for replication. The approach offers a lightweight, scalable aid for post-editing GPT-based OCR, with potential extensions to automated re-prompting and broader digitisation workflows.

Abstract

Vision-language models such as OpenAI GPT-4o can transcribe mathematical documents directly from images, yet their token-level confidence signals are seldom used to pinpoint local recognition mistakes. We present an entropy-heat-mapping proof-of-concept that turns per-token Shannon entropy into a visual ''uncertainty landscape''. By scanning the entropy sequence with a fixed-length sliding window, we obtain hotspots that are likely to contain OCR errors such as missing symbols, mismatched braces, or garbled prose. Using a small, curated set of scanned research pages rendered at several resolutions, we compare the highlighted hotspots with the actual transcription errors produced by GPT-4o. Our analysis shows that the vast majority of true errors are indeed concentrated inside the high-entropy regions. This study demonstrates--in a minimally engineered setting--that sliding-window entropy can serve as a practical, lightweight aid for post-editing GPT-based OCR. All code and annotation guidelines are released to encourage replication and further research.