Edge-Native Digitization of Handwritten Marksheets: A Hybrid Heuristic-Deep Learning Framework

TL;DR

This work tackles the edge-constraint digitization of handwritten marksheets by decoupling structure detection from content recognition. It introduces a hybrid pipeline that uses a heuristic OpenCV TSR for grid detection and a pruned Modified YOLOv8 for single-character recognition, guided by deterministic routing to assign columns to the appropriate recognizer. Empirical results on EMNIST and AMES demonstrate high recognition fidelity (≈97.5% on EMNIST; ≈95.5%–95.8% on AMES) with real-time CPU performance (≈29 FPS) and substantial efficiency gains over traditional OCR and heavier DL models. The approach achieves a favorable balance between accuracy and speed, offering a practical, edge-friendly solution for scalable document automation in education settings.

Abstract

The digitization of structured handwritten documents, such as academic marksheets, remains a significant challenge due to the dual complexity of irregular table structures and diverse handwriting styles. While recent Transformer-based approaches like TableNet and TrOCR achieve state-of-the-art accuracy, their high computational cost renders them unsuitable for resource-constrained edge deployments. This paper introduces a resource-efficient hybrid framework that integrates a heuristic OpenCV-based pipeline for rapid table structure detection with a modified lightweight YOLOv8 architecture for handwritten character recognition. By strategically removing the SPPF and deep C2f layers from the standard YOLOv8 backbone, we reduce computational overhead while maintaining high recognition fidelity. Experimental results on the EMNIST digit benchmark demonstrate that our Modified YOLOv8 model achieves 97.5% accuracy. Furthermore, we provide a comprehensive efficiency analysis showing that our framework offers a 95 times inference speedup over standard OCR pipelines and massive efficiency gains over emerging Large Multimodal Models (LMMs) like Qwen2.5-VL, achieving real-time performance 29 FPS on standard CPU hardware. A qualitative and quantitative evaluation on the AMES dataset, a challenging subset of real-world marksheets, confirms the system's robustness in handling mixed alphanumeric content, bridging the gap between high-performance deep learning and practical, scalable document automation.

Figures (8)

• Figure 1: The complete workflow of the proposed hybrid system. The pipeline integrates preprocessing, heuristic line detection, cell extraction, and a recognition stage utilizing either Modified YOLOv8 or PaddleOCR.
• Figure 2: Visualization of Our Created Dataset of Handwritten Digits.
• Figure 3: Heuristic Table Structure Detection Pipeline.
• Figure 4: Architecture of the Modified YOLOv8. We pruned the SPPF and deep C2f layers from the standard backbone and designed a lightweight classification head with a flattened linear layer, optimizing it for single-character recognition.
• Figure 5: PaddleOCR Architecture used for fallback recognition.