Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Visual Merit or Linguistic Crutch? A Close Look at DeepSeek-OCR

Yunhao Liang, Ruixuan Ying, Bo Li, Hong Li, Kai Yan, Qingwen Li, Min Yang, Okamoto Satoshi, Zhe Cui, Shiwen Ni

TL;DR

The paper questions whether DeepSeek-OCR's high OCR precision under optical compression truly reflects visual understanding or largely exploits linguistic priors. By applying sentence-level and word-level semantic disruptions across multiple OCR/VLM architectures and downstream tasks (QA/VQA), it demonstrates that end-to-end models are highly priors-dependent, while pipeline OCR shows more robustness to semantic perturbations. Downstream reasoning degrades sharply when priors are disrupted, indicating that OCR fidelity alone is insufficient for meaningful comprehension. The study reveals a hard context-length ceiling around 8k–10k tokens for optical compression, challenging the premise of scalable long-context via visual tokens and urging prior-agnostic evaluation and architectural redesign.

Abstract

DeepSeek-OCR utilizes an optical 2D mapping approach to achieve high-ratio vision-text compression, claiming to decode text tokens exceeding ten times the input visual tokens. While this suggests a promising solution for the LLM long-context bottleneck, we investigate a critical question: "Visual merit or linguistic crutch - which drives DeepSeek-OCR's performance?" By employing sentence-level and word-level semantic corruption, we isolate the model's intrinsic OCR capabilities from its language priors. Results demonstrate that without linguistic support, DeepSeek-OCR's performance plummets from approximately 90% to 20%. Comparative benchmarking against 13 baseline models reveals that traditional pipeline OCR methods exhibit significantly higher robustness to such semantic perturbations than end-to-end methods. Furthermore, we find that lower visual token counts correlate with increased reliance on priors, exacerbating hallucination risks. Context stress testing also reveals a total model collapse around 10,000 text tokens, suggesting that current optical compression techniques may paradoxically aggravate the long-context bottleneck. This study empirically defines DeepSeek-OCR's capability boundaries and offers essential insights for future optimizations of the vision-text compression paradigm. We release all data, results and scripts used in this study at https://github.com/dududuck00/DeepSeekOCR.

Visual Merit or Linguistic Crutch? A Close Look at DeepSeek-OCR

TL;DR

The paper questions whether DeepSeek-OCR's high OCR precision under optical compression truly reflects visual understanding or largely exploits linguistic priors. By applying sentence-level and word-level semantic disruptions across multiple OCR/VLM architectures and downstream tasks (QA/VQA), it demonstrates that end-to-end models are highly priors-dependent, while pipeline OCR shows more robustness to semantic perturbations. Downstream reasoning degrades sharply when priors are disrupted, indicating that OCR fidelity alone is insufficient for meaningful comprehension. The study reveals a hard context-length ceiling around 8k–10k tokens for optical compression, challenging the premise of scalable long-context via visual tokens and urging prior-agnostic evaluation and architectural redesign.

Abstract

DeepSeek-OCR utilizes an optical 2D mapping approach to achieve high-ratio vision-text compression, claiming to decode text tokens exceeding ten times the input visual tokens. While this suggests a promising solution for the LLM long-context bottleneck, we investigate a critical question: "Visual merit or linguistic crutch - which drives DeepSeek-OCR's performance?" By employing sentence-level and word-level semantic corruption, we isolate the model's intrinsic OCR capabilities from its language priors. Results demonstrate that without linguistic support, DeepSeek-OCR's performance plummets from approximately 90% to 20%. Comparative benchmarking against 13 baseline models reveals that traditional pipeline OCR methods exhibit significantly higher robustness to such semantic perturbations than end-to-end methods. Furthermore, we find that lower visual token counts correlate with increased reliance on priors, exacerbating hallucination risks. Context stress testing also reveals a total model collapse around 10,000 text tokens, suggesting that current optical compression techniques may paradoxically aggravate the long-context bottleneck. This study empirically defines DeepSeek-OCR's capability boundaries and offers essential insights for future optimizations of the vision-text compression paradigm. We release all data, results and scripts used in this study at https://github.com/dududuck00/DeepSeekOCR.
Paper Structure (28 sections, 5 figures, 8 tables)

This paper contains 28 sections, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. RQ1: How Does Sentence-Level Semantic Disruption Affect DeepSeek-OCR?
  3. Experimental Setup
  4. Results and Analysis
  5. Case Study
  6. RQ2: How Does Word-Level Semantic Disruption Affect DeepSeek-OCR?
  7. Experimental Setup
  8. Results and Analysis
  9. Case Study
  10. RQ3: How Does Linguistic Prior Dependence Manifest Across Different VLM and OCR Architectures?
  11. Experimental Setup
  12. Results and Analysis
  13. Case Study
  14. RQ4: How Does DeepSeek-OCR Perform on QA and VQA Tasks?
  15. Experimental Setup
  16. ...and 13 more sections

Figures (5)

  • Figure 1: DeepSeek-OCR Model Over-reliance on Language Priors under Semantic Disruption.
  • Figure 2: VQA and QA Performance.
  • Figure 3: Compression and decompression results for different context lengths.
  • Figure 4: Case of OCR results on natural text and unsemantic text across different models.
  • Figure 5: Case of QA and VQA results across different models.