MGA-VQA: Secure and Interpretable Graph-Augmented Visual Question Answering with Memory-Guided Protection Against Unauthorized Knowledge Use

TL;DR

The paper tackles the DocVQA challenge of jointly understanding text, layout, and visuals, especially under high-resolution and structured layouts, by proposing MGA-VQA. This unified framework combines token-level visual encoding, explicit spatial graphs, memory-augmented reasoning, and question-guided compression to enable interpretable, multi-hop inference and efficient processing of dense documents. It introduces a graph-based layout model with memory traces and a disentangled cross-modal fusion mechanism to produce accurate answers and precise localization while maintaining transparency of reasoning through auditable components. Across six diverse benchmarks, MGA-VQA achieves state-of-the-art ANLS scores and improved IoU localization, demonstrating that accuracy, efficiency, and interpretability can be jointly optimized for robust, trustworthy document understanding.

Abstract

Document Visual Question Answering (DocVQA) requires models to jointly understand textual semantics, spatial layout, and visual features. Current methods struggle with explicit spatial relationship modeling, inefficiency with high-resolution documents, multi-hop reasoning, and limited interpretability. We propose MGA-VQA, a multi-modal framework that integrates token-level encoding, spatial graph reasoning, memory-augmented inference, and question-guided compression. Unlike prior black-box models, MGA-VQA introduces interpretable graph-based decision pathways and structured memory access for enhanced reasoning transparency. Evaluation across six benchmarks (FUNSD, CORD, SROIE, DocVQA, STE-VQA, and RICO) demonstrates superior accuracy and efficiency, with consistent improvements in both answer prediction and spatial localization.

Figures (1)

• Figure 1: MGA-VQA Architecture. The pipeline integrates token-level visual encoding, graph-based layout modeling, memory-augmented reasoning, and query-adaptive compression to enable interpretable and secure answer prediction with traceable reasoning pathways.