Following the Diagnostic Trace: Visual Cognition-guided Cooperative Network for Chest X-Ray Diagnosis

TL;DR

A visual cognition-guided collaborative network (VCC-Net) that employs clinically compatible interfaces and centers on visual cognition (VC) and employs clinically compatible interfaces to capture radiologists' visual search traces and attention patterns during diagnosis to achieve the cooperative diagnostic paradigm.

Abstract

Computer-aided diagnosis (CAD) has significantly advanced automated chest X-ray diagnosis but remains isolated from clinical workflows and lacks reliable decision support and interpretability. Human-AI collaboration seeks to enhance the reliability of diagnostic models by integrating the behaviors of controllable radiologists. However, the absence of interactive tools seamlessly embedded within diagnostic routines impedes collaboration, while the semantic gap between radiologists' decision-making patterns and model representations further limits clinical adoption. To overcome these limitations, we propose a visual cognition-guided collaborative network (VCC-Net) to achieve the cooperative diagnostic paradigm. VCC-Net centers on visual cognition (VC) and employs clinically compatible interfaces, such as eye-tracking or the mouse, to capture radiologists' visual search traces and attention patterns during diagnosis. VCC-Net employs VC as a spatial cognition guide, learning hierarchical visual search strategies to localize diagnostically key regions. A cognition-graph co-editing module subsequently integrates radiologist VC with model inference to construct a disease-aware graph. The module captures dependencies among anatomical regions and aligns model representations with VC-driven features, mitigating radiologist bias and facilitating complementary, transparent decision-making. Experiments on the public datasets SIIM-ACR, EGD-CXR, and self-constructed TB-Mouse dataset achieved classification accuracies of 88.40%, 85.05%, and 92.41%, respectively. The attention maps produced by VCC-Net exhibit strong concordance with radiologists' gaze distributions, demonstrating a mutual reinforcement of radiologist and model inference. The code is available at https://github.com/IPMI-NWU/VCC-Net.

Figures (10)

• Figure 1: The collaborative paradigm (right) leverages radiologists’ visual cognition to bridge radiologists' cognition and model inference. Compared with conventional CAD (left), it enhances the consistency and reliability in clinical decisions.
• Figure 2: The proposed VCC-Net comprises two main components: (1) The VAG employs GNN and CNN to model both global and local visual search patterns of radiologists, and supervises learning through three pathways. The VAG generates radiologist-like visual attention based on the input medical images. (2) The VCC leverages radiologists' VC to construct a graph structure and align visual and feature differences across regions, ultimately producing diagnostic outputs.
• Figure 3: The CGCM aligns feature and visual distances for each image node by incorporating radiologists' VC, optimizing the feature space structure. With the addition of soft visual attention. The CGCM also guides the network in constructing a graph structure concentrated on high-attention regions, establishing a robust model aligned with radiologists' cognition.
• Figure 4: Visualization of attention maps, generated with Grad-CAM, comparing different methods on the SIIM-ACR dataset. In the first column, the pneumothorax region is highlighted in yellow. The red areas in the attention maps reflect the focus regions of each network. The second column shows that VCC-Net achieves superior anomaly localization by accurately focusing on the pneumothorax region in comparison to other models.
• Figure 5: Visualization of attention maps obtained from Grad-CAM comparing methods on the TB-Mouse dataset. In the first column, regions with exudation and nodules are marked with bounding boxes. Red areas indicate the focused regions in each network's attention map. Compared to other methods, VCC-Net accurately localizes lesion areas.