GuardTrace-VL: Detecting Unsafe Multimodel Reasoning via Iterative Safety Supervision

TL;DR

GuardTrace-VL introduces a vision-aware safety detector that monitors the complete Question–Thinking–Answer (QTA) reasoning trajectory in multimodal models, addressing safety gaps in intermediate reasoning. It builds GuardTrace, a multimodal QTA safety dataset, via multimodal expansion, full QTA generation, and human–AI collaborative annotation, yielding about 11.8K QTA examples for training. A three-stage curriculum—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Oracle-Guided Refined DPO (OGDPO)—progressively aligns the detector with nuanced safety preferences, achieving state-of-the-art results on in-domain and OOD benchmarks, with an average F1 of around 93% on GuardTrace-Test. The work demonstrates that joint multimodal QTA analysis is essential for robust safety moderation, and provides a scalable path toward integrating trajectory-level safety into alignment pipelines for future multimodal reasoning systems.

Abstract

Multimodal large reasoning models (MLRMs) are increasingly deployed for vision-language tasks that produce explicit intermediate rationales. However, reasoning traces can contain unsafe content even when the final answer is non-harmful, creating deployment risks. Existing multimodal safety guards primarily evaluate only the input question and the final answer, neglecting the intermediate reasoning process. This oversight allows undetected harm, such as biased inferences or policy-violating use of visual context, to emerge during reasoning. We introduce GuardTrace-VL, a vision-aware safety auditor that monitors the full Question-Thinking-Answer (QTA) pipeline via joint image-text analysis, enabling detection of unsafe content as it emerges in the reasoning stage. To support training and evaluation, we construct the GuardTrace dataset, which is generated through diverse prompting strategies and refined via a MLRM- and human-based voting and verification pipeline. Furthermore, we propose a three-stage progressive training scheme combined with the data refinement process, enabling the model to learn nuanced and context-dependent safety preferences according to different risk levels. On our proposed test set covering both in-domain and out-of-domain scenarios, GuardTrace-VL model achieves an F1 score of 93.1% on unsafe reasoning detection tasks, representing a 13.5% improvement in F1 score compared to the previous strongest multimodal safety defense methods. The codes will be made publicly available.

Figures (8)

• Figure 1: Multimodal Question-Thinking-Answer (QTA) moderation comparison. QA Guard is distracted by safety-aligned statements in the answer, Text-only Guard lacks visual grounding and misses contextual threats, while our GuardTrace-VL jointly models the multimodal question, reasoning trace, and answer to correctly flag harmful intent, demonstrating the necessity of holistic multimodal QTA analysis for robust safety moderation.
• Figure 2: Pipeline of GuardTrace-VL. (a) Multimodal Expansion: Converts text-only queries into multimodal inputs using image generation and jailbreak methods. Blue denotes in-domain data, purple denotes OOD data used in the test set. (b) Full Q-T-A Generation: Generates complete Question-Thinking-Answer traces with multimodal inputs via MLRMs. (c) Human-AI Collaborative Annotation: Filters and labels data through AI voting and expert evaluation. (d) Three-Stage Training: Trains the model iteratively from SFT to DPO, then refines with Oracle-Guided DPO using re-labeled data.
• Figure 3: (a) Distribution of training data sources, with example image-text pairs illustrating our construction strategies. The inner ring shows the original text-only datasets used as seed sources, and the outer ring reflects the expanded multimodal composition after augmentation. (b) Safety label distribution in training and test sets.
• Figure 4: Performance comparison between our multimodal model and three text-only baseline models, which do not support image input and are therefore provided with image captions. All values are F1-scores(%).
• Figure 5: Cosine Similarity of Voting Consistency Among Three Models. X-axis and Y-axis both represent the three models: Gemma-3-27B-it (Model 1), Mistral-3.2-Small-24B-Instruct (Model 2), and Qwen2.5-VL-32B-Instruct (Model 3).