Weather-R1: Logically Consistent Reinforcement Fine-Tuning for Multimodal Reasoning in Meteorology

TL;DR

Problem: domain gap and faithfulness gaps hamper high-stakes meteorological reasoning with Vision-Language Models. Approach: develop WeatherQA, a multimodal meteorology benchmark, and LoCo-RFT to enforce logical consistency, culminating in Weather-R1, a logically faithful reasoning VLM; evaluate on WeatherQA and ScienceQA. Key findings: Weather-R1-7B achieves 52.9% on WeatherQA (9.8-point gain) and 86.46% on ScienceQA, while significantly reducing Self-Contra compared to RFT; ablations confirm robustness to judge model choices and reward settings. Significance: provides a trustworthy, interpretable framework for domain-specific multimodal reasoning in meteorology and lays groundwork for extending logical faithfulness to open-ended generation tasks.

Abstract

While Vision Language Models (VLMs) show advancing reasoning capabilities, their application in meteorology is constrained by a domain gap and a reasoning faithfulness gap. Specifically, mainstream Reinforcement Fine-Tuning (RFT) can induce Self-Contradictory Reasoning (Self-Contra), where the model's reasoning contradicts its final answer, which is unacceptable in such a high-stakes domain. To address these challenges, we construct WeatherQA, a novel multimodal reasoning benchmark in meteorology. We also propose Logically Consistent Reinforcement Fine-Tuning (LoCo-RFT), which resolves Self-Contra by introducing a logical consistency reward. Furthermore, we introduce Weather-R1, the first reasoning VLM with logical faithfulness in meteorology, to the best of our knowledge. Experiments demonstrate that Weather-R1 improves performance on WeatherQA by 9.8 percentage points over the baseline, outperforming Supervised Fine-Tuning and RFT, and even surpassing the original Qwen2.5-VL-32B. These results highlight the effectiveness of our LoCo-RFT and the superiority of Weather-R1. Our benchmark and code are available at https://github.com/Marcowky/Weather-R1.

Figures (6)

• Figure 1: Data sample of our WeatherQA. The seven imaging modalities are: 24-hour cumulative precipitation map (Rain), FY-2G satellite infrared cloud image (Phenom), daily maximum temperature map (Max Temp), daily minimum temperature map (Min Temp), 500hPa geopotential height and wind field map (500hPa), 850hPa wind field map (850hPa), and sea level pressure map (Land). These correspond to four themes: precipitation, weather phenomena, temperature, and weather systems, respectively.
• Figure 2:
• Figure 3:
• Figure 5: The LoCo-RFT paradigm. We introduce an additional LLM-assisted logical consistency reward, $R_{LoCo}$, to suppress the Self-Contra phenomenon.
• Figure 6: Detailed cross-task performance on WeatherQA. For each model, rows represent the training task, and columns represent the testing task. The value in each cell denotes the percentage change in performance compared to the Qwen2.5-VL-7B baseline. Green signifies improvement, while red signifies a decline.