Mapping Smarter, Not Harder: A Test-Time Reinforcement Learning Agent That Improves Without Labels or Model Updates

TL;DR

The paper tackles schema mapping for enterprise logs when vendor documentation is incomplete or unavailable, formulating the problem as mapping from $S$ to $T \cup \{\emptyset\}$ and optimizing a proxy objective via a test-time reinforcement learning agent. The core approach iteratively collects external evidence, enriches the context, and applies a confidence-based reward $r_t = C_{t+1} - C_t$ to refine mappings without any model weight updates. Across a Defender-for-Endpoint case study, the method elevates accuracy from 72.73% to 93.94% over 100 iterations and dramatically reduces low-confidence mappings requiring expert review by 85%. While results show strong practical benefits and transparency through reasoning traces, the work also reveals overconfidence in some iterations and emphasizes calibration and computation trade-offs for broader deployment. Overall, this evidence-driven, test-time learning framework offers a scalable, accountable path to robust schema mapping in real-world, label-scarce environments.

Abstract

The Enterprise Intelligence Platform must integrate logs from numerous third-party vendors in order to perform various downstream tasks. However, vendor documentation is often unavailable at test time. It is either misplaced, mismatched, poorly formatted, or incomplete, which makes schema mapping challenging. We introduce a reinforcement learning agent that can self-improve without labeled examples or model weight updates. During inference, the agent: 1) Identifies ambiguous field-mapping attempts. 2) Generates targeted web-search queries to gather external evidence. 3) Applies a confidence-based reward to iteratively refine its mappings. To demonstrate this concept, we converted Microsoft Defender for Endpoint logs into a common schema. Our method increased mapping accuracy from 56.4\%(LLM-only) to 72.73\%(RAG) to 93.94\% over 100 iterations using GPT-4o. At the same time, it reduced the number of low-confidence mappings requiring expert review by 85\%. This new approach provides an evidence-driven, transparent method for solving future industry problems, paving the way for more robust, accountable, scalable, efficient, flexible, adaptable, and collaborative solutions.

Figures (3)

• Figure 1: Position of our test-time RL agent relative to prior work in the schema matching pipeline. In Section A, Logparser-LLM zhong2024logparser handles raw-to-structured parsing. In section B, Schema-Matching-LLM parciak2024schema shows baseline one-shot mapping capability. In section C, ReMatch sheetrit2024rematch adds retrieval when full documentation exists. MatchMaker seedat2024matchmaker pre-embeds target schemas for reuse. In section D, Self-consistency wang2022self improves chain-of-thought reasoning. Search-R1 jin2025search enhances LLM reasoning with search-augmented RL. This example shows that the log LocalPort is ambiguous for decision making; we need more context to determine whether this field should map to src or dst. Our research uniquely extends rightward beyond traditional enterprise knowledge bases to handle newly seen logs with ill-formatted or incomplete documentation. Unlike fine-tuning approaches that require labeled data and risk overfitting, our agent operates without test-time labels, conducting internet searches to gather evidence outside the enterprise KB scope. This addresses the critical gap where traditional methods fail on unseen vendor schemas with insufficient documentation.
• Figure 2: Comprehensive performance analysis of the reinforcement learning agent over 100 iterations. Top: Accuracy improves from 72.73% to 93.94%, while confidence trends upward and approaches 1.0, highlighting a persistent overconfidence gap. Middle: Accuracy variability is high in early iterations but stabilizes over time. Bottom: Conflict field count decreases from 26 to 4(85% reduction), demonstrating reduced ambiguity as evidence accumulates.
• Figure 3: The relationship between confidence and accuracy. Path 1 shows the starting point. Path 2 illustrates how confidence as a proxy reward improves accuracy. Path 3 highlights the challenge of overconfidence, where confidence saturates while accuracy lags behind. Therefore, more engineering and research efforts are needed to bring the confidence curve down. The diagonal line indicates perfect calibration, and the curves above it show overconfidence.