PEARL: Self-Evolving Assistant for Time Management with Reinforcement Learning

TL;DR

This paper addresses calendar conflict resolution as a long-horizon, preference-driven task and introduces CalConflictBench to benchmark LLM-based agents on evolving user preferences. It shows current LLMs struggle to adapt over time and that increasing horizon or context length yields limited gains. To bridge this gap, it proposes PEARL, a reinforcement-learning framework that uses an external memory Strategy Hub and curriculum-based round-wise rewards to progressively infer and apply user preferences, achieving notable improvements over strong baselines. The work provides a principled evaluation setup and demonstrates that memory-augmented, preference-aware RL can meaningfully enhance time-management decisions in sequential calendar conflicts, with potential for scalable, personalized scheduling applications.

Abstract

Overlapping calendar invitations force busy professionals to repeatedly decide which meetings to attend, reschedule, or decline. We refer to this preference-driven decision process as calendar conflict resolution. Automating such process is crucial yet challenging. Scheduling logistics drain hours, and human delegation often fails at scale, which motivate we to ask: Can we trust large language model (LLM) or language agent to manager time? To enable systematic study of this question, we introduce CalConflictBench, a benchmark for long-horizon calendar conflict resolution. Conflicts are presented sequentially and agents receive feedback after each round, requiring them to infer and adapt to user preferences progressively. Our experiments show that current LLM agents perform poorly with high error rates, e.g., Qwen-3-30B-Think has 35% average error rate. To address this gap, we propose PEARL, a reinforcement-learning framework that augments language agent with an external memory module and optimized round-wise reward design, enabling agent to progressively infer and adapt to user preferences on-the-fly. Experiments on CalConflictBench shows that PEARL achieves 0.76 error reduction rate, and 55% improvement in average error rate compared to the strongest baseline.

Figures (7)

• Figure 1: Illustration of the proposed calendar conflict resolution task. At decision round $t$, the agent observes (i) the conflicting events $\mathcal{E}_t$, (ii) contextual information, and (iii) the current calendar state $\mathcal{C}_t$. The agent selects exactly one event to accept ($a_t^i=1$) and declines the rest ($a_t^i=0$), producing the accepted event, declined events, a priority ranking, and rationale.
• Figure 2: Average Error Rate of Qwen3-8b under different the number of conflicting events per round ($M$) (left), and Error Reduction Rate of Qwen3-8B under different evaluation parameters (right).
• Figure 3: Average Optimal Rank Distance (ORD) over different numbers of decision rounds ($N$).
• Figure 4: Overview of PEARL.Top-left: Agent action space. At each turn, the agent can take a decision action$a_{\text{decision}}$ (accept/decline an event $e_i$) or a hub action$a_{\text{hub}}$ that queries (list) or updates (update) the external Strategy Hub. Top-right: Agent rollout. The policy model generates a multi-turn trajectory; when a decision action is emitted, the round terminates and the next conflict is presented. Bottom: Training with round-wise rewarding. For each round, we sample multiple completions, score them with the curriculum-based reward model, and aggregate rewards into group-wise advantages by each round to update the policy.
• Figure 5: Error vs. decision rounds of PEARL and zero-shot baseline