Time-Continuous Modeling for Temporal Affective Pattern Recognition in LLMs

TL;DR

This work targets the limitation of discrete, static emotion modeling in language models by proposing a time-continuous framework that steers generation along affective trajectories. It combines CTRNNs, Neural ODEs, and physics-informed regularization with In-Context Vectors to produce emotionally coherent dialogue, anchored by the CEmoFlow dataset and continuous interpolation techniques. The contributions include a continuous affective dataset pipeline, cyclic time encoding, a magnitude-based measure of emotional shift, and a continuous interpolation approach that enables PINN-guided emotion dynamics in text generation. The approach aims to yield more natural, interpretable, and user-aligned dialogue with potential benefits for affect-aware AI, digital companions, and mental health applications, while acknowledging significant computational and engineering challenges.

Abstract

This paper introduces a dataset and conceptual framework for LLMs to mimic real world emotional dynamics through time and in-context learning leveraging physics-informed neural network, opening a possibility for interpretable dialogue modeling.

Figures (4)

• Figure 1: Let $E$ of both functions $f = encoder_{out(E_{0}^{N})}$ presented in discrete fashion and $f^* = encoder \rightarrow LatentODE_{out({E}_{0}^{N})}$ in continuous fashion are output probabilities from categories of emotions $N$. The evolving emotions $E$ are interdependent on its own due to softmax constraint, however $f^*$ evolutions are determined by $s$ as delay in seconds that is directly affected by learned differential slopes $\frac{dh}{dt}(f^*(t_{-n}))$ solved with $DOPRI5$DORMAND198019 during forward pass, further serving as a mathematical form of affective sensing in general context.
• Figure 2: $optim:Lion,lr : 5e-6, lr_{scheduler}:cosine, \ batch : 32, \ \omega \ decay : 1e-2 \ \forall{\theta}, \ bf16 : 1$. The early stopping is defined as $\forall t \in [T - 5, T] \quad \left| \mathcal{L}(t) - \mathcal{L}_{\text{min}} \right| \leq 1e-2$ noting that $5$ were steps $\neq$ epochs.
• Figure 3: $surprise$ label indicates the least prediction of all due to class imbalances from the dataset saravia-etal-2018-carer, however we've mitigated with generative paraphrasing augmentation via $Qwen2.5$qwen2025qwen25technicalreport for each label $\{love,anger,fear,surprise\}$, resulted in $\texttt{2x}$ more examples but still imbalanced.
• Figure 4: Gaussian KDE with $250.000$ samples where $hh_{sin, cos}$ transforms to $arctan2(hh_{sin, cos})$ that represents each soft labels distribution on $x \in \{\text{Ante Meridiem},\text{Post Meridiem}\}$ axis following their intensity peak by the contour on $y \in \forall e_N \in [0,1]$ axis, the yellow line represents $\mu \Delta \in [0,1]$ (normalized), whereas $\bigtriangleup$ marks highest distribution and $\bigtriangledown$ marks lowest distribution density of the contour.