Solving the decision-making differential equations from eye fixation data in Unity software by using Hermite Long-Short-Term Memory neural network

TL;DR

The paper addresses how to quantify decision-making processes from eye fixation data collected in a Unity-based VR environment by deriving a coupled differential-equation model with fixation-rate $\lambda$ and separation-rate $\mu$ that governs fixation dynamics among other state variables. It then introduces a novel Hermite-activated Long-Short Term Memory (Hermite-LSTM) framework to solve the resulting system, leveraging Hermite functions as activations to improve the modeling of time-series data. A data pipeline links eye-tracking outputs with Unity-derived events to form a learn matrix that feeds the model, and results show a close match between predicted and actual trajectories for $N(t)$, $V(t)$, $D(t)$, and $G(t)$ over a 0–10 time horizon. The work demonstrates a pathway for predictive cognitive modeling in VR settings, with implications for understanding attention, decision quality, and user engagement in immersive interfaces. The approach combines domain-specific data extraction, differential-equation modeling, and advanced neural architectures to yield actionable insights for cognitive science and VR applications.

Abstract

Cognitive decision-making processes are crucial aspects of human behavior, influencing various personal and professional domains. This research delves into the application of differential equations in analyzing decision-making accuracy by leveraging eye-tracking data within a virtual industrial town setting. The study unveils a systematic approach to transforming raw data into a differential equation, essential for deciphering the relationship between eye movements during decision-making processes. Mathematical relationship extraction and variable-parameter definition pave the way for deriving a differential equation that encapsulates the growth of fixations on characters. The key factors in this equation encompass the fixation rate $(λ)$ and separation rate $(μ)$, reflecting user interaction dynamics and their impact on decision-making complexities tied to user engagement with virtual characters. For a comprehensive grasp of decision dynamics, solving this differential equation requires initial fixation counts, fixation rate, and separation rate. The formulation of differential equations incorporates various considerations such as engagement duration, character-player distance, relative speed, and character attributes, enabling the representation of fixation changes, speed dynamics, distance variations, and the effects of character attributes. This comprehensive analysis not only enhances our comprehension of decision-making processes but also provides a foundational framework for predictive modeling and data-driven insights for future research and applications in cognitive science and virtual reality environments.

Figures (8)

• Figure 1: The Industrial Town
• Figure 2: An example of a character used in Unitybib31
• Figure 3: LSTM neural network
• Figure 4: LSTM Model Architecture
• Figure 5: LSTM Model Architecture