Interactive Inference: A Neuromorphic Theory of Human-Computer Interaction

TL;DR

This work introduces Interactive Inference as a neuromorphic, information-theoretic framework for Human-Computer Interaction that treats tasks as reductions in Bayesian surprise between a goal prior and a progress posterior. By linking capacity to process Bayesian surprise with a logarithmic dependence on the signal-to-noise ratio (SNR) and mapping Bayesian surprise to KL divergence, the authors derive and unify classic HCI laws (Hick’s, Fitts’, and the Power Law) within a single model. They validate the approach empirically via a car-following driving task, showing that driver information processing capacity scales with log(SNR) and that error emerges as squared SNR beyond capacity, with capacity and error well described in bits. The findings suggest a practical tool for real-time cognitive-load assessment and design optimization, while outlining a research agenda to test and extend the framework across tasks and modalities.

Abstract

Neuromorphic Human-Computer Interaction (HCI) is a theoretical approach to designing better user experiences (UX) motivated by advances in the understanding of the neurophysiology of the brain. Inspired by the neuroscientific theory of Active Inference, Interactive Inference is a first example of such approach. It offers a simplified interpretation of Active Inference that allows designers to more readily apply this theory to design and evaluation. In Interactive Inference, user behaviour is modeled as Bayesian inference on progress and goal distributions that predicts the next action. We show how the error between goal and progress distributions, or Bayesian surprise, can be modeled as a simple mean square error of the signal-to-noise ratio (SNR) of a task. The problem is that the user's capacity to process Bayesian surprise follows the logarithm of this SNR. This means errors rise quickly once average capacity is exceeded. Our model allows the quantitative analysis of performance and error using one framework that can provide real-time estimates of the mental load in users that needs to be minimized by design. We show how three basic laws of HCI, Hick's Law, Fitts' Law and the Power Law can be expressed using our model. We then test the validity of the model by empirically measuring how well it predicts human performance and error in a car following task. Results suggest that driver processing capacity indeed is a logarithmic function of the SNR of the distance to a lead car. This result provides initial evidence that Interactive Interference can be useful as a new theoretical design tool.

Figures (7)

• Figure 1: Bayes Theorem. Distributions of the Likelihood P(o|s) of observations, in green, given a Prior P(s) that describes the current state, in blue. In black is the calculated Posterior P(s|o) that describes the chance of the state given the observation.
• Figure 2: Bayesian Fitts' Law. Distribution of the Posterior probability P(s|o) of the cursor being within the Goal, from top to bottom: in the middle of a trial, at 2/3s of the trial and at the end of a trial. Goal is the 96% area of the Prior probability distribution P(s), on both sides for a reciprocal trial. Note how the velocity of the cursor tracks the uncertainty in cursor position.
• Figure 3: Two cars with normal distributions $G(x)$ and $P(x)$ representing uncertainty in their position. The percentage of overlap between distributions equals the probability of a collision. The negative logarithm of this probability gives the number of bits of difference.
• Figure 4: Participant's experimental setup. The white bar in the center of the screen helped participants perceive the correct distance $S$. They needed to match the size of this bar with the lead car's bumper while driving. Here, it is wider than the lead car's bumper indicating that the participant needs to accelerate.
• Figure 5: Driver information capacity plotted against noise-to-signal ratio ($\Delta S/\overline{S}$, index of difficulty (ID) on a logarithmic scale below x-axis). Capacity followed a logarithmic function (blue solid), with a slope of 0.25 bits (green dash). Black solid line indicates remaining capacity.