Automated scientific minimization of regret

TL;DR

The utility of automated scientific minimization of regret (ASMR) is demonstrated in a multi-attribute decision-making task, showing that ASMR discovers cognitive models that predict human behavior at noise ceiling while retaining interpretability.

Abstract

We introduce automated scientific minimization of regret (ASMR) -- a framework for automated computational cognitive science. Building on the principles of scientific regret minimization, ASMR leverages Centaur -- a recently proposed foundation model of human cognition -- to identify gaps in an interpretable cognitive model. These gaps are then addressed through automated revisions generated by a language-based reasoning model. We demonstrate the utility of this approach in a multi-attribute decision-making task, showing that ASMR discovers cognitive models that predict human behavior at noise ceiling while retaining interpretability. Taken together, our results highlight the potential of ASMR to automate core components of the cognitive modeling pipeline.

Figures (2)

• Figure 1: ASMR pipeline. Centaur is used as a guide to identify gaps in an interpretable cognitive model, resulting in a set of data points that are, in principle, predictable but are not currently accounted for. These points, together with the model's code and a brief instruction, are then provided to a language-based reasoning model. The reasoning model generates modifications to the cognitive model -- a process which can be iterated multiple times.
• Figure 2: Summary of results. a, Improvement of aggregated AIC scores across iterations of ASMR. The solid line show the average AIC score, while the shaded areas represent the worst and best model at a given iteration. b, AIC scores from the models at the first and last iteration for each individual participant. c, Python code for one of the discovered models with the lowest AIC score.