Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Cognitive Framework for Autonomous Agents: Toward Human-Inspired Design

Francesco Guidi, Jingfeng Shan, Mehrdad Saeidi, Enrico Testi, Elia Favarelli, Andrea Giorgetti, Davide Dardari, Alberto Zanella, Giorgio Li Pira, Francesca Starita, Anna Guerra

TL;DR

The paper addresses accelerating reinforcement learning for autonomous agents operating in uncertain, partially observable environments by integrating Pavlovian cueing with instrumental model-free and model-based learning. It introduces a hybrid architecture where radio-frequency cues act as conditioned signals to bias action selection, complemented by planning and an arbitration mechanism between subsystems. In a simulated multi-agent navigation setting, cue-driven learning achieves faster convergence and improved path efficiency compared with instrumental baselines, with further gains when model-based planning is included. The work demonstrates the potential of translating human cognitive principles into digital agents and points to future work in collective learning, arbitration refinement, and richer cue extraction from communications environments.

Abstract

This work introduces a human-inspired reinforcement learning (RL) architecture that integrates Pavlovian and instrumental processes to enhance decision-making in autonomous systems. While existing engineering solutions rely almost exclusively on instrumental learning, neuroscience shows that humans use Pavlovian associations to leverage predictive cues to bias behavior before outcomes occur. We translate this dual-system mechanism into a cue-guided RL framework in which radio-frequency (RF) stimuli act as conditioned (Pavlovian) cues that modulate action selection. The proposed architecture combines Pavlovian values with instrumental policy optimization, improving navigation efficiency and cooperative behavior in unknown, partially observable environments. Simulation results demonstrate that cue-driven agents adapt faster, achieving superior performance compared to traditional instrumental-solo agents. This work highlights the potential of human learning principles to reshape digital agents intelligence.

A Cognitive Framework for Autonomous Agents: Toward Human-Inspired Design

TL;DR

The paper addresses accelerating reinforcement learning for autonomous agents operating in uncertain, partially observable environments by integrating Pavlovian cueing with instrumental model-free and model-based learning. It introduces a hybrid architecture where radio-frequency cues act as conditioned signals to bias action selection, complemented by planning and an arbitration mechanism between subsystems. In a simulated multi-agent navigation setting, cue-driven learning achieves faster convergence and improved path efficiency compared with instrumental baselines, with further gains when model-based planning is included. The work demonstrates the potential of translating human cognitive principles into digital agents and points to future work in collective learning, arbitration refinement, and richer cue extraction from communications environments.

Abstract

This work introduces a human-inspired reinforcement learning (RL) architecture that integrates Pavlovian and instrumental processes to enhance decision-making in autonomous systems. While existing engineering solutions rely almost exclusively on instrumental learning, neuroscience shows that humans use Pavlovian associations to leverage predictive cues to bias behavior before outcomes occur. We translate this dual-system mechanism into a cue-guided RL framework in which radio-frequency (RF) stimuli act as conditioned (Pavlovian) cues that modulate action selection. The proposed architecture combines Pavlovian values with instrumental policy optimization, improving navigation efficiency and cooperative behavior in unknown, partially observable environments. Simulation results demonstrate that cue-driven agents adapt faster, achieving superior performance compared to traditional instrumental-solo agents. This work highlights the potential of human learning principles to reshape digital agents intelligence.
Paper Structure (17 sections, 6 equations, 5 figures, 3 tables)

This paper contains 17 sections, 6 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Pavlovian Learning in Humans
  3. Pavlovian Conditioning
  4. Classes of Pavlovian Responses
  5. Learning in Autonomous Agents
  6. Model-Based Learning
  7. Model-Free Learning
  8. Temporal-Difference Learning
  9. SARSA and $Q$-learning
  10. Deep $Q$-Learning
  11. Toward Human-Inspired Learning
  12. A Novel Human-Inspired Architecture for Autonomous Agents
  13. Simulation Results
  14. Simulated Scenario
  15. Learning Parameters and Reward Structure
  16. ...and 2 more sections

Figures (5)

  • Figure 1: Proposed model-based and -free learning architecture inspired by lee2019decision. The brain images illustrate neuroscientific correlates of the proposed mechanisms, adapted from o2015structure: model-free learning is associated with neural activity correlating with reward prediction errors in the ventral striatum (shown in green), whereas model-based learning shows activity in posterior parietal cortex and dorsolateral prefrontal cortex correlating with state prediction errors (shown in orange).
  • Figure 2: Simulated scenario.
  • Figure 3: Learning rate expressed as the number of steps required to accomplish the mission per episode.
  • Figure 4: Pavlovian values at the end of episode 1 (top) and 400 (bottom), for agents #1 (left) to #4 (right), respectively.
  • Figure 5: Final trajectories for the instrumental-only (left) and Pavlovian (right) agents.