Buzz, Choose, Forget: A Meta-Bandit Framework for Bee-Like Decision Making

TL;DR

The paper addresses how to imitate and forecast heterogeneous bee decision strategies under memory constraints. It introduces MAYA, a meta-bandit imitation framework that maps bee trajectories to a mixture of two-armed bandits using a sliding memory window $\tau$ and trajectory-based similarity metrics (KL, Wasserstein, DTW). The authors provide both theoretical bounds on regret gaps and an extensive empirical evaluation across five datasets and a complementary Mice dataset, showing that MAYA, particularly MAYA-Wass, outperforms standard IRL baselines and GLMs while yielding interpretable per-bee policy insights. The work enables biologically plausible, memory-aware simulations for ecological governance and advances understanding of how memory and context shape pollinator learning.

Abstract

We introduce a sequential reinforcement learning framework for imitation learning designed to model heterogeneous cognitive strategies in pollinators. Focusing on honeybees, our approach leverages trajectory similarity to capture and forecast behavior across individuals that rely on distinct strategies: some exploiting numerical cues, others drawing on memory, or being influenced by environmental factors such as weather. Through empirical evaluation, we show that state-of-the-art imitation learning methods often fail in this setting: when expert policies shift across memory windows or deviate from optimality, these models overlook both fast and slow learning behaviors and cannot faithfully reproduce key decision patterns. Moreover, they offer limited interpretability, hindering biological insight. Our contribution addresses these challenges by (i) introducing a model that minimizes predictive loss while identifying the effective memory horizon most consistent with behavioral data, and (ii) ensuring full interpretability to enable biologists to analyze underlying decision-making strategies and finally (iii) providing a mathematical framework linking bee policy search with bandit formulations under varying exploration-exploitation dynamics, and releasing a novel dataset of 80 tracked bees observed under diverse weather conditions. This benchmark facilitates research on pollinator cognition and supports ecological governance by improving simulations of insect behavior in agroecosystems. Our findings shed new light on the learning strategies and memory interplay shaping pollinator decision-making.

Figures (59)

• Figure 1: MAYA (Multi-Agent Y-maze Allocation) is an imitation learning framework for policy selection via windowed regret matching. Leveraging logged bee trajectories and three similarity metrics (Wasserstein, KL, DTW), MAYA maps learning dynamics onto 2-armed bandit strategies (UCB, Epsilon-greedy, LinUCB, Uniform). Beyond performance alignment, MAYA provides interpretability of bee behaviors by revealing differences in memory span and learning aptitude, thereby distinguishing “good learners” from “poor learners” in contextual experiments.
• Figure 2: Comparative study of the best window size $\tau$ by average MSE and MAE; weather and location for each dataset are provided. The maximum window value corresponds to using the full sequence (i.e., no window). $\star$ symbol refers as best performance according standard deviation and average reward (see Tab\ref{['tab:5']}. in App \ref{['tab_mae_mse_maya']} for the full results)
• Figure 3: For bee 1 (fast learner, low regret) and bee 15 (slow learner, high regret) from dataset 2. Left: Cumulative regret of 4 MABs, GLM and MAYA ($\tau=7$) for two bees from Dataset 2. Right: Choice interpretability with MAYA–Wass ($\tau=7$).
• Figure 4: Bees trajectories
• Figure 5: MAYA-Wass