Quick and Accurate Affordance Learning

TL;DR

The paper tackles how agents can efficiently learn environmental affordances through active exploration by coupling an allocentric cognitive map with local affordance learning inside a deep architecture. An end-to-end framework combines a fixed look-up map, a local affordance encoder, and a transition predictor to forecast positional changes, trained via negative log-likelihood. The study compares three uncertainty-guided exploration signals—aleatoric mean, epistemic SD, and Jensen-Shannon divergence across ensemble predictions—finding Jensen-Shannon divergence offers the most balanced and data-efficient learning. Key findings show that encoding affordances locally, coordinating navigation with local motor control, and leveraging density-based ensemble disagreements yield faster, more robust knowledge gain, with implications for robotic curricula and insights into developmental learning processes.

Abstract

Infants learn actively in their environments, shaping their own learning curricula. They learn about their environments' affordances, that is, how local circumstances determine how their behavior can affect the environment. Here we model this type of behavior by means of a deep learning architecture. The architecture mediates between global cognitive map exploration and local affordance learning. Inference processes actively move the simulated agent towards regions where they expect affordance-related knowledge gain. We contrast three measures of uncertainty to guide this exploration: predicted uncertainty of a model, standard deviation between the means of several models (SD), and the Jensen-Shannon Divergence (JSD) between several models. We show that the first measure gets fooled by aleatoric uncertainty inherent in the environment, while the two other measures focus learning on epistemic uncertainty. JSD exhibits the most balanced exploration strategy. From a computational perspective, our model suggests three key ingredients for coordinating the active generation of learning curricula: (1) Navigation behavior needs to be coordinated with local motor behavior for enabling active affordance learning. (2) Affordances need to be encoded locally for acquiring generalized knowledge. (3) Effective active affordance learning mechanisms should use density comparison techniques for estimating expected knowledge gain. Future work may seek collaborations with developmental psychology to model active play in children in more realistic scenarios.

Figures (7)

• Figure 1: The overall architecture. The look-up map $\omega$ provides visual representations $v_t$ of the environment at positions $p_t$. The affordance model $a_M$ translates these representations into context codes $c_t$, which are utilized by the transition model $t_M$ for the generation of predictions in the form of expected positional changes $(\mu_{\Delta \tilde{p}^{t+1}}, \sigma_{\Delta \tilde{p}^{t+1}})$. During training, the negative log-likelihood loss between predicted and observed $\Delta p^{t+1}$ observations is backpropagated to $t_M$ (red arrows) and further to $a_M$ (orange arrows), training both subcomponents end-to-end. During control, potential behavioral interactions are evaluated via a reward function that combines estimates of epistemic knowledge gain with estimates of goal state proximity. In this paper we fully focus on affordance learning and thus on epistemic knowledge gain.
• Figure 2: Environments used in our experiments. A small circular agent (black) navigates its environment using four diagonally attached rocket jets (orange). Fog fields are depicted in gray, obstacles in black. Force fields accelerate the agent to the right and left in green and yellow, respectively; (a) depicts the environment used during training and (b) depicts the environment used during validation. When focusing on affordance learning, we evaluate the model's performance only while the agent is within the red rectangle.
• Figure 3: Losses for globally vs locally informative sensory information aggregated over $5$ seeds. Shaded areas indicate standard deviations. Solid lines represent validation losses, dashed lines show training losses. The agent performs significantly better in the validation environment if equipped with distance sensors that are limited in range. With distance sensors that are not limited, slight overfitting is observed.
• Figure 4: Affordance maps of an environment with four obstacles generated by a model with (a) globally informative sensory information vs (b) locally informative sensory information. The maps are produced by feeding visual representations of the environment at regularly distributed positions into the affordance model and mapping the produced context code onto RGB space via principal component analysis. True affordance maps, i.e., mappings from perceptual information to local behavioral constraints, only emerge in the latter case. Note how the obstacle's edges present the same local constraints as the borders, thus the matching borders show identical colors.
• Figure 5: Boxplots of the velocities the agent exhibits during training with the different exploration mechanisms, taken over all epochs across the entire environment. No uncertainty estimate produces velocities as high as the random heuristic.