Adaptive Environment-Aware Robotic Arm Reaching Based on a Bio-Inspired Neurodynamical Computational Framework

TL;DR

The paper tackles real-time, dynamic target reaching with a vision-guided robotic arm by integrating Dynamic Neural Fields (DNFs) with Stochastic Optimal Control (SOC) in a Neurodynamical Computational Framework (NeuCF). NeuCF generates adaptive reaching trajectories, supports action initiation, stopping, and target switching, and is evaluated against a cubic polynomial baseline in static, interruption, and switching scenarios. Experiments show NeuCF achieves high positional accuracy and smooth trajectories while enabling dynamic re-prioritization and robust interruption in open environments, closely matching or exceeding baseline performance. The approach demonstrates practical impact for real-time, perception-driven robotic manipulation in dynamic settings, with potential extensions to full 3D reaching.Key contributions include the neural-dynamical action regulation architecture, the integration of SOC for optimal control within a neurally plausible framework, and comprehensive benchmarking against a standard trajectory generator under changing environmental conditions.

Abstract

Bio-inspired robotic systems are capable of adaptive learning, scalable control, and efficient information processing. Enabling real-time decision-making for such systems is critical to respond to dynamic changes in the environment. We focus on dynamic target tracking in open areas using a robotic six-degree-of-freedom manipulator with a bird-eye view camera for visual feedback, and by deploying the Neurodynamical Computational Framework (NeuCF). NeuCF is a recently developed bio-inspired model for target tracking based on Dynamic Neural Fields (DNFs) and Stochastic Optimal Control (SOC) theory. It has been trained for reaching actions on a planar surface toward localized visual beacons, and it can re-target or generate stop signals on the fly based on changes in the environment (e.g., a new target has emerged, or an existing one has been removed). We evaluated our system over various target-reaching scenarios. In all experiments, NeuCF had high end-effector positional accuracy, generated smooth trajectories, and provided reduced path lengths compared with a baseline cubic polynomial trajectory generator. In all, the developed system offers a robust and dynamic-aware robotic manipulation approach that affords real-time decision-making.

Figures (6)

• Figure 1: Overall System Architecture. The system receives images to compute beacon locations. The reach planning field encodes the intended movement direction by combining inputs from disparate sources. The relative desirability value for each action policy captures its attractiveness compared to alternatives and acts as a weight to compute the final action policy.
• Figure 2: The robotic arm setup and sensing configuration. A camera provides bird-eye view feedback to the controller. The ball objects on the table serve as the main target beacons.
• Figure 3: Activity changes of the 181 neurons of the reach planning field during experiments. Each panel includes the instants the target appears (target onset), movement initiation (movement onset), and when the stop/switch cue appears (stop cue/switch cue). In case (d), only one beacon is available initially, and then we switch it with a new one, while in (e) both beacons appear initially and one beacon is then removed.
• Figure 4: Resulting trajectories given a selected stationary beacon at (27, 35) $cm$ in the $static\_1$ scenario. Panels (a) and (b) show the evolution of the x- and y-axis end-effector positions for the Polynomial and NeuCF controllers, respectively; top-down views are shown in panels (c) and (d).
• Figure 5: Generated trajectories by (a) the polynomial and (b) NeuCF controllers during a $stop$ experiment. When the green stop beacon appears, both controllers receive an interruption signal and can stop the reaching action successfully.