No Plan but Everything Under Control: Robustly Solving Sequential Tasks with Dynamically Composed Gradient Descent

TL;DR

The paper tackles the challenge of solving long-horizon sequential tasks without explicit planning by proposing dynamically composed gradient descent, where world regularities are encoded as interdependent components whose connections adapt to the current state. The AICON framework uses active interconnections among recursive estimators to generate a rich set of gradients, from which the steepest path guides actions via the update $a_{t+1} = a_t - k abla g(a_t)$, enabling emergent subgoals and robust behavior. It demonstrates the approach on Blocks World, achieving near-optimal performance without future-state predictions, and on a real-world drawer-opening task with uncertainty and disturbances, outperforming planning baselines. The results suggest a computationally efficient alternative to planning that aligns with biological problem-solving strategies and supports interactive perception and error recovery in dynamic environments.

Abstract

We introduce a novel gradient-based approach for solving sequential tasks by dynamically adjusting the underlying myopic potential field in response to feedback and the world's regularities. This adjustment implicitly considers subgoals encoded in these regularities, enabling the solution of long sequential tasks, as demonstrated by solving the traditional planning domain of Blocks World - without any planning. Unlike conventional planning methods, our feedback-driven approach adapts to uncertain and dynamic environments, as demonstrated by one hundred real-world trials involving drawer manipulation. These experiments highlight the robustness of our method compared to planning and show how interactive perception and error recovery naturally emerge from gradient descent without explicitly implementing them. This offers a computationally efficient alternative to planning for a variety of sequential tasks, while aligning with observations on biological problem-solving strategies.

Figures (6)

• Figure 1: By dynamically composing simpler encoded regularities within gradient descent, our approach aligns actions with both the task and the current state, leading to emergent behavior that needs no plan to have everything under control. This allows us to solve long sequential tasks like Blocks World (top) and real-world tasks under uncertainty, such as opening a drawer using only RGB and force-torque measurements (bottom).
• Figure 2: Dynamically composed gradient descent solves the classic planning domain of Blocks World (BW): (a) Gradient descent with an AICON-ic system always selects actions that advance towards the goal, identifying necessary subgoals while ignoring irrelevant blocks. (b) But a naive goal formulation can lead to suboptimal actions and necessary backtracking due to competing subgoals. (c) Actively interconnecting the goal with the current state removes competing subgoals, though some efficient shortcuts in combined problems remain challenging to find without foresight. On a set of $130$ randomly generated BW problems, we show how our approach consistently finds solutions and closely approaches optimal performance without direct state predictions.
• Figure 3: The dynamically composed gradient descent adapts the potential field depending on the current situation: (a) With initial uncertainty about the drawer’s position, the robot adjusts its viewpoint to reduce uncertainty. (b) If it looses sight of the drawer, it returns to the visible area. (c) This continues to further reduce uncertainty, continuously adapting the potential to guide to novel viewpoints. (d) Once the drawer's position is sufficiently certain, the robot approaches and then grasps it. (e) The robot then moves the drawer along its estimated kinematic axis, but we disturb the opening causing it to loose the drawer. (f) This increases uncertainty, necessitating visual re-identification. (g) After ensuring the drawer's position anew, the robot can then grasp and open the drawer along kinematic joint axis while improving its estimate of the axis parameters. We visualize the potential field by restricting the robot's motion to the xz-plane and spatially sampling the gradient across the plane post-execution. The steepest gradient is color-coded according to \ref{['fig:network']}. For a video version of both potential and the current gradient path through the network, see the https://www.tu.berlin/robotics/papers/noplan.
• Figure 4: Gradients navigate through the network of components that encodes different regularities, dynamically modulated by changing information exchanges within the active interconnections. The colored paths represent relevant gradients for the experiment shown in \ref{['fig:potentials']}.
• Figure 5: Our approach robustly resolves uncertainty and adapts to dynamic changes: If we vary the uncertainty under which the system operates (both by changing the initial prior as shown in (b) and adding sensor noise), our approach adapts its behavior to resolve uncertainty, e.g. by triangulating the drawer (a). This allows it to maintain performance under uncertainty outperforming planning approaches (c). If we instead introduce unexpected light (e) or heavy (f) disturbances, our approach can adapt to state changes and thus remains robust where planning approaches fail (d).