U-LAG: Uncertainty-Aware, Lag-Adaptive Goal Retargeting for Robotic Manipulation

TL;DR

U-LAG introduces a mid-execution, uncertainty-aware goal retargeting layer that sits between perception and a fixed Cartesian controller to handle late/noisy observations. It implements four retargeters (Nearest, ICP, UAR, UAR--PF) and reliability wrappers, with UAR--PF delivering the strongest resilience to perception lag and abrupt in-plane shifts across pick, push, stacking, and peg insertion. A Shift×Lag benchmark in PyBullet/PandaGym shows substantial gains over a no-retarget baseline, with UAR--PF maintaining high success rates (e.g., up to 0.86 in harsh conditions) while keeping end-effector travel comparable to other methods. The work demonstrates that separating goal retargeting from low-level control yields robust, task-agnostic manipulation under delay, suggesting practical usefulness for real-world robotic systems and hardware deployment with potential for learned uncertainty models and broader manipulation domains.

Abstract

Robots manipulating in changing environments must act on percepts that are late, noisy, or stale. We present U-LAG, a mid-execution goal-retargeting layer that leaves the low-level controller unchanged while re-aiming task goals (pre-contact, contact, post) as new observations arrive. Unlike motion retargeting or generic visual servoing, U-LAG treats in-flight goal re-aiming as a first-class, pluggable module between perception and control. Our main technical contribution is UAR-PF, an uncertainty-aware retargeter that maintains a distribution over object pose under sensing lag and selects goals that maximize expected progress. We instantiate a reproducible Shift x Lag stress test in PyBullet/PandaGym for pick, push, stacking, and peg insertion, where the object undergoes abrupt in-plane shifts while synthetic perception lag is injected during approach. Across 0-10 cm shifts and 0-400 ms lags, UAR-PF and ICP degrade gracefully relative to a no-retarget baseline, achieving higher success with modest end-effector travel and fewer aborts; simple operational safeguards further improve stability. Contributions: (1) UAR-PF for lag-adaptive, uncertainty-aware goal retargeting; (2) a pluggable retargeting interface; and (3) a reproducible Shift x Lag benchmark with evaluation on pick, push, stacking, and peg insertion.

Figures (5)

• Figure 1: U-LAG in action under Shift$\times$Lag. Snapshots for pick, push, stacking, and peg insertion at $(r{=}8\,\mathrm{cm},\,L{=}300\,\mathrm{ms})$. Dashed orange: stale target; green: updated goal; purple ellipse: contact region.
• Figure 2: U-LAG system overview. Left: perception & sensing. Center: retargeters (Nearest/ICP/UAR/UAR--PF), shared guards and a task-specific waypoint synthesizer. Right: unchanged Cartesian servo & gripper FSM executing waypoints.
• Figure 3: All tasks: success over Shift$\times$Lag (50 seeds/cell). Columns are baselines (No-retarget, Nearest, ICP, UAR, UAR--PF). Common color scale across all panels.
• Figure 4: Retarget latency vs. injected $L$. Means with std bars. Push tracks $L$ closely for all modes; on pick, ICP shows added overhead at high $L$.
• Figure 5: Peg insertion: XY error vs. injected lag. Mean lateral error (cm) averaged over shifts.