Apple: Toward General Active Perception via Reinforcement Learning

TL;DR

APPLE reframes active perception as a task-agnostic reinforcement learning problem that jointly trains a policy and a transformer-based perception module within a POMDP. By introducing two off-policy variants, APPLE-SAC and APPLE-CrossQ, the framework learns how to actively gather informative tactile observations across diverse downstream tasks, ranging from classification to volume estimation and pose localization. The approach demonstrates strong, robust performance without hand-crafted task-specific exploration heuristics, outperforming prior methods such as HAM and single-policy baselines on five benchmarks. This general, information-gathering paradigm has practical implications for autonomous robots operating in uncertain, tactile-rich environments, with potential extensions to multi-modal sensing and real-world deployment.

Abstract

Active perception is a fundamental skill that enables us humans to deal with uncertainty in our inherently partially observable environment. For senses such as touch, where the information is sparse and local, active perception becomes crucial. In recent years, active perception has emerged as an important research domain in robotics. However, current methods are often bound to specific tasks or make strong assumptions, which limit their generality. To address this gap, this work introduces APPLE (Active Perception Policy Learning) - a novel framework that leverages reinforcement learning (RL) to address a range of different active perception problems. APPLE jointly trains a transformer-based perception module and decision-making policy with a unified optimization objective, learning how to actively gather information. By design, APPLE is not limited to a specific task and can, in principle, be applied to a wide range of active perception problems. We evaluate two variants of APPLE across different tasks, including tactile exploration problems from the Tactile MNIST benchmark. Experiments demonstrate the efficacy of APPLE, achieving high accuracies on both regression and classification tasks. These findings underscore the potential of APPLE as a versatile and general framework for advancing active perception in robotics.

Figures (14)

• Figure 1: Our method Active Perception Policy Learning (APPLE) aims to infer properties, such as object classes, of its environment based on limited per-step information. To do so, it jointly optimizes an action policy to gather information and a prediction model for inference. Both the action policy and the prediction model use a shared transformer-based backbone to process sequences of inputs. Illustrated on the top are four benchmark tasks we use to evaluate APPLE.
• Figure 2: Active perception process in the APPLE framework. In this task the agent's goal is to classify the digit using touch alone. At each step, it receives a tactile reading and state information (e.g., sensor position). A Vision Transformer encodes the tactile input, which is concatenated with state data and processed as a sequence over time by a transformer. At every step, the model outputs a label prediction $y_t$, evaluated against the ground truth $\overset{\ast}{y}$ via a loss function $\ell$, and an action $a_t$ that controls the sensor’s next movement.
• Figure 3: Active perception benchmarks on which we evaluate our method. TactileMNIST, TactileMNISTVolume, and Toolbox are tactile perception tasks from the Tactile MNIST Benchmark Suite2025TactileMNIST. In each environment, the agent must decide how to gather information with its tactile sensor. CircleSquare and TactileMNIST are classification tasks, where the agent must decide on a class label. TactileMNISTVolume is a regression task, where the agent must determine an object's volume. Toolbox is a pose estimation task, where the agent must determine the 2D pose of the object. All tasks require the agent to gather information actively and are not accurately solvable via random exploration.
• Figure 4: Average and final prediction accuracies for our methods APPLE-SAC and APPLE-CrossQ, HAMfleer2020learning, and APPLE-RND across various tasks. MHSB refers to the tactile classification task used in fleer2020learning. All methods were trained with 5 seeds. Shaded areas represent one standard deviation. Metrics are computed on evaluation tasks with unseen objects, except for CircleSquare and the MHSB classification task, which have only two or four, respectively.
• Figure 5: Exploration efficiency of final policies on the TactileMNIST task. Shown are the predicted probability of the correct label (top) and accuracy (bottom) after $N$ glances.