Biologically Inspired Event-Based Perception and Sample-Efficient Learning for High-Speed Table Tennis Robots

Abstract

Perception and decision-making in high-speed dynamic scenarios remain challenging for current robots. In contrast, humans and animals can rapidly perceive and make decisions in such environments. Taking table tennis as a typical example, conventional frame-based vision sensors suffer from motion blur, high latency and data redundancy, which can hardly meet real-time, accurate perception requirements. Inspired by the human visual system, event-based perception methods address these limitations through asynchronous sensing, high temporal resolution, and inherently sparse data representations. However, current event-based methods are still restricted to simplified, unrealistic ball-only scenarios. Meanwhile, existing decision-making approaches typically require thousands of interactions with the environment to converge, resulting in significant computational costs. In this work, we present a biologically inspired approach for high-speed table tennis robots, combining event-based perception with sample-efficient learning. On the perception side, we propose an event-based ball detection method that leverages motion cues and geometric consistency, operating directly on asynchronous event streams without frame reconstruction, to achieve robust and efficient detection in real-world rallies. On the decision-making side, we introduce a human-inspired, sample-efficient training strategy that first trains policies in low-speed scenarios, progressively acquiring skills from basic to advanced, and then adapts them to high-speed scenarios, guided by a case-dependent temporally adaptive reward and a reward-threshold mechanism. With the same training episodes, our method improves return-to-target accuracy by 35.8%. These results demonstrate the effectiveness of biologically inspired perception and decision-making for high-speed robotic systems.

Figures (16)

• Figure 1: The mechanism of table tennis in humans and robots. Humans perceive the position of the ball through the retina, transmitting the information as spikes via the M pathway to the dorsal visual stream to support motion perception. During the process of learning table tennis skills, humans typically learn table tennis skills in a progression from easy to hard. Inspired by this, robots can train in a brain-inspired manner. They use event cameras to achieve perception by processing asynchronous event streams. In terms of decision learning, robots first train in low-speed scenarios, gradually acquiring basic skills before progressing to more advanced capabilities to achieve policy convergence, and then continue training in high-speed scenarios.
• Figure 2: Comparison between conventional cameras and event cameras. When imaging a rotating disk with black dots, conventional cameras capture image frames at a fixed frame rate. In contrast, event cameras record brightness changes in the form of an asynchronous event stream and generate events only when the intensity change exceeds a predefined threshold falanga2020dynamic.
• Figure 3: Overview framework. The simulation environment and the algorithm are connected and exchange data via the ZMQ communication protocol.
• Figure 4: Overall pipeline of the perception and decision-making modules.
• Figure 5: Ball detection pipeline illustration.(A) Raw event image. (B) Event Stream Denoising. (C) Density-Based Spatial Clustering, with green indicating negative polarity and red indicating positive polarity. (D) Polarity-based Event Filtering. (E) Geometric Verification. (F) Ball Localization and Adaptive ROI Update.