DistillGrasp: Integrating Features Correlation with Knowledge Distillation for Depth Completion of Transparent Objects

TL;DR

This work tackles depth completion for transparent objects by proposing DistillGrasp, a teacher–student framework that distills knowledge from a high-capacity teacher with a position correlation block (PCB) to a lightweight student with a consistent feature correlation module (CFCM). The teacher correlates RGB and depth via transformer-based PCB and reconstructs depth with NeWCRFs-based decoding, while the student fuses reliable RGB/depth regions with CNNs through CFCM to maintain efficiency. A composite distillation loss combining scale-invariant depth, structure, and edge cues guides the student to inherit both global context and boundary details from the teacher. Experiments on ClearGrasp show state-of-the-art performance for the teacher and competitive, real-time results for the student, with a real-robot UR10e grasping test demonstrating practical robustness and applicability to transparent-object manipulation.

Abstract

Due to the visual properties of reflection and refraction, RGB-D cameras cannot accurately capture the depth of transparent objects, leading to incomplete depth maps. To fill in the missing points, recent studies tend to explore new visual features and design complex networks to reconstruct the depth, however, these approaches tremendously increase computation, and the correlation of different visual features remains a problem. To this end, we propose an efficient depth completion network named DistillGrasp which distillates knowledge from the teacher branch to the student branch. Specifically, in the teacher branch, we design a position correlation block (PCB) that leverages RGB images as the query and key to search for the corresponding values, guiding the model to establish correct correspondence between two features and transfer it to the transparent areas. For the student branch, we propose a consistent feature correlation module (CFCM) that retains the reliable regions of RGB images and depth maps respectively according to the consistency and adopts a CNN to capture the pairwise relationship for depth completion. To avoid the student branch only learning regional features from the teacher branch, we devise a distillation loss that not only considers the distance loss but also the object structure and edge information. Extensive experiments conducted on the ClearGrasp dataset manifest that our teacher network outperforms state-of-the-art methods in terms of accuracy and generalization, and the student network achieves competitive results with a higher speed of 48 FPS. In addition, the significant improvement in a real-world robotic grasping system illustrates the effectiveness and robustness of our proposed system.

Figures (5)

• Figure 1: Overview of our proposed method DistillGrasp.
• Figure 2: Qualitative comparison of the state-of-the-art approaches, where the generated details are highlighted with red circles.
• Figure 3: Examples of transparent objects in ClearGrasp dataset
• Figure 4: Visualization of the teacher and student network variants, where the generated details are highlighted with red circles.
• Figure 5: UR10e grasping the transparent objects.