Affordance Labeling and Exploration: A Manifold-Based Approach
İsmail Özçil, A. Buğra Koku
TL;DR
This work tackles affordance labeling by reusing feature vectors from ImageNet-pretrained networks without retraining final layers. It introduces two vector-based labeling approaches, Subspace Projection Method (SPM) and Manifold Curvature Method (MCM), and evaluates them across nine networks on a large RGBD affordance dataset. MCM consistently matches or surpasses state-of-the-art labeling performance (e.g., up to 96.45% TPR on RegNetY) and demonstrates better transferability across networks than SPM, while both methods can reveal plausible, non-ground-truth affordances. The results suggest practical, training-free affordance exploration and easy scalability to new affordances, benefiting robot-object interaction and open-vocabulary reasoning.
Abstract
The advancement in computing power has significantly reduced the training times for deep learning, fostering the rapid development of networks designed for object recognition. However, the exploration of object utility, which is the affordance of the object, as opposed to object recognition, has received comparatively less attention. This work focuses on the problem of exploration of object affordances using existing networks trained on the object classification dataset. While pre-trained networks have proven to be instrumental in transfer learning for classification tasks, this work diverges from conventional object classification methods. Instead, it employs pre-trained networks to discern affordance labels without the need for specialized layers, abstaining from modifying the final layers through the addition of classification layers. To facilitate the determination of affordance labels without such modifications, two approaches, i.e. subspace clustering and manifold curvature methods are tested. These methods offer a distinct perspective on affordance label recognition. Especially, manifold curvature method has been successfully tested with nine distinct pre-trained networks, each achieving an accuracy exceeding 95%. Moreover, it is observed that manifold curvature and subspace clustering methods explore affordance labels that are not marked in the ground truth, but object affords in various cases.