INRet: A General Framework for Accurate Retrieval of INRs for Shapes

TL;DR

INRet addresses the problem of retrieving shapes represented as INRs across diverse architectures and implicit functions by learning embeddings from INR weights and spatial grids. It introduces separate Embedding Encoders for MLPs and feature grids, coupled with a Shape Decoder during training, and enforces a unified latent space via L2 alignment and a Unified Shape Decoder to enable cross-function retrieval. Experiments on ShapeNet10 and Pix3D show that INRet outperforms INR-only baselines and conversion-based retrieval, with gains of around 10% in mAP and faster retrieval when possible, while maintaining robust cross-architecture and cross-function performance. The framework significantly improves the scalability and practicality of organizing and querying large INR data stores for 3D shapes, enabling accurate retrieval without onerous conversions or architecture-specific constraints. The Category-Chamfer metric and hierarchical sampling further provide nuanced evaluation and efficient retrieval in large-scale datasets.

Abstract

Implicit neural representations (INRs) have become an important method for encoding various data types, such as 3D objects or scenes, images, and videos. They have proven to be particularly effective at representing 3D content, e.g., 3D scene reconstruction from 2D images, novel 3D content creation, as well as the representation, interpolation, and completion of 3D shapes. With the widespread generation of 3D data in an INR format, there is a need to support effective organization and retrieval of INRs saved in a data store. A key aspect of retrieval and clustering of INRs in a data store is the formulation of similarity between INRs that would, for example, enable retrieval of similar INRs using a query INR. In this work, we propose INRet, a method for determining similarity between INRs that represent shapes, thus enabling accurate retrieval of similar shape INRs from an INR data store. INRet flexibly supports different INR architectures such as INRs with octree grids, triplanes, and hash grids, as well as different implicit functions including signed/unsigned distance function and occupancy field. We demonstrate that our method is more general and accurate than the existing INR retrieval method, which only supports simple MLP INRs and requires the same architecture between the query and stored INRs. Furthermore, compared to converting INRs to other representations (e.g., point clouds or multi-view images) for 3D shape retrieval, INRet achieves higher accuracy while avoiding the conversion overhead.

Figures (8)

• Figure 1: Encoders trained to generate embeddings for grid-based INRs: INRs with hash-table/octree/triplane based feature grids are used to generate embeddings for similarity calculations using encoders ($\bm{m}$ and $\bm{c}$). The encoders take MLP weights and feature grid parameters as inputs to generate the INR Embedding. During training, the encoders ($\bm{m}$, $\bm{c}$) and the decoder ($f_\phi$) are jointly trained: the encoders to produce the INR Embedding and the decoder to reconstruct the original shape, ensuring the embeddings carry information about the shape. During inference, only the encoders are used to generate the INR Embedding for similarity calculations and retrieval.
• Figure 2: INR Embed. Creation for INRs with Different Implicit Functions. (a) For each shape, we train INRs with different implicit functions. (b) We train different encoders for INRs with different implicit functions. The differences between embeddings created by the encoders are minimized by L2 loss. (c) We feed the embeddings into a Unified Shape Decoder to recreate the UDF of the original shape.
• Figure 3: Retrieval Qualitative Comparison.
• Figure 4: INR Embedding tSNE Plot
• Figure 5: INR Embedding Creation for INRs with Different Architectures