Neural Field Classifiers via Target Encoding and Classification Loss

TL;DR

A novel Neural Field Classifier (NFC) framework is proposed which formulates existing neural field methods as classification tasks rather than regression tasks, and can easily transform arbitrary Neural Field Regressor into its classification variant via employing a novel Target Encoding module and optimizing a classification loss.

Abstract

Neural field methods have seen great progress in various long-standing tasks in computer vision and computer graphics, including novel view synthesis and geometry reconstruction. As existing neural field methods try to predict some coordinate-based continuous target values, such as RGB for Neural Radiance Field (NeRF), all of these methods are regression models and are optimized by some regression loss. However, are regression models really better than classification models for neural field methods? In this work, we try to visit this very fundamental but overlooked question for neural fields from a machine learning perspective. We successfully propose a novel Neural Field Classifier (NFC) framework which formulates existing neural field methods as classification tasks rather than regression tasks. The proposed NFC can easily transform arbitrary Neural Field Regressor (NFR) into its classification variant via employing a novel Target Encoding module and optimizing a classification loss. By encoding a continuous regression target into a high-dimensional discrete encoding, we naturally formulate a multi-label classification task. Extensive experiments demonstrate the impressive effectiveness of NFC at the nearly free extra computational costs. Moreover, NFC also shows robustness to sparse inputs, corrupted images, and dynamic scenes.

Figures (11)

• Figure 1: The illustration of standard Neural Field Regressors and our Neural Field Classifiers. Our method makes two modifications on existing neural fields. First, due to the Target Encoding module, the final output of neural networks need to be a high-dimensional color encoding rather than a three-channel color value itself. The designed encoding rule connects our high-dimensional discrete representation and the standard three-channel continuous representation. Second, we mainly use a classification loss as the main optimization objective. Note that the classification loss, as the main optimization objective, can be larger than the standard MSE loss by two orders of magnitude.
• Figure 2: The illustration of Binary-Number Target Encoding.
• Figure 3: Qualitative comparisons of NFC and NFR for static scenes. Top Row: NeRF. Bottom Row: DVGO.
• Figure 4: Qualitative comparison of NFC and NFR. Top Row: Dynamic Scenes. Middle Row: Sparse Inputs. Bottom Row: Corrupted Images.
• Figure 5: Qualitative comparisons of RGB rendering, depth rendering and normal rendering between NeuS-R and NeuS-C for neural surface reconstruction. Dataset: Replica Scene 8 (Office 4).