Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Learning at the Intersection: Certified Robustness as a Tool for 3D Vision

Gabriel Pérez S, Juan C. Pérez, Motasem Alfarra, Jesús Zarzar, Sara Rojas, Bernard Ghanem, Pablo Arbeláez

TL;DR

The paper investigates a link between certified robustness and 3D object modeling through the Signed Distance Function (SDF). It proposes computing SDFs via Randomized Smoothing (RS) in low-dimensional 3D space by modeling occupancy on voxel grids and performing Gaussian smoothing, establishing the relation $\text{SDF}(x) \equiv \text{MCR}(f_{occ}, x)$ with the smoothed estimate $\hat{f}(x) = (f_\theta * \mathcal{N}(0, \sigma^2 I))(x)$. The method is demonstrated by integrating the resulting weak SDF into a NeRF-like novel view synthesis pipeline, deriving occupancy and density fields and performing volume rendering, with mesh extraction via marching cubes. This cross-domain approach offers a scalable way to obtain geometry-aware guarantees in 3D vision and graphics and suggests ample avenues for future robustness-geometry research.

Abstract

This paper presents preliminary work on a novel connection between certified robustness in machine learning and the modeling of 3D objects. We highlight an intriguing link between the Maximal Certified Radius (MCR) of a classifier representing a space's occupancy and the space's Signed Distance Function (SDF). Leveraging this relationship, we propose to use the certification method of randomized smoothing (RS) to compute SDFs. Since RS' high computational cost prevents its practical usage as a way to compute SDFs, we propose an algorithm to efficiently run RS in low-dimensional applications, such as 3D space, by expressing RS' fundamental operations as Gaussian smoothing on pre-computed voxel grids. Our approach offers an innovative and practical tool to compute SDFs, validated through proof-of-concept experiments in novel view synthesis. This paper bridges two previously disparate areas of machine learning, opening new avenues for further exploration and potential cross-domain advancements.

Deep Learning at the Intersection: Certified Robustness as a Tool for 3D Vision

TL;DR

The paper investigates a link between certified robustness and 3D object modeling through the Signed Distance Function (SDF). It proposes computing SDFs via Randomized Smoothing (RS) in low-dimensional 3D space by modeling occupancy on voxel grids and performing Gaussian smoothing, establishing the relation with the smoothed estimate . The method is demonstrated by integrating the resulting weak SDF into a NeRF-like novel view synthesis pipeline, deriving occupancy and density fields and performing volume rendering, with mesh extraction via marching cubes. This cross-domain approach offers a scalable way to obtain geometry-aware guarantees in 3D vision and graphics and suggests ample avenues for future robustness-geometry research.

Abstract

This paper presents preliminary work on a novel connection between certified robustness in machine learning and the modeling of 3D objects. We highlight an intriguing link between the Maximal Certified Radius (MCR) of a classifier representing a space's occupancy and the space's Signed Distance Function (SDF). Leveraging this relationship, we propose to use the certification method of randomized smoothing (RS) to compute SDFs. Since RS' high computational cost prevents its practical usage as a way to compute SDFs, we propose an algorithm to efficiently run RS in low-dimensional applications, such as 3D space, by expressing RS' fundamental operations as Gaussian smoothing on pre-computed voxel grids. Our approach offers an innovative and practical tool to compute SDFs, validated through proof-of-concept experiments in novel view synthesis. This paper bridges two previously disparate areas of machine learning, opening new avenues for further exploration and potential cross-domain advancements.
Paper Structure (14 sections, 6 equations, 3 figures, 3 tables)

This paper contains 14 sections, 6 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminaries: Certified Robustness
  3. Randomized Smoothing
  4. Efficient Randomized Smoothing in Low-dimensional Spaces
  5. SDF as Certified Radius: Applications in 3D Vision
  6. Novel View Synthesis
  7. Experiments & Results
  8. Setup and Implementation Details
  9. Dataset
  10. Implementation
  11. 3D object extraction
  12. Metrics
  13. Results and Analysis
  14. Conclusions

Figures (3)

  • Figure 1: Connection between certified robustness and 3D modeling via signed distance function (SDF). We observe an equivalence between the Maximal Certified Radius (MCR) of a space's occupancy function $f_\text{occ}$ at a point $x$ and the value of $\text{SDF}(x)$, i.e. the (signed) distance to the closest surface.
  • Figure 2: Rendering pipeline: A voxel grid $f_\theta$ is trained within [0, 1]. Gaussian smoothing via 3D convolution produces $\hat{f}$. Utilizing this, a Weak Signed Distance Function (SDF) incorporates $\hat{f}$, the normal distribution's CDF, and $\sigma$. High-eccentricity sigmoid application to $\hat{f}$ generates $G(x)$ for occupancy. Rendering density, obtained from $G(x)$, is calculated using a differentiable density activation function resembling a negative logarithm asymptotically approaching $1+\epsilon$, yielding $g(x)$. The renderer queries $g(x)$ for density.
  • Figure 3: Qualitative results Ground truth test image, rendered image, radiance field's depth map and SDF axis cuts are presented for the Lego and Chair scenes.