Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Superpixel-informed Implicit Neural Representation for Multi-Dimensional Data

Jiayi Li, Xile Zhao, Jianli Wang, Chao Wang, Min Wang

TL;DR

This work proposes a novel Superpixel-informed INR (S-INR), utilizing generalized superpixel instead of pixel as an alternative basic unit of INR for multi-dimensional data recovery (e.g., images and weather data).

Abstract

Recently, implicit neural representations (INRs) have attracted increasing attention for multi-dimensional data recovery. However, INRs simply map coordinates via a multi-layer perception (MLP) to corresponding values, ignoring the inherent semantic information of the data. To leverage semantic priors from the data, we propose a novel Superpixel-informed INR (S-INR). Specifically, we suggest utilizing generalized superpixel instead of pixel as an alternative basic unit of INR for multi-dimensional data (e.g., images and weather data). The coordinates of generalized superpixels are first fed into exclusive attention-based MLPs, and then the intermediate results interact with a shared dictionary matrix. The elaborately designed modules in S-INR allow us to ingenuously exploit the semantic information within and across generalized superpixels. Extensive experiments on various applications validate the effectiveness and efficacy of our S-INR compared to state-of-the-art INR methods.

Superpixel-informed Implicit Neural Representation for Multi-Dimensional Data

TL;DR

This work proposes a novel Superpixel-informed INR (S-INR), utilizing generalized superpixel instead of pixel as an alternative basic unit of INR for multi-dimensional data recovery (e.g., images and weather data).

Abstract

Recently, implicit neural representations (INRs) have attracted increasing attention for multi-dimensional data recovery. However, INRs simply map coordinates via a multi-layer perception (MLP) to corresponding values, ignoring the inherent semantic information of the data. To leverage semantic priors from the data, we propose a novel Superpixel-informed INR (S-INR). Specifically, we suggest utilizing generalized superpixel instead of pixel as an alternative basic unit of INR for multi-dimensional data (e.g., images and weather data). The coordinates of generalized superpixels are first fed into exclusive attention-based MLPs, and then the intermediate results interact with a shared dictionary matrix. The elaborately designed modules in S-INR allow us to ingenuously exploit the semantic information within and across generalized superpixels. Extensive experiments on various applications validate the effectiveness and efficacy of our S-INR compared to state-of-the-art INR methods.

Paper Structure

This paper contains 20 sections, 5 equations, 7 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. The Proposed Model
  3. Definition of Generalized Superpixel
  4. The Proposed S-INR
  5. The Proposed Recovery Model
  6. Experiments
  7. Experimental Settings
  8. Image Data.
  9. Point Data.
  10. Hyperparameters Settings.
  11. Experimental Results
  12. Image Reconstruction Results.
  13. Image Completion Results.
  14. Image Denoising Results.
  15. 3D Surface Completion Results.
  16. ...and 5 more sections

Figures (7)

  • Figure 1: Illustration of the overall processing of the traditional INR and our proposed Superpixel-informed INR (S-INR) on image denoising task. In the proposed S-INR, generalized superpixels are used as basic units. The coordinates of generalized superpixels are fed into the exclusive attention-based MLPs, and then the intermediate results interact with a shared dictionary matrix to obtain recovery results.
  • Figure 2: Illustration of generalized superpixel segmentation on the (a) image data, (b) 3D surface data, and (c) weather data.
  • Figure 3: The results of image reconstruction by different methods on RGB image $\it{Kodim}$.
  • Figure 4: From top to bottom list the results of image completion and image denoising by different methods on $\it{Mor}$ in Case2 and $\it{Lehavim}$ in Case1, respectively.
  • Figure 5: From top to down respectively list the results of 3D surface completion recovery by different methods for the 3D surface data $\it{Scene1}$, $\it{Scene2}$, and $\it{Scene3}$ with the random sampling rate of 0.025.
  • ...and 2 more figures

Theorems & Definitions (2)

  • definition 1: Generalized Superpixel
  • remark 1