Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Implicit neural representation of textures

Albert Kwok, Zheyuan Hu, Dounia Hammou

TL;DR

This work investigates implicit neural representations (INRs) as continuous texture representations to replace traditional discrete texture maps, aiming for memory efficiency and high rendering quality. It compares four INR designs (including MLPs with SIREN and Fourier encodings) and demonstrates Mitsuba 3 integration, along with mipmap fitting and INR-space generation via hypernetworks. A dataset-driven evaluation on 25 textures with Laplacian-variance selection shows that Fourier-encoded MLPs and SIREN achieve strong perceptual quality (LPIPS, VMAF) at competitive bitrates, often outperforming ASTC on perceptual metrics while highlighting artefacts and the importance of hyperparameter tuning. The results support practical deployment in real-time rendering and downstream tasks, such as material generation and asset baking, and point to future work in hyperparameter selection, multi-texture compression, and advanced filtering models.

Abstract

Implicit neural representation (INR) has proven to be accurate and efficient in various domains. In this work, we explore how different neural networks can be designed as a new texture INR, which operates in a continuous manner rather than a discrete one over the input UV coordinate space. Through thorough experiments, we demonstrate that these INRs perform well in terms of image quality, with considerable memory usage and rendering inference time. We analyze the balance between these objectives. In addition, we investigate various related applications in real-time rendering and down-stream tasks, e.g. mipmap fitting and INR-space generation.

Implicit neural representation of textures

TL;DR

This work investigates implicit neural representations (INRs) as continuous texture representations to replace traditional discrete texture maps, aiming for memory efficiency and high rendering quality. It compares four INR designs (including MLPs with SIREN and Fourier encodings) and demonstrates Mitsuba 3 integration, along with mipmap fitting and INR-space generation via hypernetworks. A dataset-driven evaluation on 25 textures with Laplacian-variance selection shows that Fourier-encoded MLPs and SIREN achieve strong perceptual quality (LPIPS, VMAF) at competitive bitrates, often outperforming ASTC on perceptual metrics while highlighting artefacts and the importance of hyperparameter tuning. The results support practical deployment in real-time rendering and downstream tasks, such as material generation and asset baking, and point to future work in hyperparameter selection, multi-texture compression, and advanced filtering models.

Abstract

Implicit neural representation (INR) has proven to be accurate and efficient in various domains. In this work, we explore how different neural networks can be designed as a new texture INR, which operates in a continuous manner rather than a discrete one over the input UV coordinate space. Through thorough experiments, we demonstrate that these INRs perform well in terms of image quality, with considerable memory usage and rendering inference time. We analyze the balance between these objectives. In addition, we investigate various related applications in real-time rendering and down-stream tasks, e.g. mipmap fitting and INR-space generation.
Paper Structure (32 sections, 10 equations, 20 figures, 1 table)

This paper contains 32 sections, 10 equations, 20 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related work
  3. Texture compression.
  4. INR design.
  5. Method
  6. Dataset analysis and sample selection
  7. Texture dataset.
  8. INR architectures
  9. Multi-Layer Perceptron (MLP)
  10. Mipmap fitting
  11. INR-space generation
  12. Data augmentation.
  13. Evaluation
  14. Evaluation metrics.
  15. Single texture performance evaluation
  16. ...and 17 more sections

Figures (20)

  • Figure 1: (left:) A 3D scene rendered with our INR textures, showcasing high visual quality and straightforward deployment. (right:) A three-channel mipmap pyramid represented by an MLP.
  • Figure 2: Laplacian response histogram with its Cumulative Distribution Function (red curve).
  • Figure 3: MLP architectures used for INR texture, both without and with positional encoding (pos. enc.).
  • Figure 4: Texture RGB augmentation for generation.
  • Figure 5: Texture INR performance against bitrate (bits per pixel).
  • ...and 15 more figures