NeuralFur: Animal Fur Reconstruction From Multi-View Images

TL;DR

NeuralFur introduces the first multi-view, strand-based animal fur reconstruction method that leverages external knowledge from vision-language models to inform furless body geometry, fur length, and growth direction. The workflow combines NeuS-based furless geometry with a defurring step guided by a VLM, followed by a neural fur representation that decodes latent strand codes into centimeter-scale fur using a Gaussian Splatting rendering pipeline. The approach achieves generalization across diverse animals and fur types, enabling downstream rendering and physics-based simulation within standard graphics pipelines. By integrating external knowledge when data priors are scarce, NeuralFur advances realistic animal fur reconstruction from images and opens avenues for broader animal species and fur styles in practical CG applications.

Abstract

Reconstructing realistic animal fur geometry from images is a challenging task due to the fine-scale details, self-occlusion, and view-dependent appearance of fur. In contrast to human hairstyle reconstruction, there are also no datasets that can be leveraged to learn a fur prior for different animals. In this work, we present a first multi-view-based method for high-fidelity 3D fur modeling of animals using a strand-based representation, leveraging the general knowledge of a vision language model. Given multi-view RGB images, we first reconstruct a coarse surface geometry using traditional multi-view stereo techniques. We then use a vision language model (VLM) system to retrieve information about the realistic length structure of the fur for each part of the body. We use this knowledge to construct the animal's furless geometry and grow strands atop it. The fur reconstruction is supervised with both geometric and photometric losses computed from multi-view images. To mitigate orientation ambiguities stemming from the Gabor filters that are applied to the input images, we additionally utilize the VLM to guide the strands' growth direction and their relation to the gravity vector that we incorporate as a loss. With this new schema of using a VLM to guide 3D reconstruction from multi-view inputs, we show generalization across a variety of animals with different fur types. For additional results and code, please refer to https://neuralfur.is.tue.mpg.de.

Figures (16)

• Figure 1: Our method, NeuralFur, consists of two stages: (i) extracting a furless mesh geometry by intelligently shrinking the full mesh reconstructed from multi-view images, and (ii) reconstructing strand-based fur by initializing roots from the furless mesh. For both stages, external knowledge from a VLM is leveraged. Based on the depicted animal, the VLM provides information about fur thickness, length, and orientation. This guidance is then used to train a neural fur strand representation (MLP), which can be queried at any mesh surface location to generate fur strands suitable for rendering.
• Figure 2: Annotation. Automatic part annotation for each animal obtained from the fitted SMAL Zuffi:CVPR:2017 model.
• Figure 3: From left to right: input image, initial shape $M_{NeuS}$, furless geometry $M_{bald}$, and their overlay.
• Figure 4: Qualitative results of our reconstruction method compared with existing baselines. Surface reconstruction baselines produce very coarse geometry. Applying the existing state-of-the-art hair reconstruction method, Gaussian Haircut zakharov2024gaussianhaircut leads to inconsistent strand lengths and noticeable artifacts. Our method produces accurate strand-based geometry. A digital zoom-in is recommended.
• Figure 5: Ablation study. Qualitative evaluation of our design choices regarding length, fur parametrization, and the importance of the defurring approach for accurate geometry modeling.