3D Visual Illusion Depth Estimation

TL;DR

This work addresses how 3D visual illusions disrupt depth estimation by introducing the expansive 3D-Visual-Illusion dataset, which spans five illusion types and includes virtual and real-world scenes. It proposes a VLM-driven monocular–stereo fusion framework that leverages commonsense reasoning to assess the reliability of depth cues and guide fusion, aligning monocular depth with stereo disparity via learnable affine parameters. Experimental results show that SOTA monocular, stereo, and multi-view methods are easily fooled by illusions, whereas the proposed fusion approach achieves state-of-the-art performance across illusion scenarios and across both real and synthetic data. The findings advance robust depth perception in challenging visual contexts with potential impact on AR/VR, robotics, and related perception systems, while outlining limitations and future directions such as dataset automation and efficiency improvements.

Abstract

3D visual illusion is a perceptual phenomenon where a two-dimensional plane is manipulated to simulate three-dimensional spatial relationships, making a flat artwork or object look three-dimensional in the human visual system. In this paper, we reveal that the machine visual system is also seriously fooled by 3D visual illusions, including monocular and binocular depth estimation. In order to explore and analyze the impact of 3D visual illusion on depth estimation, we collect a large dataset containing almost 3k scenes and 200k images to train and evaluate SOTA monocular and binocular depth estimation methods. We also propose a 3D visual illusion depth estimation framework that uses common sense from the vision language model to adaptively fuse depth from binocular disparity and monocular depth. Experiments show that SOTA monocular, binocular, and multi-view depth estimation approaches are all fooled by various 3D visual illusions, while our method achieves SOTA performance.

Figures (15)

• Figure 1: The visualization of 3D visual illusions.
• Figure 2: The data generation pipeline for web-sourced data.
• Figure 3: The data generation pipeline for videos from generative models.
• Figure 4: The pipeline of VLM-driven binocular and monocular disparity fusion model.
• Figure 5: Visualization on our dataset.