HouseTour: A Virtual Real Estate A(I)gent

TL;DR

HouseTour presents a unified framework for spatially grounded 3D camera trajectory generation and scene-level real estate summaries from posed images. It combines a diffusion-based Residual Diffuser to produce human-like trajectories constrained by known poses with a 3D-grounded vision-language model (Qwen2-VL-3D) to generate coherent architectural narratives, further visualized through 3D Gaussian splatting. The authors introduce the HouseTour dataset, a large-scale collection of house-tour videos with ground-truth poses, 3D reconstructions, and real-estate descriptions, enabling robust evaluation of joint trajectory-and-text tasks. Experiments show that incorporating 3D spatial priors improves both trajectory fidelity and descriptive quality, outperforming task-wise baselines on end-to-end metrics and human judgments. The work enables accessible, professional-quality remote real estate video creation and points toward tighter integration of VLMs with geometry-guided diffusion for spatially consistent video synthesis.

Abstract

We introduce HouseTour, a method for spatially-aware 3D camera trajectory and natural language summary generation from a collection of images depicting an existing 3D space. Unlike existing vision-language models (VLMs), which struggle with geometric reasoning, our approach generates smooth video trajectories via a diffusion process constrained by known camera poses and integrates this information into the VLM for 3D-grounded descriptions. We synthesize the final video using 3D Gaussian splatting to render novel views along the trajectory. To support this task, we present the HouseTour dataset, which includes over 1,200 house-tour videos with camera poses, 3D reconstructions, and real estate descriptions. Experiments demonstrate that incorporating 3D camera trajectories into the text generation process improves performance over methods handling each task independently. We evaluate both individual and end-to-end performance, introducing a new joint metric. Our work enables automated, professional-quality video creation for real estate and touristic applications without requiring specialized expertise or equipment.

Figures (11)

• Figure 1: HouseTour. Given a set of images captured in an existing 3D space and their corresponding camera poses, our method tackles the task of 3D camera trajectory and textual summary generation. We focus on generating human-like trajectories and descriptions that can be used for real estate remote video tours of properties, thus highlighting spatial characteristics such as layout, functionality, architectural features, static building elements (e.g., appliances, windows, and doors), materials, and ambiance. To support the task, we present a novel dataset with real estate video tours, descriptions, and 3D reconstructions.
• Figure 2: Pipeline Overview. Given an prior tuple $(\mathcal{C}, I)$ of camera poses and images, our goal is to generate scene-level real estate summaries alongside a continuous camera trajectory that emulates human navigation. The sparse camera pose observations $\mathcal{C}$ are refined and completed by the proposed Residual Diffuser to obtain a smooth path. The resulting spatial features, along with the provided RGB frames $I$, are then processed by the Qwen2-VL-3D model to generate a coherent real estate summary.
• Figure 3: Trajectory Visualization Within the 3D Reconstructions (top view). Our method, Residual Diffuser, achieves a more human-like and smooth trajectory than the baseline Catmull-Rom Spline. Black: Ground-Truth, Green:Residual Diffuser and Red: Catmull-Rom Spline.
• Figure 4: Qualitative Results for Scene-Level Summary Generation. Our method creates accurate real-estate descriptions that capture the architectural style and elements of the space. We showcase an example of our summary generation, including sampled images of the space and a mapping between generated text and spatial elements to facilitate the reader.
• Figure 5: Gallery of sample captures from the HouseTourataset. The showcased regions originate from the 3D reconstructions.