Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Using Images as Covariates: Measuring Curb Appeal with Deep Learning

Ardyn Nordstrom, Morgan Nordstrom, Matthew D. Webb

TL;DR

The study tackles unobserved determinants of housing prices by incorporating exterior image information into hedonic models. It introduces a framework that processes exterior photos with multiple pretrained encoders (ResNet50, VGG16, InceptionV3, MobileNet) and panoptic segmentation to generate a rich covariate set, evaluated through OLS, neural networks, and a convoluted pipeline. An extensive ensemble ('tout ensemble') of encoders demonstrates out-of-sample improvements of about $3\%$ in predictive accuracy for Toronto single-family homes, highlighting the practical value of combining computer vision features with econometric models. Overall, the work demonstrates the viability and value of cross-disciplinary methods that leverage unstructured image data to enhance forecasting in real estate markets.

Abstract

This paper details an innovative methodology to integrate image data into traditional econometric models. Motivated by forecasting sales prices for residential real estate, we harness the power of deep learning to add "information" contained in images as covariates. Specifically, images of homes were categorized and encoded using an ensemble of image classifiers (ResNet-50, VGG16, MobileNet, and Inception V3). Unique features presented within each image were further encoded through panoptic segmentation. Forecasts from a neural network trained on the encoded data results in improved out-of-sample predictive power. We also combine these image-based forecasts with standard hedonic real estate property and location characteristics, resulting in a unified dataset. We show that image-based forecasts increase the accuracy of hedonic forecasts when encoded features are regarded as additional covariates. We also attempt to "explain" which covariates the image-based forecasts are most highly correlated with. The study exemplifies the benefits of interdisciplinary methodologies, merging machine learning and econometrics to harness untapped data sources for more accurate forecasting.

Using Images as Covariates: Measuring Curb Appeal with Deep Learning

TL;DR

The study tackles unobserved determinants of housing prices by incorporating exterior image information into hedonic models. It introduces a framework that processes exterior photos with multiple pretrained encoders (ResNet50, VGG16, InceptionV3, MobileNet) and panoptic segmentation to generate a rich covariate set, evaluated through OLS, neural networks, and a convoluted pipeline. An extensive ensemble ('tout ensemble') of encoders demonstrates out-of-sample improvements of about in predictive accuracy for Toronto single-family homes, highlighting the practical value of combining computer vision features with econometric models. Overall, the work demonstrates the viability and value of cross-disciplinary methods that leverage unstructured image data to enhance forecasting in real estate markets.

Abstract

This paper details an innovative methodology to integrate image data into traditional econometric models. Motivated by forecasting sales prices for residential real estate, we harness the power of deep learning to add "information" contained in images as covariates. Specifically, images of homes were categorized and encoded using an ensemble of image classifiers (ResNet-50, VGG16, MobileNet, and Inception V3). Unique features presented within each image were further encoded through panoptic segmentation. Forecasts from a neural network trained on the encoded data results in improved out-of-sample predictive power. We also combine these image-based forecasts with standard hedonic real estate property and location characteristics, resulting in a unified dataset. We show that image-based forecasts increase the accuracy of hedonic forecasts when encoded features are regarded as additional covariates. We also attempt to "explain" which covariates the image-based forecasts are most highly correlated with. The study exemplifies the benefits of interdisciplinary methodologies, merging machine learning and econometrics to harness untapped data sources for more accurate forecasting.
Paper Structure (10 sections, 2 equations, 3 figures, 2 tables)

This paper contains 10 sections, 2 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Data
  3. Methods
  4. Turning Images Into Data
  5. How Image Data are Used to Predict House Prices
  6. Assessing the Information Content of the Images
  7. Results with an Application to Housing Price Predictions
  8. Do Images Capture Useful Information about Houses?
  9. Can Data from Images Improve our Housing Price Predictions?
  10. Conclusions

Figures (3)

  • Figure 1: ResNet50 Architecture with Convolution-Layer Predictions
  • Figure 2: Examples of Encoded Images and Features Within Images
  • Figure 3: Comparison of Different Models