Urban Incident Prediction with Graph Neural Networks: Integrating Government Ratings and Crowdsourced Reports

TL;DR

This work tackles latent state estimation of urban incidents by fusing sparse unbiased ground-truth ratings with dense biased crowdsourced reports using URBAN, a multiview, multi-output GNN. The method jointly learns true incident states and reporting biases across many incident types, enabling more accurate predictions than models using a single data source. Through a large-scale NYC case study and semi-synthetic experiments, URBAN recovers type-specific reporting coefficients and reveals demographic patterns in reporting, while producing spatially meaningful latent-state predictions across neighborhoods and incident types. The approach offers a general framework for combining heterogeneous, biased data with sparse ground truth to improve urban decision-making and resource allocation, with applicability to other spatiotemporal domains.

Abstract

Graph neural networks (GNNs) are widely used in urban spatiotemporal forecasting, such as predicting infrastructure problems. In this setting, government officials wish to know in which neighborhoods incidents like potholes or rodent issues occur. The true state of incidents (e.g., street conditions) for each neighborhood is observed via government inspection ratings. However, these ratings are only conducted for a sparse set of neighborhoods and incident types. We also observe the state of incidents via crowdsourced reports, which are more densely observed but may be biased due to heterogeneous reporting behavior. First, for such settings, we propose a multiview, multioutput GNN-based model that uses both unbiased rating data and biased reporting data to predict the true latent state of incidents. Second, we investigate a case study of New York City urban incidents and collect, standardize, and make publicly available a dataset of 9,615,863 crowdsourced reports and 1,041,415 government inspection ratings over 3 years and across 139 types of incidents. Finally, we show on both real and semi-synthetic data that our model can better predict the latent state compared to models that use only reporting data or models that use only rating data, especially when rating data is sparse and reports are predictive of ratings. We also quantify demographic biases in crowdsourced reporting, e.g., higher-income neighborhoods report problems at higher rates. Our analysis showcases a widely applicable approach for latent state prediction using heterogeneous, sparse, and biased data.

Figures (7)

• Figure 1: Model. We introduce URBAN (Unbiased Ratings and Biased reports Aggregation Network), a GNN-based model to estimate inspection ratings and reports of incidents. We model inspection ratings $r_{i kt}$ using node $i$'s learned embedding $e_n[i]$ and type $k$'s learned embedding $e_\tau[k]$. We model reports $T_{i kt}$ as a function of the rating $r_{i kt}$ and demographics $X_i$.
• Figure 2: Semi-synthetic results. We calculate the correlation between the average predicted and true rating for each node/type pair. We find that (a) our full URBAN model predicts ratings that are more correlated with ground truth than a model that uses only reporting data, and (b) as ratings are more sparsely observed, for types where reports are predictive of ratings, our full model predicts ratings that are more correlated with ground truth than a model that uses only rating data. (c) Our model's predicted coefficients $[\hat{\theta_k},\hat{\alpha_k}]$ match the true coefficients $[\theta_k,\alpha_k]$ for all types with observed ratings. Panels (a) and (c) show results averaged across 20 semi-synthetic datasets. Panel (b) shows results averaged across 5 datasets.
• Figure 3: Real data results. We calculate the correlation between the average predicted and true rating for each node/type pair. We find that (a) our full URBAN model predicts ratings that are more correlated with ground truth than a model that uses only reporting data, and (b) as ratings are more sparsely observed, for the rodent type where reports are predictive of ratings, our full model predicts ratings that are more correlated with ground truth than a model that uses only rating data. We plot the mean correlation and 95% CIs over contiguous two-year periods between 2021 and 2023.
• Figure 4: Data processing. We process the rodent inspections to remove all inspections triggered by crowdsourced reports. This figure shows the location of rodent inspections for the first 8 weeks of 2023. While the responsive inspections are distributed throughout the city (left), after we apply a heuristic, we can identify only the non-responsive inspections (right). The non-responsive inspections are clustered together because these inspections are conducted block-by-block in a scheduled manner rodent_ratings.
• Figure 5: Node-level results. For both our URBAN model and baselines, performance varies significantly across nodes. In (a) we plot the distribution of errors across all nodes. In (b) we compare the error of our full model's predicted ratings across nodes to the error of our baseline methods' predicted ratings. We find that the full model has less error than the reports-only model and comparable error to the ratings-only model. Panel (a) shows results for one model and panel (b) shows results over all contiguous two-year periods between 2021 and 2023.