Visualizing Routes with AI-Discovered Street-View Patterns

TL;DR

The paper addresses the lack of street-level visual context in route planning by introducing VivaRoutes, an interactive system that discovers visual appearance patterns (VaPatterns) from street-view imagery using semantic latent vectors. It defines a two-step computational pipeline: (1) quantify image similarity with AI-based encodings (favoring semantic latent vectors) and (2) cluster images into VaPatterns to summarize visual environments along routes. VivaRoutes visually integrates VaPatterns with traditional routing, enabling left/right route comparisons, multi-scale exploration, and direct viewing of street-view images. Case studies in New York City and a Midwest town, plus a user study, demonstrate that these visual patterns aid route understanding, preference discovery, and potential planning applications for tourism, urban studies, and navigation. The work lays groundwork for augmenting routing with visual-appearance cues, with future directions including pattern-based routing and city-scale visualization.

Abstract

Street-level visual appearances play an important role in studying social systems, such as understanding the built environment, driving routes, and associated social and economic factors. It has not been integrated into a typical geographical visualization interface (e.g., map services) for planning driving routes. In this paper, we study this new visualization task with several new contributions. First, we experiment with a set of AI techniques and propose a solution of using semantic latent vectors for quantifying visual appearance features. Second, we calculate image similarities among a large set of street-view images and then discover spatial imagery patterns. Third, we integrate these discovered patterns into driving route planners with new visualization techniques. Finally, we present VivaRoutes, an interactive visualization prototype, to show how visualizations leveraged with these discovered patterns can help users effectively and interactively explore multiple routes. Furthermore, we conducted a user study to assess the usefulness and utility of VivaRoutes.

Figures (10)

• Figure 1: VivaRoutes interface for visual exploration of driving routes in Manhattan, NY. (A) Route image window shows the captured street-view images (extracted from Google Street View API) on two selected routes from a starting location (S) to a destination (E). (B) Map view with the two routes colored according to discovered VaPatterns. (C) The VaPatterns are visualized to show the street-view patterns. (D) A map inlet for route information, VaPattern distribution, and user control.
• Figure 2: Illustration of the workflow of VaPattern extraction and visualization.
• Figure 3: The architecture of our autoencoder model.
• Figure 4: A street-view image (left) extracted from Google Street View API and its visual semantic categories extracted by PSPNet (right). The pixels are colored by their categories, such as blue for the sky, grey for buildings, etc. The category distribution is presented in Table \ref{['tbl:semantic_vector']}.
• Figure 5: Using different image encoding approaches for image similarity for a diverse set of sample streetview images extracted from Google Street View API. (Left) Using autoencoder; (Middle) Using semantic categories; (Right) Using semantic latent vector;