Exploring Sidewalk Sheds in New York City through Chatbot Surveys and Human Computer Interaction

TL;DR

An AI-based chatbot survey is developed that collects image-based annotations and route choices from pedestrians, linking these responses to specific shed design features, including clearance height, post spacing, and color and demonstrates a novel method for evaluating sidewalk shed designs.

Abstract

Sidewalk sheds are a common feature of the streetscape in New York City, reflecting ongoing construction and maintenance activities. However, policymakers and local business owners have raised concerns about reduced storefront visibility and altered pedestrian navigation. Although sidewalk sheds are widely used for safety, their effects on pedestrian visibility and movement are not directly measured in current planning practices. To address this, we developed an AI-based chatbot survey that collects image-based annotations and route choices from pedestrians, linking these responses to specific shed design features, including clearance height, post spacing, and color. This AI chatbot survey integrates a large language model (e.g., Google's Gemini-1.5-flash-001 model) with an image-annotation interface, allowing users to interact with street images, mark visual elements, and provide structured feedback through guided dialogue. To explore pedestrian perceptions and behaviors, this paper conducts a grid-based analysis of entrance annotations and applies logistic mixed-effects modeling to assess sidewalk choice patterns. Analysis of the dataset (n = 25) shows that: (1) the presence of scaffolding significantly reduces pedestrians' ability to identify ground-floor retail entrances, and (2) variations in weather conditions and shed design features significantly influence sidewalk selection behavior. By integrating generative AI into urban research, this study demonstrates a novel method for evaluating sidewalk shed designs and provides empirical evidence to support adjustments to shed guidelines that improve the pedestrian experience without compromising safety.

Figures (8)

• Figure 1: Image of study sites used in the sidewalk-shed perception survey. Each row corresponds to a different location in downtown Manhattan (Locations A–C; A: University Place & East 12th Street/Broadway, B: West 12th Street & 7th Avenue South, C: West 12th Street & Washington Street), and each column shows one of three visual conditions: current sidewalk scaffolding (author-captured), past scaffolding configuration (archival Google Maps imagery), and no scaffolding (baseline Google Maps imagery). The triplet design enables controlled comparison of pedestrian perception and visibility across shed conditions at the same site.
• Figure 2: System architecture of the AI chatbot for pedestrian perception data collection. The system consists of three coordinated modules. The User Interface presents street-level images and allows participants to draw directly on images using a transparent annotation canvas, enabling them to mark storefront entrances, obstructions, and preferred walking paths that are difficult to describe using text alone; a chatbot interface guides users through the tasks via short, adaptive prompts. The Processing Module receives and organizes user inputs, maintains session context, and securely stores annotated images and dialogue through the Google Cloud Platform. The Inference Module combines a Prompt Manager with the Gemini-1.5-flash-001 large language model to interpret user actions and generate consistent, task-oriented responses. Together, these components support multi-modal interaction and the collection of spatially explicit pedestrian feedback on sidewalk shed designs.
• Figure 3: User interface components of the AI chatbot survey system.(a) Interactive base map displaying existing sidewalk sheds (orange) and study sites (blue); users click on a site to begin the survey. (b) Photo display area with annotation functionality, allowing users to mark features directly on images. (c–d) chatbot interface guiding participants through text-based survey questions and follow-up prompts based on user responses.
• Figure 4: Chat flow of the AI-assisted survey. The figure illustrates the end-to-end interaction design used to address three research questions on sidewalk sheds and pedestrian perception. Participants select study locations, annotate storefront entrances under varying scaffolding and environmental conditions, and make sidewalk preference choices. Structured tasks are complemented by AI-generated follow-up prompts that elicit perceived difficulty and reasoning when responses are incomplete. The flow integrates spatial annotations, subjective assessments, and controlled scenario comparisons (design vs. environmental variation), enabling precise measurement of visibility errors, navigation preferences, and contextual influences while maintaining a consistent, participant-centered survey experience.
• Figure 5: Heatmaps of participant annotation density across sidewalk shed conditions. Each image is $2988\times1380$ pixels and subdivided into $50\times50$-pixel grids; numbers indicate the count of annotations within each grid cell. Rows correspond to study locations (A–C), and columns show the three visual scenarios: current sidewalk shed, past sidewalk shed, and no shed. Warmer colors indicate higher concentrations of participant annotations identifying storefront entrances. Across all locations, annotations are more spatially dispersed and less concentrated under current and past shed conditions, whereas the no-shed condition shows tighter clustering around storefront entrances. This pattern suggests that the presence of sidewalk sheds reduces the visual salience and visibility of storefront entrances.