Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Data-Informed Interventions: Opportunities and Challenges of Street-level Multimodal Sensing

Joao Rulff, Giancarlo Pereira, Maryam Hosseini, Marcos Lage, Claudio Silva

Abstract

Over the past decades, improvements in data collection hardware coupled with novel artificial intelligence algorithms have made it possible for researchers to understand urban environments at an unprecedented scale. From local interactions between actors to city-wide infrastructural problems, this new data-driven approach enables a more informed and trustworthy decision-making process aiming at transforming cities into safer and more equitable places for living. This new moment unfolded new opportunities to understand various phenomena that directly impact how accessible cities are to heterogeneous populations. Specifically, sensing localized physical interactions among actors under different scenarios can drive substantial interventions in urban environments to make them safer for all. In this manuscript, we list opportunities and associated challenges to leverage street-level multimodal sensing data to empower domain experts in making more informed decisions and, ultimately, supporting a data-informed policymaking framework. The challenges presented here can motivate research in different areas, such as computer vision and human-computer interaction, to support cities in growing more sustainably.

Towards Data-Informed Interventions: Opportunities and Challenges of Street-level Multimodal Sensing

Abstract

Over the past decades, improvements in data collection hardware coupled with novel artificial intelligence algorithms have made it possible for researchers to understand urban environments at an unprecedented scale. From local interactions between actors to city-wide infrastructural problems, this new data-driven approach enables a more informed and trustworthy decision-making process aiming at transforming cities into safer and more equitable places for living. This new moment unfolded new opportunities to understand various phenomena that directly impact how accessible cities are to heterogeneous populations. Specifically, sensing localized physical interactions among actors under different scenarios can drive substantial interventions in urban environments to make them safer for all. In this manuscript, we list opportunities and associated challenges to leverage street-level multimodal sensing data to empower domain experts in making more informed decisions and, ultimately, supporting a data-informed policymaking framework. The challenges presented here can motivate research in different areas, such as computer vision and human-computer interaction, to support cities in growing more sustainably.

Paper Structure

This paper contains 11 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Opportunities
  4. Affordances of Audio and Video
  5. Movement Analysis
  6. Adaptive Signal Timing
  7. Challenges
  8. Expert in the Loop
  9. Optimal Deployment
  10. Privacy
  11. Conclusion

Figures (2)

  • Figure 1: This figure highlights situations where pedestrians with limited mobility levels cross streets (A, C, D). These examples were identified using open-vocabulary detection models. (B) shows how street-level videos can contain valuable information regarding traffic lights.
  • Figure 2: Distribution of pedestrian density over an entire video. Crowds are known to reduce the speed of pedestrians crossing the streets. Adapting traffic light timing based on pedestrian speed can be achieved by leveraging street-level RGB videos.