The Streetscape Application Services Stack (SASS): Towards a Distributed Sensing Architecture for Urban Applications

TL;DR

The paper presents SASS, a Streetscape Application Services Stack designed to tackle the challenges of distributed, multimodal sensing in urban environments by delivering three core services: multimodal data synchronization, spatiotemporal data fusion, and distributed edge computing. It introduces a modular, service-oriented architecture with an SDK and APIs that abstract hardware diversity, enforce access control, and streamline data flow from sensors to streetscape applications. Through two real-world testbeds and three real-time application services, SASS demonstrates significant gains: within-seconds synchronization accuracy improved to about $50\ \text{ms}$, multicamera fusion boosting detection accuracy by over $10\%$, and a tenfold increase in system throughput. These results support the framework’s potential to enable scalable, privacy-conscious, and low-latency urban analytics, bridging sensing infrastructure and actionable intelligence for smart cities.

Abstract

As urban populations grow, cities are becoming more complex, driving the deployment of interconnected sensing systems to realize the vision of smart cities. These systems aim to improve safety, mobility, and quality of life through applications that integrate diverse sensors with real-time decision-making. Streetscape applications-focusing on challenges like pedestrian safety and adaptive traffic management-depend on managing distributed, heterogeneous sensor data, aligning information across time and space, and enabling real-time processing. These tasks are inherently complex and often difficult to scale. The Streetscape Application Services Stack (SASS) addresses these challenges with three core services: multimodal data synchronization, spatiotemporal data fusion, and distributed edge computing. By structuring these capabilities as clear, composable abstractions with clear semantics, SASS allows developers to scale streetscape applications efficiently while minimizing the complexity of multimodal integration. We evaluated SASS in two real-world testbed environments: a controlled parking lot and an urban intersection in a major U.S. city. These testbeds allowed us to test SASS under diverse conditions, demonstrating its practical applicability. The Multimodal Data Synchronization service reduced temporal misalignment errors by 88%, achieving synchronization accuracy within 50 milliseconds. Spatiotemporal Data Fusion service improved detection accuracy for pedestrians and vehicles by over 10%, leveraging multicamera integration. The Distributed Edge Computing service increased system throughput by more than an order of magnitude. Together, these results show how SASS provides the abstractions and performance needed to support real-time, scalable urban applications, bridging the gap between sensing infrastructure and actionable streetscape intelligence.

Figures (10)

• Figure 1: SASS system architecture with core services for multimodal data synchronization, spatiotemporal fusion, and edge computing. Key subsystems support device management, data routing, and distillation, while the Control API ensures transaction reliability with rollback and action queues. The App Gateway secures access through authentication and auditing, and the Runtime Environment manages resources across edge nodes and various sensor types.
• Figure 2: Illustrative examples of the JSON-based data format used in SASS for device management and monitoring.
• Figure 3: Perspective transformation workflow using satellite and camera imagery. Left to right: (1) Original satellite image, (2) Original camera image, (3) Extracted background, (4) Area segmentation, (5) Histogram-equalized satellite image, (6) Histogram-equalized camera image, (7) Feature matching between satellite and camera images.
• Figure 4: Multimodal Data Streams from various devices to be Synchronized based on Event Detection.
• Figure 5: Multi-camera integration process. First, an object detection model is applied to each camera. Then, CoordinateTransformNet projects bounding boxes to a top-level view, using a Euclidean distance threshold to remove duplicates.