Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Data streaming platform for crowd-sourced vehicle dataset generation

Felipe Mogollon, Zaloa Fernandez, Angel Martin, Juan Diego Ortega, Gorka Velez

TL;DR

An edge-cloud data platform is proposed to connect car data producers with multiple and heterogeneous services, addressing key challenges in Data Spaces, such as data sovereignty, governance, interoperability, and privacy.

Abstract

Vehicles are sophisticated machines equipped with sensors that provide real-time data for onboard driving assistance systems. Due to the wide variety of traffic, road, and weather conditions, continuous system enhancements are essential. Connectivity allows vehicles to transmit previously unknown data, expanding datasets and accelerating the development of new data models. This enables faster identification and integration of novel data, improving system reliability and reducing time to market. Data Spaces aim to create a data-driven, interconnected, and innovative data economy, where edge and cloud infrastructures support a virtualised IoT platform that connects data sources and development servers. This paper proposes an edge-cloud data platform to connect car data producers with multiple and heterogeneous services, addressing key challenges in Data Spaces, such as data sovereignty, governance, interoperability, and privacy. The paper also evaluates the data platform's performance limits for text, image, and video data workloads, examines the impact of connectivity technologies, and assesses latencies. The results show that latencies drop to 33ms with 5G connectivity when pipelining data to consuming applications hosted at the edge, compared to around 77ms when crossing both edge and cloud processing infrastructures. The results offer guidance on the necessary processing assets to avoid bottlenecks in car data platforms.

Data streaming platform for crowd-sourced vehicle dataset generation

TL;DR

An edge-cloud data platform is proposed to connect car data producers with multiple and heterogeneous services, addressing key challenges in Data Spaces, such as data sovereignty, governance, interoperability, and privacy.

Abstract

Vehicles are sophisticated machines equipped with sensors that provide real-time data for onboard driving assistance systems. Due to the wide variety of traffic, road, and weather conditions, continuous system enhancements are essential. Connectivity allows vehicles to transmit previously unknown data, expanding datasets and accelerating the development of new data models. This enables faster identification and integration of novel data, improving system reliability and reducing time to market. Data Spaces aim to create a data-driven, interconnected, and innovative data economy, where edge and cloud infrastructures support a virtualised IoT platform that connects data sources and development servers. This paper proposes an edge-cloud data platform to connect car data producers with multiple and heterogeneous services, addressing key challenges in Data Spaces, such as data sovereignty, governance, interoperability, and privacy. The paper also evaluates the data platform's performance limits for text, image, and video data workloads, examines the impact of connectivity technologies, and assesses latencies. The results show that latencies drop to 33ms with 5G connectivity when pipelining data to consuming applications hosted at the edge, compared to around 77ms when crossing both edge and cloud processing infrastructures. The results offer guidance on the necessary processing assets to avoid bottlenecks in car data platforms.

Paper Structure

This paper contains 13 sections, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. MLOps for CCAM
  4. IoT platforms
  5. Proposed solution
  6. CCAM-driven design
  7. Edge pipeline
  8. Cloud services
  9. Experimentation
  10. Experimentation setup
  11. Metrics
  12. Results
  13. Conclusion

Figures (8)

  • Figure 1: Features of the proposed data platform for developing CCAM applications and services.
  • Figure 2: CCAM applications and services casting events to sensors and devices in an area.
  • Figure 3: Experimentation setup including data producer and consumer and concurrent workload.
  • Figure 4: Data Access Delay for image data with producers connected by Ethernet and consumers accessing through the cloud.
  • Figure 5: Data Delivery Delay for image data with producers connected by Ethernet and consumers accessing through the cloud.
  • ...and 3 more figures