A Serverless Edge-Native Data Processing Architecture for Autonomous Driving Training

TL;DR

The paper addresses the data bottleneck and distribution imbalance in autonomous driving datasets, emphasizing the need to capture rare, safety-critical events. It proposes the Lambda framework, an edge-native serverless platform that runs per-function runtimes on-vehicle, providing FaaS-inspired abstractions to filter and record data with low latency while remaining compatible with ROS 2. Empirical evaluation on a Jetson Orin Nano demonstrates competitive latency and jitter performance compared to native ROS 2 deployments, with strong improvements for multi-sensor workloads and neural inference tasks. This work demonstrates the practicality of in-vehicle, lambda-based data processing for scalable, diverse autonomous driving training data, and outlines future directions in transport backends, learning-based relevance estimation, and security.

Abstract

Data is both the key enabler and a major bottleneck for machine learning in autonomous driving. Effective model training requires not only large quantities of sensor data but also balanced coverage that includes rare yet safety-critical scenarios. Capturing such events demands extensive driving time and efficient selection. This paper introduces the Lambda framework, an edge-native platform that enables on-vehicle data filtering and processing through user-defined functions. The framework provides a serverless-inspired abstraction layer that separates application logic from low-level execution concerns such as scheduling, deployment, and isolation. By adapting Function-as-a-Service (FaaS) principles to resource-constrained automotive environments, it allows developers to implement modular, event-driven filtering algorithms while maintaining compatibility with ROS 2 and existing data recording pipelines. We evaluate the framework on an NVIDIA Jetson Orin Nano and compare it against native ROS 2 deployments. Results show competitive performance, reduced latency and jitter, and confirm that lambda-based abstractions can support real-time data processing in embedded autonomous driving systems. The source code is available at https://github.com/LASFAS/jblambda.

Figures (5)

• Figure 1: System architecture overview. The orchestrator communicates with the cloud to fetch lambda functions and launches a dedicated runtime process for each function. Sensor data is delivered to the runtimes via DDS (ROS2), which also manages the data recorder.
• Figure 2: Architecture of the runtime's concurrency model. Two DDS receivers feed ring buffers in a many-producer, single-consumer pattern. The upper topic triggers event-driven execution, while periodic execution polls both buffers.
• Figure 3: Schematic of the Performance Evaluation Loop with Timestamp Alignment and RTT Measurement
• Figure 4: Results of RTT measurements for three distinct lambda functions implemented as native ROS 2 nodes and lambda functions; Lambdas demonstrate lower latency for the IMU FFT and YOLO functions and exhibit consistently reduced jitter; time scale is logarithmic.
• Figure 5: RAM and power consumption for the measurements displayed in figure \ref{['fig:rtt_measurements']}