Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AVS: A Computational and Hierarchical Storage System for Autonomous Vehicles

Yuxin Wang, Yuankai He, Weisong Shi

TL;DR

The paper tackles the lack of a general-purpose, queryable onboard storage system for autonomous vehicles by introducing AVS, a co-designed, hierarchical storage architecture that couples real-time computation with hot-cold storage and a lightweight metadata index. It develops use-guided, modality-aware reduction and compression (e.g., LiDAR voxel downsampling and perceptual image deduplication) and validates them end-to-end on real L4 traces, showing major footprint reductions and predictable performance. The authors benchmark filesystem choices (EXT4 vs XFS) and embedded databases (SQLite vs RocksDB), settle on a Raspberry Pi 5 prototype with 3 days of driving data, and demonstrate end-to-end ingest, archival, and retrieval with low tail latency. The work establishes storage as a first-class component in AV stacks and provides a blueprint for scalable, long-term on-board data management that underpins data-driven analytics and third-party applications.

Abstract

Autonomous vehicles (AVs) are evolving into mobile computing platforms, equipped with powerful processors and diverse sensors that generate massive heterogeneous data, for example 14 TB per day. Supporting emerging third-party applications calls for a general-purpose, queryable onboard storage system. Yet today's data loggers and storage stacks in vehicles fail to deliver efficient data storage and retrieval. This paper presents AVS, an Autonomous Vehicle Storage system that co-designs computation with a hierarchical layout: modality-aware reduction and compression, hot-cold tiering with daily archival, and a lightweight metadata layer for indexing. The design is grounded with system-level benchmarks on AV data that cover SSD and HDD filesystems and embedded indexing, and is validated on embedded hardware with real L4 autonomous driving traces. The prototype delivers predictable real-time ingest, fast selective retrieval, and substantial footprint reduction under modest resource budgets. The work also outlines observations and next steps toward more scalable and longer deployments to motivate storage as a first-class component in AV stacks.

AVS: A Computational and Hierarchical Storage System for Autonomous Vehicles

TL;DR

The paper tackles the lack of a general-purpose, queryable onboard storage system for autonomous vehicles by introducing AVS, a co-designed, hierarchical storage architecture that couples real-time computation with hot-cold storage and a lightweight metadata index. It develops use-guided, modality-aware reduction and compression (e.g., LiDAR voxel downsampling and perceptual image deduplication) and validates them end-to-end on real L4 traces, showing major footprint reductions and predictable performance. The authors benchmark filesystem choices (EXT4 vs XFS) and embedded databases (SQLite vs RocksDB), settle on a Raspberry Pi 5 prototype with 3 days of driving data, and demonstrate end-to-end ingest, archival, and retrieval with low tail latency. The work establishes storage as a first-class component in AV stacks and provides a blueprint for scalable, long-term on-board data management that underpins data-driven analytics and third-party applications.

Abstract

Autonomous vehicles (AVs) are evolving into mobile computing platforms, equipped with powerful processors and diverse sensors that generate massive heterogeneous data, for example 14 TB per day. Supporting emerging third-party applications calls for a general-purpose, queryable onboard storage system. Yet today's data loggers and storage stacks in vehicles fail to deliver efficient data storage and retrieval. This paper presents AVS, an Autonomous Vehicle Storage system that co-designs computation with a hierarchical layout: modality-aware reduction and compression, hot-cold tiering with daily archival, and a lightweight metadata layer for indexing. The design is grounded with system-level benchmarks on AV data that cover SSD and HDD filesystems and embedded indexing, and is validated on embedded hardware with real L4 autonomous driving traces. The prototype delivers predictable real-time ingest, fast selective retrieval, and substantial footprint reduction under modest resource budgets. The work also outlines observations and next steps toward more scalable and longer deployments to motivate storage as a first-class component in AV stacks.

Paper Structure

This paper contains 15 sections, 6 equations, 11 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Motivation and Storage Gap Analysis
  3. Vehicle Computing Storage Needs
  4. Storage Systems Gap Analysis
  5. AVS System Architecture
  6. Computational Design: Reduction and Compression
  7. Data Reduction
  8. Data Compression
  9. Low-level System Selection
  10. Filesystem Choosing and Benchmark
  11. Database Selection and Comparison
  12. Prototype and Implementation
  13. System Implementation
  14. System Performance
  15. Conclusion and Future Work

Figures (11)

  • Figure 1: Vehicle computing paradigm.
  • Figure 2: The autonomous vehicle storage system (AVS) architecture.
  • Figure 3: KISS-ICP performance. Red line: Absolute Trajector Error (baseline is 1.831 m). Blue line: Average Rotation Error (baseline is 0.0014 deg/m)
  • Figure 4: Number of points per frame comparison for different voxel leaf sizes and sequences. (Larger voxel leaf size, more aggressive data reduction)
  • Figure 5: CenterTrack performance with different deduplication settings. (a) Shows multiple object tracking and detection accuracy for cars and pedestrians (the larger the better). (b) Shows ID switches probability for cars and pedestrians (the smaller the better)
  • ...and 6 more figures