An Open Software Suite for Event-Based Video

TL;DR

The paper addresses the lack of a cohesive software stack for event-based video by introducing ADΔER, a unified, camera-agnostic representation and accompanying open-source framework. It defines an event tuple $\\langle x,y,c,D,t\\rangle$ with intensity $\\frac{2^D}{\\Delta t}$ and provides modules for transcoding, rate-adaptive compression, metadata inspection, reconstructions to framed sequences, and an interactive GUI. Its contributions include the adder-codec-core encoder/decoder, adder-info metadata tools, a middleware layer for event generation and reconstruction, and the adder-viz interface for real-time exploration and application development, all designed to work across frame-based, DVS, and multimodal sources. By enabling forward compatibility with future event sensors and supporting neuromorphic workflows, the framework aims to accelerate research in event-based video, compression, and downstream vision applications.

Abstract

While traditional video representations are organized around discrete image frames, event-based video is a new paradigm that forgoes image frames altogether. Rather, pixel samples are temporally asynchronous and independent of one another. Until now, researchers have lacked a cohesive software framework for exploring the representation, compression, and applications of event-based video. I present the AD$Δ$ER software suite to fill this gap. This framework includes utilities for transcoding framed and multimodal event-based video sources to a common representation, rate control mechanisms, lossy compression, application support, and an interactive GUI for transcoding and playback. In this paper, I describe these various software components and their usage.

Figures (4)

• Figure 1: Overview of the AD$\Delta$ER framework. Italicized names reflect the names of software packages in the Rust Package Registry. This figure is modified and expanded from an earlier version adder_framework to include recent additions, such as the CRF quality parameter and Prophesee camera support.
• Figure 2: Example output of the adder-info utility. The program reports metadata from the file header and calculates the dynamic range of the video.
• Figure 3: The adder-viz transcoder user interface. This short video clip was transcoded to AD$\Delta$ER at successively higher quality levels. As the CRF level decreases, the quality metrics improve (top graph), but the bitrate increases (bottom graph).
• Figure 4: The adder-viz player interface for AD$\Delta$ER video, with different visualization modes shown.