Rate-Distortion Analysis of Optically Passive Vision Compression

TL;DR

Addresses real-time transmission of high-volume vision data under bandwidth limits by proposing optically passive vision compression (OPVC) that uses an optically computed cosine transform with an event camera. The method couples an optical cosine transform with event-based encoding and is evaluated against a standalone event camera (SAEC) within a rate-distortion framework, defined as $R(D; \pi) = \inf_\delta \mathbb{E}_{I\sim\mathbb{P}}[K/(mnT)]$ and distortion measured by $MS$-SSIM. Simulations on the UVG dataset show OPVC consistently outperforms SAEC across resolutions, with the advantage growing as spatial resolution increases due to concentration of energy in low-frequency content. These results suggest a practical path toward low-complexity, high-speed vision compression for real-time robotic control, while future work should address sensor noise, more faithful frame-to-event mappings, and the role of initializing frames.

Abstract

The use of remote vision sensors for autonomous decision-making poses the challenge of transmitting high-volume visual data over resource-constrained channels in real-time. In robotics and control applications, many systems can quickly destabilize, which can exacerbate the issue by necessitating higher sampling frequencies. This work proposes a novel sensing paradigm in which an event camera observes the optically generated cosine transform of a visual scene, enabling high-speed, computation-free video compression inspired by modern video codecs. In this study, we simulate this optically passive vision compression (OPVC) scheme and compare its rate-distortion performance to that of a standalone event camera (SAEC). We find that the rate-distortion performance of the OPVC scheme surpasses that of the SAEC and that this performance gap increases as the spatial resolution of the event camera increases.

Figures (4)

• Figure 1: Block diagram of optically passive vision compression scheme.
• Figure 2: Illustration of the simulation methodology. The red triangles represent the log-intensity of the raw video data, $L(s)$. The solid red curve depicts the assumed continuous pixel variation between frames and the dotted blue curve represents the corresponding continuous reconstruction. The blue circles depict the frame-based reconstruction, $\Tilde{L}(s)$. The lower plot depicts the event function $E(s)$ defined in \ref{['eq:evFunc']}.
• Figure 3: Rate-distortion performance of SAEC and OPVC on UVG dataset over multiple spatial resolutions.
• Figure 4: Qualitative comparison of compression results. The top image is a frame from the original video, which depicts a jockey riding a horse to the left as the camera pans with him. The middle image is a frame reconstructed using an SAEC at 0.9 MS-SSIM. The bottom image is a frame reconstructed using an OPVC at 0.9 MS-SSIM.