A Label-Free and Non-Monotonic Metric for Evaluating Denoising in Event Cameras
Chenyang Shi, Shasha Guo, Boyi Wei, Hanxiao Liu, Yibo Zhang, Ningfang Song, Jing Jin
TL;DR
This work introduces AOCC, a label-free and non-monotonic evaluation metric for denoising in event cameras, grounded in the area under the continuous contrast curve (CCC). By leveraging edge-contour–driven contrast captured across multiple time windows, AOCC jointly rewards effective noise removal while preserving genuine edge events, avoiding the biases of label-dependent and monotonic metrics. The authors define event-frame contrast, construct the CCC, and compute AOCC, validating the approach theoretically and through experiments on synthetic and real datasets. AOCC proves effective for parameter selection, offers a unified, label-free benchmark, and shows advantages over ESR and traditional metrics in identifying meaningful denoising performance. The metric promises practical impact for developing robust event-camera denoising methods applicable to real-world data.
Abstract
Event cameras are renowned for their high efficiency due to outputting a sparse, asynchronous stream of events. However, they are plagued by noisy events, especially in low light conditions. Denoising is an essential task for event cameras, but evaluating denoising performance is challenging. Label-dependent denoising metrics involve artificially adding noise to clean sequences, complicating evaluations. Moreover, the majority of these metrics are monotonic, which can inflate scores by removing substantial noise and valid events. To overcome these limitations, we propose the first label-free and non-monotonic evaluation metric, the area of the continuous contrast curve (AOCC), which utilizes the area enclosed by event frame contrast curves across different time intervals. This metric is inspired by how events capture the edge contours of scenes or objects with high temporal resolution. An effective denoising method removes noise without eliminating these edge-contour events, thus preserving the contrast of event frames. Consequently, contrast across various time ranges serves as a metric to assess denoising effectiveness. As the time interval lengthens, the curve will initially rise and then fall. The proposed metric is validated through both theoretical and experimental evidence.