Learning Event-guided Exposure-agnostic Video Frame Interpolation via Adaptive Feature Blending

TL;DR

The paper addresses exposure-agnostic video frame interpolation by leveraging high-temporal-resolution events. It introduces two modules, Target-adaptive Event Sampling (TES) and Target-adaptive Importance Mapping (TIM), to enforce temporal constraints and adaptively blend features from consecutive frames for target-frame synthesis within the Event-based Double Integral (EDI) framework. The approach demonstrates strong improvements on synthetic and real-world datasets (GoPro, HighREV, RealBlur-DAVIS), achieving up to a few dB gains in PSNR over state-of-the-art methods and displaying superior temporal coherence. This work enhances practical VFI under blind exposure conditions and suggests a path toward resolution-agnostic fusion using future implicit representations.

Abstract

Exposure-agnostic video frame interpolation (VFI) is a challenging task that aims to recover sharp, high-frame-rate videos from blurry, low-frame-rate inputs captured under unknown and dynamic exposure conditions. Event cameras are sensors with high temporal resolution, making them especially advantageous for this task. However, existing event-guided methods struggle to produce satisfactory results on severely low-frame-rate blurry videos due to the lack of temporal constraints. In this paper, we introduce a novel event-guided framework for exposure-agnostic VFI, addressing this limitation through two key components: a Target-adaptive Event Sampling (TES) and a Target-adaptive Importance Mapping (TIM). Specifically, TES samples events around the target timestamp and the unknown exposure time to better align them with the corresponding blurry frames. TIM then generates an importance map that considers the temporal proximity and spatial relevance of consecutive features to the target. Guided by this map, our framework adaptively blends consecutive features, allowing temporally aligned features to serve as the primary cues while spatially relevant ones offer complementary support. Extensive experiments on both synthetic and real-world datasets demonstrate the effectiveness of our approach in exposure-agnostic VFI scenarios.

Figures (13)

• Figure 1: Schematic comparison with a prior event-guided exposure-agnostic VFI method. (a) EBFI EBFI restores sharp frames from a single blurry input via event modulation, but degrades as the target timestamp deviates due to the lack of temporal constraints. (b) Our method addresses this by explicitly leveraging temporal constraints via adaptive feature blending. (c) PSNR comparison on GoPro gopro shows that our method maintains more stable performance across varying timestamps.
• Figure 2: Overview of our framework. Given blurry frames $(I_0, I_1)$ with unknown exposures $(T_{e_0}, T_{e_1})$, stacked events $\mathbb{E}_N$ over $2T$, and target timestamp $\tau$, the model reconstructs the target frame $\hat{I}_\tau$. TES samples events around $\tau$ and unknown exposure, which are fused with frames. TIM generates an importance map $\omega_\tau$ to adaptively blend the fused features $(F_0, F_1)$ into $F_\tau$, which is then decoded into $\hat{I}_\tau$. Shared encoders are color-coded.
• Figure 3: Architectures of the TES (Sec. \ref{['ssec:tes']}) and TIM (Sec. \ref{['ssec:tim']}) modules.
• Figure 4: Qualitative results on RealBlur-DAVIS (real blur & events)
• Figure 5: Channel-wise importance score $\frac{\partial\mathcal{E}_{\tau \rightarrow [t_s, t_e]}}{\partial\mathbb{E}_N}$ indicating the sampling contribution of each time index. TES effectively selects events near the unknown exposure $T_e$ (green) and target timestamp $\tau$ (blue), regardless of whether $\tau$ lies outside the exposure window.