Delving into Cascaded Instability: A Lipschitz Continuity View on Image Restoration and Object Detection Synergy

TL;DR

The paper tackles instability in Image Restoration → Object Detection pipelines under adverse conditions by adopting a Lipschitz-continuity framework. It identifies a fundamental mismatch: restoration networks exhibit smoother mappings while detectors have sharp, discontinuous decision boundaries, leading to amplified perturbations and unstable training in cascades. To address this, it introduces Lipschitz-regularized object detection (LROD), combining low-Lipschitz restoration with parameter-space smoothing, implemented as LR-YOLO and integrated into YOLO detectors. Across haze and low-light benchmarks, the method achieves more stable optimization and higher detection accuracy, while remaining plug-and-play with existing detectors and generalizable to other backbones. This Lipschitz-informed approach offers a practical path to robust perception in challenging environments.

Abstract

To improve detection robustness in adverse conditions (e.g., haze and low light), image restoration is commonly applied as a pre-processing step to enhance image quality for the detector. However, the functional mismatch between restoration and detection networks can introduce instability and hinder effective integration -- an issue that remains underexplored. We revisit this limitation through the lens of Lipschitz continuity, analyzing the functional differences between restoration and detection networks in both the input space and the parameter space. Our analysis shows that restoration networks perform smooth, continuous transformations, while object detectors operate with discontinuous decision boundaries, making them highly sensitive to minor perturbations. This mismatch introduces instability in traditional cascade frameworks, where even imperceptible noise from restoration is amplified during detection, disrupting gradient flow and hindering optimization. To address this, we propose Lipschitz-regularized object detection (LROD), a simple yet effective framework that integrates image restoration directly into the detector's feature learning, harmonizing the Lipschitz continuity of both tasks during training. We implement this framework as Lipschitz-regularized YOLO (LR-YOLO), extending seamlessly to existing YOLO detectors. Extensive experiments on haze and low-light benchmarks demonstrate that LR-YOLO consistently improves detection stability, optimization smoothness, and overall accuracy.

Figures (10)

• Figure 1: Visualization of network functional behaviors under input perturbations. (a) Image Restoration networks exhibit smooth, continuous mappings, where input changes lead to gradual adjustments. (b) Object Detection networks display sharp discontinuities due to abrupt decision boundaries in classification and bounding box regression. (c) Cascade frameworks (Image Restoration $\rightarrow$ Object Detection) magnify instability, resulting in fragmented and non-smooth behavior. (d) Our method integrates low-Lipschitz image restoration into the feature learning of high-Lipschitz object detection, promoting smoother transitions and enhanced stability.
• Figure 2: Impact of haze density variations on feature stability and Lipschitz continuity. (a) Distribution of the detector backbone's features between two haze inputs $\boldsymbol{x}$ and $\boldsymbol{x}+\Delta \boldsymbol{x}$ under minor haze density variations $\Delta \boldsymbol{x}$ on Pascal VOC everingham2015pascal with synthetic haze. In the cascade framework, nearly half of the features shift under slight haze density variations, while our Lipschitz-regularized object detection remains stable. (b) Box-plot distribution of Jacobian norms $\|\nabla_{\boldsymbol{x}} f_{\mathbf{\theta}} (\boldsymbol{x})\|$ at each sample $\boldsymbol{x}$ between image restoration and object detection task on Pascal VOC everingham2015pascal with synthetic haze. The Lipschitz constant of the object detection network is nearly an order of magnitude larger than that of the restoration network. This large disparity in Lipschitz continuity between the two tasks exacerbates the non-smoothness in the cascade framework. Our method constrains the Lipschitz constant of object detection to harmonize these two tasks better. ConvIR cui2024revitalizing and YOLOv8 yolov8 are taken as restoration and detection methods, respectively.
• Figure 3: Parameter-space smoothness and optimization stability comparison between the cascade framework and our Lipschitz-regularized object detection. (a) Loss landscapes of restoration and detection tasks: restoration networks demonstrate smooth trajectories, while detection networks encounter sharp gradient transitions, indicating instability. (b) The cascade framework amplifies this imbalance, leading to inefficient convergence and oscillatory optimization paths. (c) Our method introduces Lipschitz regularization to smooth the parameter space of object detection, enhancing stability and harmonizing it with restoration. ConvIR cui2024revitalizing and YOLOv8 yolov8 are used as representative restoration and detection methods, respectively.
• Figure 4: Lipschitz regularization ablation study. We evaluate the effect of two Lipschitz regularization parts $\mathcal{L}_{\mathrm{res}}$ and $\| \nabla_{{\mathbf{\theta}}} f_{{\mathbf{\theta}}}(\boldsymbol{x}) \|$.
• Figure 5: Comparison of changes in Lipschitz continuity in both the input space and parameter space during training. Methods include ConvIR$\rightarrow$YOLOv8 (Cascade), YOLOv8 (Baseline), LR-YOLOv8* (only trained with $\mathcal{L}_{\mathrm{res}}$), and LR-YOLOv8 (trained with both $\mathcal{L}_{\mathrm{res}}$ and $\| \nabla_{{\mathbf{\theta}}} f_{{\mathbf{\theta}}}(\boldsymbol{x}) \|$).

Theorems & Definitions (5)

• Remark 1
• Remark 2
• Remark 3: Lipschitz Regularization via Low-Lipschitz Restoration
• Remark 4: Lipschitz Regularization via Parameter-Space Smoothing
• proof