Adaptive Segmentation-Based Initialization for Steered Mixture of Experts Image Regression

TL;DR

The paper tackles the computational burden of gradient-descent optimization in kernel image regression by introducing an adaptive segmentation-based initialization for Steered-Mixture-of-Experts (SMoE) and steer-kernel RBFs. It presents a two-stage pipeline—Segmentation Reconstruction and Parameter Exportation—that locally optimizes kernel counts and parameters per image segment and then fuses these into a globally consistent initialization. Empirical results on grayscale images show substantial gains in both objective (PSNR/SSIM/LPIPS) and subjective quality, along with dramatic sparsity reductions (fewer kernels) and up to 50% reductions in global optimization run-time on multi-GPU setups. The approach emphasizes parallelism, interpretability, and potential applicability to higher-dimensional data and other kernel regression methods.

Abstract

Kernel image regression methods have shown to provide excellent efficiency in many image processing task, such as image and light-field compression, Gaussian Splatting, denoising and super-resolution. The estimation of parameters for these methods frequently employ gradient descent iterative optimization, which poses significant computational burden for many applications. In this paper, we introduce a novel adaptive segmentation-based initialization method targeted for optimizing Steered-Mixture-of Experts (SMoE) gating networks and Radial-Basis-Function (RBF) networks with steering kernels. The novel initialization method allocates kernels into pre-calculated image segments. The optimal number of kernels, kernel positions, and steering parameters are derived per segment in an iterative optimization and kernel sparsification procedure. The kernel information from "local" segments is then transferred into a "global" initialization, ready for use in iterative optimization of SMoE, RBF, and related kernel image regression methods. Results show that drastic objective and subjective quality improvements are achievable compared to widely used regular grid initialization, "state-of-the-art" K-Means initialization and previously introduced segmentation-based initialization methods, while also drastically improving the sparsity of the regression models. For same quality, the novel initialization results in models with around 50% reduction of kernels. In addition, a significant reduction of convergence time is achieved, with overall run-time savings of up to 50%. The segmentation-based initialization strategy itself admits heavy parallel computation; in theory, it may be divided into as many tasks as there are segments in the images. By accessing only four parallel GPUs, run-time savings of already 50% for initialization are achievable.

Figures (10)

• Figure 1: Illustration of edge-aware Steered Mixture of Experts (SMoE) modelling for compression and denoising at 0.43 bpp (bits per pixel).
• Figure 2: The overall pipeline of the proposed method. Two novel intermediate steps highlighted in green: Segmentation reconstruction and Parameter exportation are proposed to adaptively optimize the number of kernels as well as kernel parameters. The input data is segmented to dynamically adjust the kernels' number, location, and covariance in Segmentation reconstruction. This output then feeds into parameter exportation, where local parameters are aggregated, scaled, and prepared for global optimization, refining kernel parameters for efficient reconstruction.
• Figure 3: Illustration of Cropping and Rescaling in Segmentation Reconstruction. The output of the cropping process results in a series of images containing the segmented region along with its surrounding pixels. The segmented region is highlighted yellow within each cropped image. Subsequently, these cropped images are resized to a standardized dimension to serve as input data for the local SMoE model.
• Figure 4: Pixel intensity difference threshold $d_{Th}$ versus the width (=height) of the cropped segments. The generated size of the segments (measured as pixel width=height of a bounding box) is in the range between 20 and 40 for the different thresholds $d_{Th}$.
• Figure 5: Comparison of the reconstruction quality of Grid bochinski_regularized_2018, K-Means verhack_steered_2020, S-SMoE li_segmentation-based_2023, and AS-SMoE