Edge-Aligned Initialization of Kernels for Steered Mixture-of-Experts

TL;DR

The paper tackles the high computational cost of per-image gradient-based optimization in Steered Mixture of Experts (SMoE) by introducing a deterministic, edge-informed initialization. Using Canny edge detection to extract line segments, followed by line-segment clustering and orthogonal kernel placement, the method initializes kernel means and expert amplitudes with a brief gradient-free refinement, reducing memory and runtime. The approach yields improved PSNR/SSIM over a uniform grid and enables tile-based training with pruning and careful reassembly, achieving faster convergence while maintaining competitive reconstruction quality relative to state-of-the-art initializations. Overall, the edge-aligned initialization enhances scalability and efficiency for SMoE-based image representation tasks such as compression, denoising, and super-resolution.

Abstract

Steered Mixture-of-Experts (SMoE) has recently emerged as a powerful framework for spatial-domain image modeling, enabling high-fidelity image representation using a remarkably small number of parameters. Its ability to steer kernel-based experts toward structural image features has led to successful applications in image compression, denoising, super-resolution, and light field processing. However, practical adoption is hindered by the reliance on gradient-based optimization to estimate model parameters on a per-image basis - a process that is computationally intensive and difficult to scale. Initialization strategies for SMoE are an essential component that directly affects convergence and reconstruction quality. In this paper, we propose a novel, edge-based initialization scheme that achieves good reconstruction qualities while reducing the need for stochastic optimization significantly. Through a method that leverages Canny edge detection to extract a sparse set of image contours, kernel positions and orientations are deterministically inferred. A separate approach enables the direct estimation of initial expert coefficients. This initialization reduces both memory consumption and computational cost.

Figures (7)

• Figure 1: Overview of the proposed SMoE initialization pipeline. Structural features are extracted via Canny edge detection and converted to a compact line-segment representation. This guides kernel placement and enables direct, gradient-free initialization of expert coefficients. A final refinement via gradient descent produces high-quality reconstructions.
• Figure 2: (a) Canny edge mask. (b) Initial line segments. (c) Segments after reduction. (d) Final kernel centers.
• Figure 3: PSNR across expert initialization iterations.
• Figure 4: Visualization of the procedure used to train the SMoE models for the test images. As with the S-SMoE and the AS-SMoE, the image is split into tiles, before training a SMoE to represent each tile li2024li20242. Finally, the models are grouped by translating the kernel positions relative to the tile positions. The resulting model is then fine-tuned to represent the full image.
• Figure 5: PSNR vs. number of kernels for different initialization techniques. The proposed method outperforms the grid baseline for the most part. The proposed method yields slightly poorer results than the S-SMoE and AS-SMoE techniques proposed in li2024li20242.