Rethinking Few-Shot Image Fusion: Granular Ball Priors Enable General-Purpose Deep Fusion

TL;DR

This work tackles few-shot, multi-modal image fusion by introducing Granular Ball Pixel Computation (GBPC) based on rough-set theory to generate incomplete priors. The GBFF framework couples this prior generator with a lightweight CNN via a dynamically weighted loss that distinguishes reliable (POS) and uncertain (BND) regions, enabling re-inference from limited data. Across MEF, MFF, VIF, and MIF tasks, GBFF achieves competitive or superior performance with only 10 training pairs and exhibits strong efficiency, highlighting high practical potential for deployment. The study also analyzes the benefits of incomplete priors, modality-awareness, and loss components, while noting limitations and proposing future extension to additional subspaces and tasks.

Abstract

In image fusion tasks, the absence of real fused images as priors forces most deep learning approaches to rely on large-scale paired datasets to extract global weighting features or to generate pseudo-supervised images through algorithmic constructions. Unlike previous methods, this work re-examines prior-guided learning under few-shot conditions by introducing rough set theory. We regard the traditional algorithm as a prior generator, while the network re-inferrs and adaptively optimizes the prior through a dynamic loss function, reducing the inference burden of the network and enabling effective few-shot learning.To provide the prior, we propose the Granular Ball Pixel Computation (GBPC) algorithm. GBPC models pixel pairs in a luminance subspace using meta-granular balls and mines intra-ball information at multiple granular levels. At the fine-grained level, sliding granular balls assign adaptive weights to individual pixels to produce pixel-level prior fusion. At the coarse-grained level, the algorithm performs split computation within a single image to estimate positive and boundary domain distributions, enabling modality awareness and prior confidence estimation, which dynamically guide the loss weighting.The network and the algorithmic prior are coupled through the loss function to form an integrated framework. Thanks to the dynamic weighting mechanism, the network can adaptively adjust to different priors during training, enhancing its perception and fusion capability across modalities. We name this framework GBFF (Granular Ball Fusion Framework). Experiments on four fusion tasks demonstrate that even with only ten training image pairs per task, GBFF achieves superior performance in both visual quality and model compactness. Code is available at: https://github.com/DMinjie/GBFF

Figures (12)

• Figure 1: Illustration of the Granular Ball Pixel Computing (GBPC) framework and visualization of the prior formation process.
• Figure 2: Illustration of the network architecture and the processing steps involved during the learning procedure.
• Figure 3: Visualization of the meta-granular ball capturing process in Granular Ball Pixel Computing for clearer understanding.
• Figure 4: Objective evaluation variations of the prior image under different values of $k$ and $\Delta d$.
• Figure 5: The prior is obtained from the fragment library using Granular Ball Pixel Computing, and the loss function is dynamically adjusted according to $r_{\mathrm{POS}}$ and $r_{\mathrm{BND}}$.