Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neurodynamics-Driven Coupled Neural P Systems for Multi-Focus Image Fusion

Bo Li, Yunkuo Lei, Tingting Bao, Hang Yan, Yaxian Wang, Weiping Fu, Lingling Zhang, Jun Liu

Abstract

Multi-focus image fusion (MFIF) is a crucial technique in image processing, with a key challenge being the generation of decision maps with precise boundaries. However, traditional methods based on heuristic rules and deep learning methods with black-box mechanisms are difficult to generate high-quality decision maps. To overcome this challenge, we introduce neurodynamics-driven coupled neural P (CNP) systems, which are third-generation neural computation models inspired by spiking mechanisms, to enhance the accuracy of decision maps. Specifically, we first conduct an in-depth analysis of the model's neurodynamics to identify the constraints between the network parameters and the input signals. This solid analysis avoids abnormal continuous firing of neurons and ensures the model accurately distinguishes between focused and unfocused regions, generating high-quality decision maps for MFIF. Based on this analysis, we propose a Neurodynamics-Driven CNP Fusion model (ND-CNPFuse) tailored for the challenging MFIF task. Unlike current ideas of decision map generation, ND-CNPFuse distinguishes between focused and unfocused regions by mapping the source image into interpretable spike matrices. By comparing the number of spikes, an accurate decision map can be generated directly without any post-processing. Extensive experimental results show that ND-CNPFuse achieves new state-of-the-art performance on four classical MFIF datasets, including Lytro, MFFW, MFI-WHU, and Real-MFF. The code is available at https://github.com/MorvanLi/ND-CNPFuse.

Neurodynamics-Driven Coupled Neural P Systems for Multi-Focus Image Fusion

Abstract

Multi-focus image fusion (MFIF) is a crucial technique in image processing, with a key challenge being the generation of decision maps with precise boundaries. However, traditional methods based on heuristic rules and deep learning methods with black-box mechanisms are difficult to generate high-quality decision maps. To overcome this challenge, we introduce neurodynamics-driven coupled neural P (CNP) systems, which are third-generation neural computation models inspired by spiking mechanisms, to enhance the accuracy of decision maps. Specifically, we first conduct an in-depth analysis of the model's neurodynamics to identify the constraints between the network parameters and the input signals. This solid analysis avoids abnormal continuous firing of neurons and ensures the model accurately distinguishes between focused and unfocused regions, generating high-quality decision maps for MFIF. Based on this analysis, we propose a Neurodynamics-Driven CNP Fusion model (ND-CNPFuse) tailored for the challenging MFIF task. Unlike current ideas of decision map generation, ND-CNPFuse distinguishes between focused and unfocused regions by mapping the source image into interpretable spike matrices. By comparing the number of spikes, an accurate decision map can be generated directly without any post-processing. Extensive experimental results show that ND-CNPFuse achieves new state-of-the-art performance on four classical MFIF datasets, including Lytro, MFFW, MFI-WHU, and Real-MFF. The code is available at https://github.com/MorvanLi/ND-CNPFuse.

Paper Structure

This paper contains 17 sections, 5 theorems, 20 equations, 8 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. DL-based MFIF methods
  4. Coupled neural P systems
  5. Neurodynamics-Driven CNP Systems
  6. Mathematical model of CNP systems
  7. Why neurodynamics analysis is necessary?
  8. Methodology
  9. Problem statement
  10. Overview of the architecture
  11. Experiments
  12. Experimental configurations
  13. Performance comparison
  14. Parameter sensitivity analysis
  15. Ablation study
  16. ...and 2 more sections

Key Result

Theorem 1

For the feeding input unit U, the value increases with iterations and is represented as

Figures (8)

  • Figure 1: Workflow comparison of the existing decision method and the proposed ND-CNPFuse. ND-CNPFuse encodes source images into spike matrices and generates the decision map by comparing spike counts. Focused regions produce more spikes than unfocused ones, consistent with human visual perception jin2021analyze.
  • Figure 2: CNP neuron firing states under different inputs. (a) $I=0.5$; (b) $I=1.0$; (c) $I=1.5$; and (d) $I=2.0$. When $I$ exceeds the continuous firing condition, the neuron exhibits an abnormal state, as observed in (c) and (d).
  • Figure 3: Overview of ND-CNPFuse for MFIF. Fused image F quality primarily relies on the decision map generated by the ND-CNP system. A and B are the multi-focus source images. $\boldsymbol{\otimes}$ and $\boldsymbol{\oplus}$ denote element-wise multiplication and addition.
  • Figure 4: Working principle of the ND-CNP system. $M$ and $N$ denote image height and width. Please zoom in for a better view.
  • Figure 5: Qualitative results on "MFFW-4" from MFFW, with red and green boxes zoomed in 4 times for easy observation.
  • ...and 3 more figures

Theorems & Definitions (9)

  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Theorem 4
  • proof
  • Corollary 1