Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MINER: Mining the Underlying Pattern of Modality-Specific Neurons in Multimodal Large Language Models

Kaichen Huang, Jiahao Huo, Yibo Yan, Kun Wang, Yutao Yue, Xuming Hu

TL;DR

MINER, a transferable framework for mining modality-specific neurons (MSNs) in MLLMs, which comprises four stages: modality separation, importance score calculation, importance score aggregation, modality-specific neuron selection, and modality-specific neuron selection.

Abstract

In recent years, multimodal large language models (MLLMs) have significantly advanced, integrating more modalities into diverse applications. However, the lack of explainability remains a major barrier to their use in scenarios requiring decision transparency. Current neuron-level explanation paradigms mainly focus on knowledge localization or language- and domain-specific analyses, leaving the exploration of multimodality largely unaddressed. To tackle these challenges, we propose MINER, a transferable framework for mining modality-specific neurons (MSNs) in MLLMs, which comprises four stages: (1) modality separation, (2) importance score calculation, (3) importance score aggregation, (4) modality-specific neuron selection. Extensive experiments across six benchmarks and two representative MLLMs show that (I) deactivating ONLY 2% of MSNs significantly reduces MLLMs performance (0.56 to 0.24 for Qwen2-VL, 0.69 to 0.31 for Qwen2-Audio), (II) different modalities mainly converge in the lower layers, (III) MSNs influence how key information from various modalities converges to the last token, (IV) two intriguing phenomena worth further investigation, i.e., semantic probing and semantic telomeres. The source code is available at this URL.

MINER: Mining the Underlying Pattern of Modality-Specific Neurons in Multimodal Large Language Models

TL;DR

MINER, a transferable framework for mining modality-specific neurons (MSNs) in MLLMs, which comprises four stages: modality separation, importance score calculation, importance score aggregation, modality-specific neuron selection, and modality-specific neuron selection.

Abstract

In recent years, multimodal large language models (MLLMs) have significantly advanced, integrating more modalities into diverse applications. However, the lack of explainability remains a major barrier to their use in scenarios requiring decision transparency. Current neuron-level explanation paradigms mainly focus on knowledge localization or language- and domain-specific analyses, leaving the exploration of multimodality largely unaddressed. To tackle these challenges, we propose MINER, a transferable framework for mining modality-specific neurons (MSNs) in MLLMs, which comprises four stages: (1) modality separation, (2) importance score calculation, (3) importance score aggregation, (4) modality-specific neuron selection. Extensive experiments across six benchmarks and two representative MLLMs show that (I) deactivating ONLY 2% of MSNs significantly reduces MLLMs performance (0.56 to 0.24 for Qwen2-VL, 0.69 to 0.31 for Qwen2-Audio), (II) different modalities mainly converge in the lower layers, (III) MSNs influence how key information from various modalities converges to the last token, (IV) two intriguing phenomena worth further investigation, i.e., semantic probing and semantic telomeres. The source code is available at this URL.
Paper Structure (31 sections, 8 equations, 14 figures, 4 tables)

This paper contains 31 sections, 8 equations, 14 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Definitions of Modality, Sample and Dataset
  5. Relevant Transformer Concepts
  6. Framework of MINER
  7. Problem Analysis
  8. Stage 1: Modality Separation
  9. Stage 2: Importance Score Calculation
  10. Stage 3: Importance Score Aggregation
  11. Stage 4: Modality-Specific Neuron Selection
  12. Experiment
  13. Experiment Setup
  14. Main Results (RQ1)
  15. Neuron Functionality Principles (RQ2)
  16. ...and 16 more sections

Figures (14)

  • Figure 1: Comparison of Language-specific (a), Domain-specific (b), and our proposed Modality-specific Neuron detection and analysis framework, MINER (c).
  • Figure 2: Four stages of MINER. ❶ Prompt tokens are divided into modality sets before being input into the LLM. ❷ Each neuron computes an importance score for the tokens of each modality. ❸ Aggregate these values to compute the Importance Score Matrix (ISM), reflecting the modality-level importance of each neuron. ❹ Various selection methods, as detailed in \ref{['stage-4']}, are employed to identify modality-specific neurons for each modality.
  • Figure 3: A VQA demo.
  • Figure 4: Selection strategies.
  • Figure 5: (a) t-SNE plots for VocalSound, showcasing three masking settings: no masking, Mask(VocalSound), and complementary masking (from top to bottom). (b) Display the change ($\Delta$) in contribution scores between different token sets and the last token before and after masking, based on 100 samples (detail in \ref{['RQ2']}).
  • ...and 9 more figures