Optimizing Multimodal Language Models through Attention-based Interpretability

TL;DR

This work tackles the interpretability of multimodal language models and the efficiency of fine-tuning by introducing Head Impact (HI), an IoU-based measure that identifies attention heads focusing on image key objects. By analyzing per-head, per-layer attention between language responses and image tokens, the authors select layers for parameter-efficient fine-tuning (using LoRA) and demonstrate that tuning the top HI layers yields the most substantial performance shifts on image-captioning and visual-question-answering tasks. They validate the approach on 2–3B-parameter MLMs and provide a dataset of COCO-derived images with key-object masks and descriptions to support interpretation and fine-tuning experiments. The findings suggest that adjusting a tiny fraction of parameters (around 0.01%) in crucial heads can significantly influence image understanding, offering a scalable path to improve multimodal models while enhancing interpretability.

Abstract

Modern large language models become multimodal, analyzing various data formats like text and images. While fine-tuning is effective for adapting these multimodal language models (MLMs) to downstream tasks, full fine-tuning is computationally expensive. Parameter-Efficient Fine-Tuning (PEFT) methods address this by training only a small portion of model weights. However, MLMs are difficult to interpret, making it challenging to identify which components are most effective for training to balance efficiency and performance. We propose an attention-based interpretability method for MLMs by analyzing attention scores relative to image tokens. The core idea is to identify attention heads that focus on image key objects. We utilize this information to select optimal model components for PEFT in multimodal models. Our contributions include a method for identifying attention heads associated with image key objects, its application to PEFT for image captioning, and the creation of a new dataset containing images, key object masks, and their textual descriptions. We conducted experiments on MLMs with 2-3 billion parameters to validate the method's effectiveness. By calculating Head Impact (HI) scores we quantify an attention head's focus on key objects, indicating its significance in image understanding. Our fine-tuning experiments demonstrate that adapting layers with the highest HI scores leads to the most significant shifts in metrics compared to pre-trained, randomly selected, or lowest-HI-score layers. This indicates that fine-tuning a small percentage (around 0.01%) of parameters in these crucial layers can substantially influence image understanding capabilities.

Figures (5)

• Figure 1: An example of a dataset element. Left: original image. Middle: image key object mask. Right: text caption of the image key object
• Figure 2: Example of selecting key objects in an image. Left: original image. Middle: image key object mask. Right: representation of the mask as a sequence of visual tokens with fixed size image patches
• Figure 3: The architecture of calculating attention scores. Left: Representation of the attention mechanism within an LLM. To calculate attention scores, queries are taken from the object caption tokens ([CAP]) in the Assistant prompt and keys and values from the visual tokens ([VIS]) in the User prompt. Right: The process of computing average scores for an image token. The calculated attention scores (AS) are averaged for each caption token from the Assistant prompt.
• Figure 4: Head Impact scores for the models.
• Figure 5: Average Head Impact scores. Solid line with dots --- scores for the layer/head, dashed line --- regression line, filled area --- standard error. Top: layer-wise averaged scores. Bottom: head-wise averaged scores.