Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-Improvement in Multimodal Large Language Models: A Survey

Shijian Deng, Kai Wang, Tianyu Yang, Harsh Singh, Yapeng Tian

TL;DR

Self-improvement in multimodal LLMs seeks to automate data collection, organization, and model optimization to recursively enhance multimodal capabilities while reducing human effort. The paper formalizes the process with a three-stage loop and a taxonomy of autonomy, and surveys seed models, data-collection strategies, verification, training methods (SFT, RL, DPO), and multimodal benchmarks. It provides meta-analysis across benchmarks to identify robust patterns, bottlenecks, and efficiency considerations, and discusses downstream applications and future directions. The work aims to guide future research toward scalable, highly autonomous self-improvement frameworks that can operate across diverse modalities and tasks.

Abstract

Recent advancements in self-improvement for Large Language Models (LLMs) have efficiently enhanced model capabilities without significantly increasing costs, particularly in terms of human effort. While this area is still relatively young, its extension to the multimodal domain holds immense potential for leveraging diverse data sources and developing more general self-improving models. This survey is the first to provide a comprehensive overview of self-improvement in Multimodal LLMs (MLLMs). We provide a structured overview of the current literature and discuss methods from three perspectives: 1) data collection, 2) data organization, and 3) model optimization, to facilitate the further development of self-improvement in MLLMs. We also include commonly used evaluations and downstream applications. Finally, we conclude by outlining open challenges and future research directions.

Self-Improvement in Multimodal Large Language Models: A Survey

TL;DR

Self-improvement in multimodal LLMs seeks to automate data collection, organization, and model optimization to recursively enhance multimodal capabilities while reducing human effort. The paper formalizes the process with a three-stage loop and a taxonomy of autonomy, and surveys seed models, data-collection strategies, verification, training methods (SFT, RL, DPO), and multimodal benchmarks. It provides meta-analysis across benchmarks to identify robust patterns, bottlenecks, and efficiency considerations, and discusses downstream applications and future directions. The work aims to guide future research toward scalable, highly autonomous self-improvement frameworks that can operate across diverse modalities and tasks.

Abstract

Recent advancements in self-improvement for Large Language Models (LLMs) have efficiently enhanced model capabilities without significantly increasing costs, particularly in terms of human effort. While this area is still relatively young, its extension to the multimodal domain holds immense potential for leveraging diverse data sources and developing more general self-improving models. This survey is the first to provide a comprehensive overview of self-improvement in Multimodal LLMs (MLLMs). We provide a structured overview of the current literature and discuss methods from three perspectives: 1) data collection, 2) data organization, and 3) model optimization, to facilitate the further development of self-improvement in MLLMs. We also include commonly used evaluations and downstream applications. Finally, we conclude by outlining open challenges and future research directions.

Paper Structure

This paper contains 60 sections, 9 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Overview
  3. Definition
  4. Related and Representative Works
  5. Self-Refinement and Peer-Improvement
  6. Self-Improvement in Image Large Language Models
  7. Extensions to Video
  8. Multimodal Agents
  9. Seed Models
  10. Data Collection
  11. Random Sampling
  12. Guided Data Generation
  13. Negative Samples
  14. Data Organization
  15. Verification Methods
  16. ...and 45 more sections

Figures (3)

  • Figure 1: An illustration of self-improvement in Multimodal Large Language Models. The process involves selecting a seed MLLM to generate new data, organizing it into a dataset (which can optionally guide further data collection), and finally obtaining an improved model through training. This process can be iterated to achieve recursive self-improvement.
  • Figure 2: An overview of three steps for self-improvement in MLLMs. Each step can involve different methods based on requirements. For the full taxonomy please check Figure \ref{['fig:tax']}.
  • Figure 3: The taxonomy of three steps for self-improvement in MLLMs. Each step can involve different methods based on requirements.