Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Iterative Self-Alignment for Radiology Report Generation

Ting Xiao, Lei Shi, Yang Zhang, HaoFeng Yang, Zhe Wang, Chenjia Bai

TL;DR

This work tackles data coverage limits in Radiology Report Generation by introducing Online Iterative Self-Alignment (OISA), a four-stage, self-driven post-training framework that continuously expands data quality through self-generation, self-evaluation, self-alignment, and self-iteration. A lightweight RRG model is conditioned on a one-hot weight vector $\mathbf{w}$ to capture multi-objective radiologist preferences, and optimized via Multi-Objective Direct Preference Optimization (MODPO) over progressively constructed preference datasets. Theoretical guarantees are provided under a linear reward assumption, including an estimation error bound and a sub-optimality bound for the multi-objective policy, which tighten as the model gathers higher-quality data in iterations. Empirically, OISA achieves state-of-the-art or competitive results on MIMIC-CXR and IU-Xray across multiple radiology and NLG metrics, producing well-balanced Pareto fronts that reflect robust multi-objective alignment while maintaining efficiency with a lightweight model.

Abstract

Radiology Report Generation (RRG) is an important research topic for relieving radiologist' heavy workload. Existing RRG models mainly rely on supervised fine-tuning (SFT) based on different model architectures using data pairs of radiological images and corresponding radiologist-annotated reports. Recent research has shifted focus to post-training improvements, aligning RRG model outputs with human preferences using reinforcement learning (RL). However, the limited data coverage of high-quality annotated data poses risks of overfitting and generalization. This paper proposes a novel Online Iterative Self-Alignment (OISA) method for RRG that consists of four stages: self-generation of diverse data, self-evaluation for multi-objective preference data,self-alignment for multi-objective optimization and self-iteration for further improvement. Our approach allows for generating varied reports tailored to specific clinical objectives, enhancing the overall performance of the RRG model iteratively. Unlike existing methods, our frame-work significantly increases data quality and optimizes performance through iterative multi-objective optimization. Experimental results demonstrate that our method surpasses previous approaches, achieving state-of-the-art performance across multiple evaluation metrics.

Online Iterative Self-Alignment for Radiology Report Generation

TL;DR

This work tackles data coverage limits in Radiology Report Generation by introducing Online Iterative Self-Alignment (OISA), a four-stage, self-driven post-training framework that continuously expands data quality through self-generation, self-evaluation, self-alignment, and self-iteration. A lightweight RRG model is conditioned on a one-hot weight vector to capture multi-objective radiologist preferences, and optimized via Multi-Objective Direct Preference Optimization (MODPO) over progressively constructed preference datasets. Theoretical guarantees are provided under a linear reward assumption, including an estimation error bound and a sub-optimality bound for the multi-objective policy, which tighten as the model gathers higher-quality data in iterations. Empirically, OISA achieves state-of-the-art or competitive results on MIMIC-CXR and IU-Xray across multiple radiology and NLG metrics, producing well-balanced Pareto fronts that reflect robust multi-objective alignment while maintaining efficiency with a lightweight model.

Abstract

Radiology Report Generation (RRG) is an important research topic for relieving radiologist' heavy workload. Existing RRG models mainly rely on supervised fine-tuning (SFT) based on different model architectures using data pairs of radiological images and corresponding radiologist-annotated reports. Recent research has shifted focus to post-training improvements, aligning RRG model outputs with human preferences using reinforcement learning (RL). However, the limited data coverage of high-quality annotated data poses risks of overfitting and generalization. This paper proposes a novel Online Iterative Self-Alignment (OISA) method for RRG that consists of four stages: self-generation of diverse data, self-evaluation for multi-objective preference data,self-alignment for multi-objective optimization and self-iteration for further improvement. Our approach allows for generating varied reports tailored to specific clinical objectives, enhancing the overall performance of the RRG model iteratively. Unlike existing methods, our frame-work significantly increases data quality and optimizes performance through iterative multi-objective optimization. Experimental results demonstrate that our method surpasses previous approaches, achieving state-of-the-art performance across multiple evaluation metrics.
Paper Structure (22 sections, 3 theorems, 20 equations, 3 figures, 8 tables)

This paper contains 22 sections, 3 theorems, 20 equations, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Methods
  4. Preference Datasets Construction
  5. Multi-Objective Alignment
  6. Self-Iteration
  7. Theoretical Analysis
  8. Experiments
  9. Datasets and Experiment Setting
  10. Results and Analysis
  11. Related Work
  12. Conclusion
  13. Limitations
  14. Acknowledgment
  15. Deduplication Rules in Self-Generation
  16. ...and 7 more sections

Key Result

Lemma 1

zhu2023principled For any $\lambda > 0$, letting $\gamma = 1/(2 + e^{-B} + e^{B})$, with probability at least $1 - \delta$, we have where $\Sigma_{\mathcal{D}_k} = \frac{1}{K} \sum\limits_{j = 1}^{K}( \phi(x_j, y_j^w) - \phi(x_j, y_j^l) )( \phi(x_j, y_j^w) - \phi(x_j, y_j^l) )^{\top}.$

Figures (3)

  • Figure 1: The illustration of the proposed OISA pipeline, comprising the Preference Dataset Construction (PDC) module and the Multi-Objective Alignment (MOA) module. The pipeline involves four steps: self-generation to obtain diverse data, self-evaluation to obtain a multi-objective preference dataset, self-alignment for multi-objective optimization, and self-iteration to further improve the performance of the RRG model.
  • Figure 2: The distribution of preference dataset on different evaluation metrics in three iterations.
  • Figure 3: The multi-objective alignment fronts across three iterations.

Theorems & Definitions (5)

  • Lemma 1
  • Theorem 1
  • Lemma 2
  • proof : Proof of Lemma \ref{['lemma:single_obj_subopt']}
  • proof : Proof of Theorem \ref{['thm:subop_gap']}