Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MOSAIC: Multi-Objective Slice-Aware Iterative Curation for Alignment

Yipu Dou, Wang Yang

Abstract

We study how to allocate a fixed supervised fine-tuning budget when three objectives must be balanced at once: multi-turn safety alignment, low over-refusal on benign boundary queries, and instruction following under verifiable constraints. We propose MOSAIC (Multi-Objective Slice-Aware Iterative Curation for Alignment), a multi-objective framework for closed-loop data mixture search built on a unified L1-L3 evaluation interface. MOSAIC turns slice-level failure profiles into executable data actions, including dataset-level mixture ratios, bucket-level weights, and focus criteria. Under a fixed 1M-token budget and five rounds of independent fine-tuning from the same base model, MOSAIC improves internal XGuard from 2.76 to 4.67 while keeping OrBench at 4.41 and IFEval at 3.65. The final Pareto solution also generalizes better than a random static LoRA baseline on independent attack, over-refusal, and capability tests, suggesting that structured failure diagnosis can serve as a practical control signal for budgeted data construction. Code is available at https://github.com/douyipu/mosaic.

MOSAIC: Multi-Objective Slice-Aware Iterative Curation for Alignment

Abstract

We study how to allocate a fixed supervised fine-tuning budget when three objectives must be balanced at once: multi-turn safety alignment, low over-refusal on benign boundary queries, and instruction following under verifiable constraints. We propose MOSAIC (Multi-Objective Slice-Aware Iterative Curation for Alignment), a multi-objective framework for closed-loop data mixture search built on a unified L1-L3 evaluation interface. MOSAIC turns slice-level failure profiles into executable data actions, including dataset-level mixture ratios, bucket-level weights, and focus criteria. Under a fixed 1M-token budget and five rounds of independent fine-tuning from the same base model, MOSAIC improves internal XGuard from 2.76 to 4.67 while keeping OrBench at 4.41 and IFEval at 3.65. The final Pareto solution also generalizes better than a random static LoRA baseline on independent attack, over-refusal, and capability tests, suggesting that structured failure diagnosis can serve as a practical control signal for budgeted data construction. Code is available at https://github.com/douyipu/mosaic.
Paper Structure (19 sections, 8 equations, 5 figures, 3 tables)

This paper contains 19 sections, 8 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. The MOSAIC Framework
  4. Problem Setting
  5. Unified L1--L3 Evaluation Interface
  6. From Failure Profiles to Data Actions
  7. Task-Specific Instantiations
  8. XGuard.
  9. OrBench.
  10. IFEval.
  11. Experimental Setup
  12. Training Pools and Protocol
  13. Evaluation and Hardware
  14. Results
  15. Closed-Loop Search under a Fixed Budget
  16. ...and 4 more sections

Figures (5)

  • Figure 1: Overview of MOSAIC. The agent updates the next mixture from slice-level failure profiles under a fixed token budget.
  • Figure 2: MOSAIC search trajectory under a fixed 1M-token budget. The proposal agent quickly converges on a stable macro mixture, while the number of selected windows still varies because the three datasets have different average lengths.
  • Figure 3: Controlled MOSAIC comparison for iterations 2--4. The macro mixture is identical, so the performance differences are attributable to bucket weights and focus criteria.
  • Figure 4: Diagnostic evidence from the final MOSAIC Pareto solution. The L1 score can be decomposed into concrete slice-level and atomic failure signals that directly inform data actions.
  • Figure 5: Granular external validation of the final MOSAIC solution. The Pareto model avoids the over-refusal spike and instruction-following collapse observed in random static mixing.