Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MENLO: From Preferences to Proficiency -- Evaluating and Modeling Native-like Quality Across 47 Languages

Chenxi Whitehouse, Sebastian Ruder, Tony Lin, Oksana Kurylo, Haruka Takagi, Janice Lam, Nicolò Busetto, Denise Diaz, Francisco Guzmán

TL;DR

MENLO presents a scalable framework for evaluating native-like multilingual output by decomposing quality into four dimensions and using audience-design prompts. It delivers the Menlo dataset (6,423 prompt–response pairs across 47 languages) with high annotator reliability, demonstrates the superiority of pairwise LLM judgments and rubric-grounded evaluation, and shows that reinforcement learning with shaped rewards can train judges that rival human annotators. Moreover, RL-trained judges can serve as generative reward models to post-train policy models, improving multilingual proficiency, though gaps between automated and human judgments remain. The work unifies evaluation and optimization into a practical framework and releases the dataset and tools to advance multilingual LLM evaluation and alignment.

Abstract

Ensuring native-like quality of large language model (LLM) responses across many languages is challenging. To address this, we introduce MENLO, a framework that operationalizes the evaluation of native-like response quality based on audience design-inspired mechanisms. Using MENLO, we create a dataset of 6,423 human-annotated prompt-response preference pairs covering four quality dimensions with high inter-annotator agreement in 47 language varieties. Our evaluation reveals that zero-shot LLM judges benefit significantly from pairwise evaluation and our structured annotation rubrics, yet they still underperform human annotators on our dataset. We demonstrate substantial improvements through fine-tuning with reinforcement learning, reward shaping, and multi-task learning approaches. Additionally, we show that RL-trained judges can serve as generative reward models to enhance LLMs' multilingual proficiency, though discrepancies with human judgment remain. Our findings suggest promising directions for scalable multilingual evaluation and preference alignment. We release our dataset and evaluation framework to support further research in multilingual LLM evaluation.

MENLO: From Preferences to Proficiency -- Evaluating and Modeling Native-like Quality Across 47 Languages

TL;DR

MENLO presents a scalable framework for evaluating native-like multilingual output by decomposing quality into four dimensions and using audience-design prompts. It delivers the Menlo dataset (6,423 prompt–response pairs across 47 languages) with high annotator reliability, demonstrates the superiority of pairwise LLM judgments and rubric-grounded evaluation, and shows that reinforcement learning with shaped rewards can train judges that rival human annotators. Moreover, RL-trained judges can serve as generative reward models to post-train policy models, improving multilingual proficiency, though gaps between automated and human judgments remain. The work unifies evaluation and optimization into a practical framework and releases the dataset and tools to advance multilingual LLM evaluation and alignment.

Abstract

Ensuring native-like quality of large language model (LLM) responses across many languages is challenging. To address this, we introduce MENLO, a framework that operationalizes the evaluation of native-like response quality based on audience design-inspired mechanisms. Using MENLO, we create a dataset of 6,423 human-annotated prompt-response preference pairs covering four quality dimensions with high inter-annotator agreement in 47 language varieties. Our evaluation reveals that zero-shot LLM judges benefit significantly from pairwise evaluation and our structured annotation rubrics, yet they still underperform human annotators on our dataset. We demonstrate substantial improvements through fine-tuning with reinforcement learning, reward shaping, and multi-task learning approaches. Additionally, we show that RL-trained judges can serve as generative reward models to enhance LLMs' multilingual proficiency, though discrepancies with human judgment remain. Our findings suggest promising directions for scalable multilingual evaluation and preference alignment. We release our dataset and evaluation framework to support further research in multilingual LLM evaluation.

Paper Structure

This paper contains 32 sections, 2 equations, 17 figures, 23 tables.

Table of Contents

  1. Introduction
  2. The Menlo Dataset
  3. Evaluating LLM-Judges on Menlo
  4. Pointwise vs. Pairwise
  5. With and without Grading Rubrics
  6. Training LLM-Judges on Menlo
  7. Reward Designs for RL
  8. Overall Performance: SFT vs. RL
  9. Ablation of RL Rewards
  10. Per-Dimension Performance and Single vs. Multi-Task
  11. Cross-Language Performance
  12. From LLM-Judges to Reward Models
  13. RL with Judges as Generative Reward Models
  14. Two-Stage Evaluation Strategy
  15. Related Work
  16. ...and 17 more sections

Figures (17)

  • Figure 1: Menlo framework and annotation process. 1) Human-written prompt templates evoking local contexts are created in English for the four dimensions. 2) Prompt templates are translated and localized into 47 language varieties. 3) Annotation guidelines are created that break down each dimension into easy-to-follow rubrics. 4) LLMs are used to generate response pairs for each prompt, which are annotated with Likert-scale ratings and preferences.
  • Figure 2: Dimensions of native-like response quality in Menlo and example prompt (template).
  • Figure 3: Preference Accuracy per Language of pairwise RL-trained Qwen3-4B.
  • Figure 4: Annotation interface used for Menlo.
  • Figure 5: Example of 5-Point Grading Rubrics for Localized Tone (Formality & politeness).
  • ...and 12 more figures