Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SiamGPT: Quality-First Fine-Tuning for Stable Thai Text Generation

Thittipat Pairatsuppawat, Abhibhu Tachaapornchai, Paweekorn Kusolsomboon, Chutikan Chaiwong, Thodsaporn Chay-intr, Kobkrit Viriyayudhakorn, Nongnuch Ketui, Aslan B. Wong

TL;DR

SiamGPT-32B introduces a Quality-First, supervised fine-tuning approach for Thai that couples translation-based instruction transfer with a Thai-adapted AutoIF constraint framework. By using a compact, high-fidelity corpus (~320k I-R pairs) derived from SystemChat-2.0 translations and AutoIF seeds, the model improves stability, instruction following, and multi-turn robustness without continual pretraining or large-scale Thai data collection. Evaluations on SEA-HELM show it achieving the strongest overall performance among open-weight Thai models in its size class, with notable gains in instruction adherence and dialogue robustness, though translation-based supervision introduces some NLG trade-offs. The work demonstrates that careful, constraint-aware supervision can address key generation-time failures in open-weights Thai LLMs and highlights avenues for grounding, domain specialization, and native-language optimization in future iterations.

Abstract

Open-weights large language models remain difficult to deploy for Thai due to unstable generation under complex instructions, despite strong English performance. To mitigate these limitations, We present SiamGPT-32B, an open-weights model based on Qwen3-32B, fine-tuned with a Quality-First strategy emphasizing curated supervision over data scale. The fine-tuning pipeline combines translated high-complexity English instruction data with a Thai-adapted AutoIF framework for instruction and linguistic constraints. Using supervised fine-tuning only, without continual pretraining or corpus expansion, SiamGPT-32B improves instruction adherence, multi-turn robustness, and linguistic stability. Evaluations on the SEA-HELM benchmark show that SiamGPT-32B achieves the strongest overall performance among similar-scale open-weights Thai models, with consistent gains in instruction following, multi-turn dialogue, and natural language understanding.

SiamGPT: Quality-First Fine-Tuning for Stable Thai Text Generation

TL;DR

SiamGPT-32B introduces a Quality-First, supervised fine-tuning approach for Thai that couples translation-based instruction transfer with a Thai-adapted AutoIF constraint framework. By using a compact, high-fidelity corpus (~320k I-R pairs) derived from SystemChat-2.0 translations and AutoIF seeds, the model improves stability, instruction following, and multi-turn robustness without continual pretraining or large-scale Thai data collection. Evaluations on SEA-HELM show it achieving the strongest overall performance among open-weight Thai models in its size class, with notable gains in instruction adherence and dialogue robustness, though translation-based supervision introduces some NLG trade-offs. The work demonstrates that careful, constraint-aware supervision can address key generation-time failures in open-weights Thai LLMs and highlights avenues for grounding, domain specialization, and native-language optimization in future iterations.

Abstract

Open-weights large language models remain difficult to deploy for Thai due to unstable generation under complex instructions, despite strong English performance. To mitigate these limitations, We present SiamGPT-32B, an open-weights model based on Qwen3-32B, fine-tuned with a Quality-First strategy emphasizing curated supervision over data scale. The fine-tuning pipeline combines translated high-complexity English instruction data with a Thai-adapted AutoIF framework for instruction and linguistic constraints. Using supervised fine-tuning only, without continual pretraining or corpus expansion, SiamGPT-32B improves instruction adherence, multi-turn robustness, and linguistic stability. Evaluations on the SEA-HELM benchmark show that SiamGPT-32B achieves the strongest overall performance among similar-scale open-weights Thai models, with consistent gains in instruction following, multi-turn dialogue, and natural language understanding.
Paper Structure (20 sections, 1 figure, 3 tables)

This paper contains 20 sections, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Approach
  3. Design Goal and Quality-First Strategy
  4. Data Curation Pipeline
  5. Translation-Based Instruction Transfer
  6. Thai-Adapted AutoIF Constraint Enforcement
  7. Final Corpus Construction and Ablation Findings
  8. Training Objective and Fine-Tuning Design
  9. Experimental Settings
  10. Training Corpus
  11. Training Framework and Infrastructure
  12. Optimization and Hyperparameters
  13. Evaluation Benchmarks
  14. Results
  15. Impact of Quality-First Fine-Tuning on Qwen3-32B
  16. ...and 5 more sections

Figures (1)

  • Figure 2: SiamGPT data curation pipeline. The framework follows a dual-stream design: (top) an LLM-based translation pipeline that converts high-quality English instruction datasets into Thai using gemma-3-27b-it; (bottom) a Thai-adapted AutoIF pipeline that generates and verifies instruction-following data from Thai seed instructions and executable test code. Outputs from both streams are merged into the final training corpus for supervised fine-tuning.