Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

KAT-Coder Technical Report

Zizheng Zhan, Ken Deng, Jinghui Wang, Xiaojiang Zhang, Huaixi Tang, Minglei Zhang, Zhiyi Lai, Haoyang Huang, Wen Xiang, Kun Wu, Wenhao Zhuang, Shaojie Wang, Shangpeng Yan, Kepeng Lei, Zongxian Feng, Huiming Wang, Zheng Lin, Mengtong Li, Mengfei Xie, Yinghan Cui, Xuxing Chen, Chao Wang, Weihao Li, Wenqiang Zhu, Jiarong Zhang, Jingxuan Xu, Songwei Yu, Yifan Yao, Xinping Lei, C. Zhang, Han Li, Junqi Xiong, Zuchen Gao, Dailin Li, Haimo Li, Jiaheng Liu, Yuqun Zhang, Junyi Peng, Haotian Zhang, Bin Chen

TL;DR

KAT-Coder addresses the gap between static pretraining and real-world agentic coding by proposing a four-stage curriculum that progressively builds reasoning, tool use, and deployment robustness. The approach combines Mid-Term Training with real-world corpora and synthetic traces, broad SFT across 20+ languages and 10 contexts with 10 task archetypes, a reinforcement-to-deployment adaptation using EM-SFT and Tree-Structured Trajectory Training, and Agentic RL with Trie-Packed Training and entropy-aware advantage rescaling. Empirical results show strong, balanced performance across instruction following, tool invocation, reasoning, and code tasks, with competitive rankings against proprietary models and successful production-adaptation signals. The open-sourcing of KAT-Dev (32B) on HuggingFace supports reproducibility and practical adoption, signaling a path to deployable intelligent coding agents capable of long-horizon planning and multi-tool collaboration in real IDEs.

Abstract

Recent advances in large language models (LLMs) have enabled progress in agentic coding, where models autonomously reason, plan, and act within interactive software development workflows. However, bridging the gap between static text-based training and dynamic real-world agentic execution remains a core challenge. In this technical report, we present KAT-Coder, a large-scale agentic code model trained through a multi-stage curriculum encompassing Mid-Term Training, Supervised Fine-Tuning (SFT), Reinforcement Fine-Tuning (RFT), and Reinforcement-to-Deployment Adaptation. The Mid-Term stage enhances reasoning, planning, and reflection capabilities through a corpus of real software engineering data and synthetic agentic interactions. The SFT stage constructs a million-sample dataset balancing twenty programming languages, ten development contexts, and ten task archetypes. The RFT stage introduces a novel multi-ground-truth reward formulation for stable and sample-efficient policy optimization. Finally, the Reinforcement-to-Deployment phase adapts the model to production-grade IDE environments using Error-Masked SFT and Tree-Structured Trajectory Training. In summary, these stages enable KAT-Coder to achieve robust tool-use reliability, instruction alignment, and long-context reasoning, forming a deployable foundation for real-world intelligent coding agents. Our KAT series 32B model, KAT-Dev, has been open-sourced on https://huggingface.co/Kwaipilot/KAT-Dev.

KAT-Coder Technical Report

TL;DR

KAT-Coder addresses the gap between static pretraining and real-world agentic coding by proposing a four-stage curriculum that progressively builds reasoning, tool use, and deployment robustness. The approach combines Mid-Term Training with real-world corpora and synthetic traces, broad SFT across 20+ languages and 10 contexts with 10 task archetypes, a reinforcement-to-deployment adaptation using EM-SFT and Tree-Structured Trajectory Training, and Agentic RL with Trie-Packed Training and entropy-aware advantage rescaling. Empirical results show strong, balanced performance across instruction following, tool invocation, reasoning, and code tasks, with competitive rankings against proprietary models and successful production-adaptation signals. The open-sourcing of KAT-Dev (32B) on HuggingFace supports reproducibility and practical adoption, signaling a path to deployable intelligent coding agents capable of long-horizon planning and multi-tool collaboration in real IDEs.

Abstract

Recent advances in large language models (LLMs) have enabled progress in agentic coding, where models autonomously reason, plan, and act within interactive software development workflows. However, bridging the gap between static text-based training and dynamic real-world agentic execution remains a core challenge. In this technical report, we present KAT-Coder, a large-scale agentic code model trained through a multi-stage curriculum encompassing Mid-Term Training, Supervised Fine-Tuning (SFT), Reinforcement Fine-Tuning (RFT), and Reinforcement-to-Deployment Adaptation. The Mid-Term stage enhances reasoning, planning, and reflection capabilities through a corpus of real software engineering data and synthetic agentic interactions. The SFT stage constructs a million-sample dataset balancing twenty programming languages, ten development contexts, and ten task archetypes. The RFT stage introduces a novel multi-ground-truth reward formulation for stable and sample-efficient policy optimization. Finally, the Reinforcement-to-Deployment phase adapts the model to production-grade IDE environments using Error-Masked SFT and Tree-Structured Trajectory Training. In summary, these stages enable KAT-Coder to achieve robust tool-use reliability, instruction alignment, and long-context reasoning, forming a deployable foundation for real-world intelligent coding agents. Our KAT series 32B model, KAT-Dev, has been open-sourced on https://huggingface.co/Kwaipilot/KAT-Dev.
Paper Structure (21 sections, 5 equations, 3 figures, 1 table)

This paper contains 21 sections, 5 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Mid-Term Training
  3. Training Recipe Overview
  4. Data Sources and Statistical Analysis
  5. Programming Language Dimension
  6. Development Context Dimension
  7. Task Type Dimension
  8. Dataset Scale and Distribution
  9. Reinforcement-to-Deployment Adaptation: Bridging Research Agents and Real-World Workflows
  10. Motivation
  11. Data Construction across Production Environments
  12. Training Challenges in Production-Grade Trajectories
  13. Methodology: Error-Masked SFT and Tree-Structured Trajectory Training
  14. RFT
  15. Training Stability and Sample Efficiency
  16. ...and 6 more sections

Figures (3)

  • Figure 1: KAT-Coder Training Pipeline
  • Figure 2: Method overview of RFT. (1)Trajectory Ground Truth: provides evaluation and correction signals for generated trajectories. (2) Trajectory Correction: performs online correction for outputs deviating from reference trajectories. (3) Rule-Based Test: generates stable and interpretable rule-based reward signals. (4) Group Computation: normalizes and aggregates group-level advantages within the GRPO framework.
  • Figure 3: Overview of Trie-Packed Training in agentic RL.