K-EXAONE Technical Report

TL;DR

K-EXAONE introduces a $236\mathrm{B}$ parameter Mixture-of-Experts multilingual model with a $256\mathrm{K}$-token context window, designed to close the gap between open-weight and closed-weight models. It combines a hybrid attention mechanism with a finely tuned MoE routing, a reworked $150\mathrm{K}$ vocabulary tokenizer using SuperBPE, and a two-stage context-length extension to support long-range inputs. The training pipeline features a three-stage pretraining, thinking-augmented data synthesis, two-stage length extension, post-training with SFT/RL/Preference learning, agentic tool use, and safety/data-compliance considerations, including K-AUT and KGC-Safety benchmarks. Evaluations across reasoning, agentic, general, Korean, multilingual, and safety tasks show K-EXAONE achieving competitive performance with strong long-context and multilingual capabilities, indicating significant potential for industrial and research applications. Overall, the work demonstrates scalable, safe, and multilingual foundation-model capabilities that can drive real-world AI applications with extended memory and robust reasoning.

Abstract

This technical report presents K-EXAONE, a large-scale multilingual language model developed by LG AI Research. K-EXAONE is built on a Mixture-of-Experts architecture with 236B total parameters, activating 23B parameters during inference. It supports a 256K-token context window and covers six languages: Korean, English, Spanish, German, Japanese, and Vietnamese. We evaluate K-EXAONE on a comprehensive benchmark suite spanning reasoning, agentic, general, Korean, and multilingual abilities. Across these evaluations, K-EXAONE demonstrates performance comparable to open-weight models of similar size. K-EXAONE, designed to advance AI for a better life, is positioned as a powerful proprietary AI foundation model for a wide range of industrial and research applications.

Figures (10)

• Figure 1: The main evaluation results of K-EXAONE across eight categories: world knowledge ([1]MMLU-Pro), math ([1]AIME 2025), coding ([1]LiveCodeBench v6), agentic tool use ([1]$\tau^2$-Bench), instruction following ([1]IFBench), Korean ([1]KoBALT), multilinguality ([1]MMMLU), and safety ([1]KGC-Safety). All models used in assessment are reasoning models. $\tau^2$-Bench scores are weighted average.
• Figure 2: An illustration of K-EXAONE model architecture. The model comprises a stack of MoE blocks, with only the first layer implemented as a dense layer for training stability. In each MoE block, eight routed experts are selected from a pool of 128 experts, together with one shared expert, resulting in a total of nine concurrently utilized experts per routing decision. An MTP-based auxiliary objective is applied during training to supervise the prediction of an additional +1 future token.
• Figure 3: Comparison of tokenizer efficiency, measured in bytes per token, between K-EXAONE and EXAONE 4.0 across diverse text domains.
• Figure 4: Prompt template used for multiple-choice questions.
• Figure 5: Prompt template used for Korean multiple-choice questions.