Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HyperCLOVA X 32B Think

NAVER Cloud HyperCLOVA X Team

TL;DR

HyperCLOVA X 32B Think (THINK) introduces a Korean-centric, decoder-only vision-language model with strong reasoning and agentic capabilities. It employs a unified text-vision Transformer with a Korean-tailored tokenizer and a ViT-based vision encoder, trained via a four-stage curriculum and a post-training RL loop (SFT and multiple RL objectives) to align with human preferences. Across Korean text-to-text, vision-to-text, and agent benchmarks, THINK achieves competitive performance and demonstrates robust agentic behavior, while revealing English-language limitations and insights into catastrophic forgetting from sequential modality expansion. The model is released under an open-weight license, signaling practical impact for both academic and industry users and motivating further research, including joint multimodal training approaches such as HyperCLOVA X 8B Omni.

Abstract

In this report, we present HyperCLOVA X 32B Think, a vision-language model designed with particular emphasis on reasoning within the Korean linguistic and cultural context, as well as agentic ability. HyperCLOVA X 32B Think is pre-trained with a strong focus on reasoning capabilities and subsequently post-trained to support multimodal understanding, enhanced reasoning, agentic behaviors, and alignment with human preferences. Experimental evaluations against comparably sized models demonstrate that our model achieves strong performance on Korean text-to-text and vision-to-text benchmarks, as well as on agent-oriented evaluation tasks. By open-sourcing HyperCLOVA X 32B Think, we aim to support broader adoption and facilitate further research and innovation across both academic and industrial communities.

HyperCLOVA X 32B Think

TL;DR

HyperCLOVA X 32B Think (THINK) introduces a Korean-centric, decoder-only vision-language model with strong reasoning and agentic capabilities. It employs a unified text-vision Transformer with a Korean-tailored tokenizer and a ViT-based vision encoder, trained via a four-stage curriculum and a post-training RL loop (SFT and multiple RL objectives) to align with human preferences. Across Korean text-to-text, vision-to-text, and agent benchmarks, THINK achieves competitive performance and demonstrates robust agentic behavior, while revealing English-language limitations and insights into catastrophic forgetting from sequential modality expansion. The model is released under an open-weight license, signaling practical impact for both academic and industry users and motivating further research, including joint multimodal training approaches such as HyperCLOVA X 8B Omni.

Abstract

In this report, we present HyperCLOVA X 32B Think, a vision-language model designed with particular emphasis on reasoning within the Korean linguistic and cultural context, as well as agentic ability. HyperCLOVA X 32B Think is pre-trained with a strong focus on reasoning capabilities and subsequently post-trained to support multimodal understanding, enhanced reasoning, agentic behaviors, and alignment with human preferences. Experimental evaluations against comparably sized models demonstrate that our model achieves strong performance on Korean text-to-text and vision-to-text benchmarks, as well as on agent-oriented evaluation tasks. By open-sourcing HyperCLOVA X 32B Think, we aim to support broader adoption and facilitate further research and innovation across both academic and industrial communities.
Paper Structure (51 sections, 3 figures, 8 tables)

This paper contains 51 sections, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. HyperCLOVA X 32B Think (THINK)
  3. Architecture.
  4. Text-Tokenization.
  5. Vision-Encoder.
  6. Pre‑Training
  7. Data Preparation
  8. Data Filtering.
  9. Synthetic Data Generation.
  10. Pre-Training Curriculum
  11. Stage 1: Foundation Knowledge Construction.
  12. Stage 2: Context Extension and Quality Up-sampling.
  13. Stage 3: Advanced Reasoning and Long-Context Adaptation.
  14. Stage 4: High-Quality Annealing.
  15. Post-Training
  16. ...and 36 more sections

Figures (3)

  • Figure 1: Overview of HyperCLOVA X 32B Think. Text is tokenized into discrete tokens and mapped to token embeddings, while images are encoded into continuous visual embeddings by the vision encoder. Visual embeddings are projected to the Transformer embedding space and interleaved with text embeddings, enabling joint multimodal processing with a single decoder-only Transformer.
  • Figure 2: Training pipeline of HyperCLOVA X 32B Think.
  • Figure 3: Multimodal Example.