Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

InCoder-32B: Code Foundation Model for Industrial Scenarios

Jian Yang, Wei Zhang, Jiajun Wu, Junhang Cheng, Shawn Guo, Haowen Wang, Weicheng Gu, Yaxin Du, Joseph Li, Fanglin Xu, Yizhi Li, Lin Jing, Yuanbo Wang, Yuhan Gao, Ruihao Gong, Chuan Hao, Ran Tao, Aishan Liu, Tuney Zheng, Ganqu Cui, Zhoujun Li, Mingjie Tang, Chenghua Lin, Wayne Xin Zhao, Xianglong Liu, Ming Zhou, Bryan Dai, Weifeng Lv

Abstract

Recent code large language models have achieved remarkable progress on general programming tasks. Nevertheless, their performance degrades significantly in industrial scenarios that require reasoning about hardware semantics, specialized language constructs, and strict resource constraints. To address these challenges, we introduce InCoder-32B (Industrial-Coder-32B), the first 32B-parameter code foundation model unifying code intelligence across chip design, GPU kernel optimization, embedded systems, compiler optimization, and 3D modeling. By adopting an efficient architecture, we train InCoder-32B from scratch with general code pre-training, curated industrial code annealing, mid-training that progressively extends context from 8K to 128K tokens with synthetic industrial reasoning data, and post-training with execution-grounded verification. We conduct extensive evaluation on 14 mainstream general code benchmarks and 9 industrial benchmarks spanning 4 specialized domains. Results show InCoder-32B achieves highly competitive performance on general tasks while establishing strong open-source baselines across industrial domains.

InCoder-32B: Code Foundation Model for Industrial Scenarios

Abstract

Recent code large language models have achieved remarkable progress on general programming tasks. Nevertheless, their performance degrades significantly in industrial scenarios that require reasoning about hardware semantics, specialized language constructs, and strict resource constraints. To address these challenges, we introduce InCoder-32B (Industrial-Coder-32B), the first 32B-parameter code foundation model unifying code intelligence across chip design, GPU kernel optimization, embedded systems, compiler optimization, and 3D modeling. By adopting an efficient architecture, we train InCoder-32B from scratch with general code pre-training, curated industrial code annealing, mid-training that progressively extends context from 8K to 128K tokens with synthetic industrial reasoning data, and post-training with execution-grounded verification. We conduct extensive evaluation on 14 mainstream general code benchmarks and 9 industrial benchmarks spanning 4 specialized domains. Results show InCoder-32B achieves highly competitive performance on general tasks while establishing strong open-source baselines across industrial domains.
Paper Structure (99 sections, 27 figures, 6 tables)

This paper contains 99 sections, 27 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Scaling Industrial Data under Simulation Environments
  3. Chip Design
  4. GPU Optimization
  5. 3D Modeling
  6. Code Optimization
  7. Training Strategy
  8. Stage 1 Pre-Training
  9. Data Collection.
  10. Data Cleaning and Refinement.
  11. Stage 2 Mid-Training
  12. Context Extension
  13. Industrial Data Synthesis and Curation
  14. Synthetic Industrial Code QA.
  15. Agent Trajectories.
  16. ...and 84 more sections

Figures (27)

  • Figure 1: Scope of industrial code intelligence. InCoder-32B aims to serve as a unified foundation model for (left) general software development and (right) industrial programming domains. The model supports a spectrum of capabilities, ranging from general coding tasks such as code generation, agentic development, and tool use to industrial workloads including chip design, GPU kernel optimization, CAD/CAM modeling, and compiler-level optimization.
  • Figure 2: Comparison of CUDA grid configuration strategies for RMS Normalization on large spatial dimensions (512×512). (Left) Claude assigns spatial_size (262,144) to gridDim.y, exceeding the CUDA hardware limit of 65,535, resulting in an invalid configuration argument runtime error. (Right) InCoder-32B flattens all dimensions into a 1D grid, safely utilizing the gridDim.x limit and avoiding the hardware constraint violation.
  • Figure 3: The three-stage training pipeline of InCoder-32B. Pre-train performs data curation and enhancement, Mid-train constructs an industrial-aware foundation with progressive context scaling from 8K to 128K, and Post-train produces simulation-grounded SFT data across industrial domains.
  • Figure 4: The performance of models on industrial code benchmarks.
  • Figure 5: Error distribution of InCoder-32B across 9 industrial benchmarks. The center number indicates total failures per benchmark. Errors are color-coded by type: reds/oranges for compilation and syntax errors, blues for functional/logic errors, purples for format errors, greens for performance issues.
  • ...and 22 more figures