Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Apriel-1.5-15b-Thinker

Shruthan Radhakrishna, Aman Tiwari, Aanjaneya Shukla, Masoud Hashemi, Rishabh Maheshwary, Shiva Krishna Reddy Malay, Jash Mehta, Pulkit Pattnaik, Saloni Mittal, Khalil Slimi, Kelechi Ogueji, Akintunde Oladipo, Soham Parikh, Oluwanifemi Bamgbose, Toby Liang, Ahmed Masry, Khyati Mahajan, Sai Rajeswar Mudumba, Vikas Yadav, Sathwik Tejaswi Madhusudhan, Torsten Scholak, Sagar Davasam, Srinivas Sunkara, Nicholas Chapados

TL;DR

Apriel-1.5-15B-Thinker tackles frontier-level multimodal reasoning with a compact, open-weight model suitable for on-premises deployment by emphasizing data-centric mid-training. It employs a three-stage pipeline—depth upscaling of a Pixtral-12B base, staged continual pretraining to build foundational and visual reasoning capabilities via synthetic data, and high-quality supervised fine-tuning with explicit reasoning traces—without reinforcement learning. The model achieves strong results across the Artificial Analysis Intelligence Index and multiple multimodal benchmarks, approaching larger models despite running on a single GPU. This work demonstrates that carefully designed mid-training curricula and data curation can close substantial capability gaps at moderate scale, and it provides open-source assets to accelerate research in privacy-preserving, cost-aware deployments.

Abstract

We present Apriel-1.5-15B-Thinker, a 15-billion parameter open-weights multimodal reasoning model that achieves frontier-level performance through training design rather than sheer scale. Starting from Pixtral-12B, we apply a progressive three-stage methodology: (1) depth upscaling to expand reasoning capacity without pretraining from scratch, (2) staged continual pre-training that first develops foundational text and vision understanding, then enhances visual reasoning through targeted synthetic data generation addressing spatial structure, compositional understanding, and fine-grained perception, and (3) high-quality text-only supervised fine-tuning on curated instruction-response pairs with explicit reasoning traces spanning mathematics, coding, science, and tool use. Notably, our model achieves competitive results without reinforcement learning or preference optimization, isolating the contribution of our data-centric continual pre-training approach. On the Artificial Analysis Intelligence Index, Apriel-1.5-15B-Thinker attains a score of 52, matching DeepSeek-R1-0528 despite requiring significantly fewer computational resources. Across ten image benchmarks, its performance is on average within five points of Gemini-2.5-Flash and Claude Sonnet-3.7, a key achievement for a model operating within single-GPU deployment constraints. Our results demonstrate that thoughtful mid-training 2 design can close substantial capability gaps without massive scale, making frontier-level multimodal reasoning accessible to organizations with limited infrastructure. We release the model checkpoint, all training recipes, and evaluation protocols under the MIT license to to advance open-source research.

Apriel-1.5-15b-Thinker

TL;DR

Apriel-1.5-15B-Thinker tackles frontier-level multimodal reasoning with a compact, open-weight model suitable for on-premises deployment by emphasizing data-centric mid-training. It employs a three-stage pipeline—depth upscaling of a Pixtral-12B base, staged continual pretraining to build foundational and visual reasoning capabilities via synthetic data, and high-quality supervised fine-tuning with explicit reasoning traces—without reinforcement learning. The model achieves strong results across the Artificial Analysis Intelligence Index and multiple multimodal benchmarks, approaching larger models despite running on a single GPU. This work demonstrates that carefully designed mid-training curricula and data curation can close substantial capability gaps at moderate scale, and it provides open-source assets to accelerate research in privacy-preserving, cost-aware deployments.

Abstract

We present Apriel-1.5-15B-Thinker, a 15-billion parameter open-weights multimodal reasoning model that achieves frontier-level performance through training design rather than sheer scale. Starting from Pixtral-12B, we apply a progressive three-stage methodology: (1) depth upscaling to expand reasoning capacity without pretraining from scratch, (2) staged continual pre-training that first develops foundational text and vision understanding, then enhances visual reasoning through targeted synthetic data generation addressing spatial structure, compositional understanding, and fine-grained perception, and (3) high-quality text-only supervised fine-tuning on curated instruction-response pairs with explicit reasoning traces spanning mathematics, coding, science, and tool use. Notably, our model achieves competitive results without reinforcement learning or preference optimization, isolating the contribution of our data-centric continual pre-training approach. On the Artificial Analysis Intelligence Index, Apriel-1.5-15B-Thinker attains a score of 52, matching DeepSeek-R1-0528 despite requiring significantly fewer computational resources. Across ten image benchmarks, its performance is on average within five points of Gemini-2.5-Flash and Claude Sonnet-3.7, a key achievement for a model operating within single-GPU deployment constraints. Our results demonstrate that thoughtful mid-training 2 design can close substantial capability gaps without massive scale, making frontier-level multimodal reasoning accessible to organizations with limited infrastructure. We release the model checkpoint, all training recipes, and evaluation protocols under the MIT license to to advance open-source research.

Paper Structure

This paper contains 27 sections, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Architecture and Model Upscaling
  3. Base Model
  4. Depth Upscaling
  5. Projection network realignment
  6. Training Setup
  7. Continual Pretraining (CPT)
  8. CPT Stage 1
  9. Foundational Reasoning and Multimodal Data
  10. Training Setup
  11. CPT Stage 2
  12. Targeted Visual Reasoning Data via Synthetic Augmentation
  13. Data Hygiene and Difficulty Control
  14. Training Setup
  15. Evaluating effectiveness of Stage-2
  16. ...and 12 more sections

Figures (5)

  • Figure 1: Apriel-1.5-15B-Thinker compared to the best open source LLMs on the Artificial Analysis Intelligence Index.
  • Figure 2: Apriel-1.5-15B-Thinker ranks first in Artificial Analysis Intelligence index among the SOTA small open-source models and delivers performance competitive to larger open-source and proprietary models (as of September 26th, 2025).
  • Figure 3: Artificial Analysis Intelligence Index vs. Total Parameters (log scale). Apriel-1.5-15B-Thinker lies in the "most attractive quadrant."
  • Figure 4: Average performance across the benchmark suite (higher is better). The chart aggregates scores from MMMU yue2023mmmu, MMMU-Pro yue2024mmmupro, LogicVista xiao2024logicvista, MathVision wang2024mathvision, MathVista lu2023mathvista, MathVerse zhang2024mathverse, MMStar chen2024mmstar, CharXiv wang2024charxiv, AI2D kembhavi2016ai2d, and BLINK fu2024blink.
  • Figure :