Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Chronicals: A High-Performance Framework for LLM Fine-Tuning with 3.51x Speedup over Unsloth

Arjun S. Nair

TL;DR

This work presents Chronicals, an open-source training framework achieving 3.51x speedup over Unsloth through four synergistic optimizations, and provides complete mathematical foundations: online softmax correctness proofs, FlashAttention IO complexity bounds O(N^2 d^2 M^{-1}), LoRA+ learning rate derivations from gradient magnitude analysis, and bin-packing approximation guarantees.

Abstract

Large language model fine-tuning is bottlenecked by memory: a 7B parameter model requires 84GB--14GB for weights, 14GB for gradients, and 56GB for FP32 optimizer states--exceeding even A100-40GB capacity. We present Chronicals, an open-source training framework achieving 3.51x speedup over Unsloth through four synergistic optimizations: (1) fused Triton kernels eliminating 75% of memory traffic via RMSNorm (7x), SwiGLU (5x), and QK-RoPE (2.3x) fusion; (2) Cut Cross-Entropy reducing logit memory from 5GB to 135MB through online softmax computation; (3) LoRA+ with theoretically-derived 16x differential learning rates between adapter matrices; and (4) Best-Fit Decreasing sequence packing recovering 60-75% of compute wasted on padding. On Qwen2.5-0.5B with A100-40GB, Chronicals achieves 41,184 tokens/second for full fine-tuning versus Unsloth's 11,736 tokens/second (3.51x). For LoRA at rank 32, we reach 11,699 tokens/second versus Unsloth MAX's 2,857 tokens/second (4.10x). Critically, we discovered that Unsloth's reported 46,000 tokens/second benchmark exhibited zero gradient norms--the model was not training. We provide complete mathematical foundations: online softmax correctness proofs, FlashAttention IO complexity bounds O(N^2 d^2 M^{-1}), LoRA+ learning rate derivations from gradient magnitude analysis, and bin-packing approximation guarantees. All implementations, benchmarks, and proofs are available at https://github.com/Ajwebdevs/Chronicals with pip installation via https://pypi.org/project/chronicals/.

Chronicals: A High-Performance Framework for LLM Fine-Tuning with 3.51x Speedup over Unsloth

TL;DR

This work presents Chronicals, an open-source training framework achieving 3.51x speedup over Unsloth through four synergistic optimizations, and provides complete mathematical foundations: online softmax correctness proofs, FlashAttention IO complexity bounds O(N^2 d^2 M^{-1}), LoRA+ learning rate derivations from gradient magnitude analysis, and bin-packing approximation guarantees.

Abstract

Large language model fine-tuning is bottlenecked by memory: a 7B parameter model requires 84GB--14GB for weights, 14GB for gradients, and 56GB for FP32 optimizer states--exceeding even A100-40GB capacity. We present Chronicals, an open-source training framework achieving 3.51x speedup over Unsloth through four synergistic optimizations: (1) fused Triton kernels eliminating 75% of memory traffic via RMSNorm (7x), SwiGLU (5x), and QK-RoPE (2.3x) fusion; (2) Cut Cross-Entropy reducing logit memory from 5GB to 135MB through online softmax computation; (3) LoRA+ with theoretically-derived 16x differential learning rates between adapter matrices; and (4) Best-Fit Decreasing sequence packing recovering 60-75% of compute wasted on padding. On Qwen2.5-0.5B with A100-40GB, Chronicals achieves 41,184 tokens/second for full fine-tuning versus Unsloth's 11,736 tokens/second (3.51x). For LoRA at rank 32, we reach 11,699 tokens/second versus Unsloth MAX's 2,857 tokens/second (4.10x). Critically, we discovered that Unsloth's reported 46,000 tokens/second benchmark exhibited zero gradient norms--the model was not training. We provide complete mathematical foundations: online softmax correctness proofs, FlashAttention IO complexity bounds O(N^2 d^2 M^{-1}), LoRA+ learning rate derivations from gradient magnitude analysis, and bin-packing approximation guarantees. All implementations, benchmarks, and proofs are available at https://github.com/Ajwebdevs/Chronicals with pip installation via https://pypi.org/project/chronicals/.
Paper Structure (178 sections, 47 theorems, 119 equations, 23 figures, 23 tables, 39 algorithms)

This paper contains 178 sections, 47 theorems, 119 equations, 23 figures, 23 tables, 39 algorithms.

Table of Contents

  1. Introduction
  2. Background and Theoretical Foundations
  3. Cut Cross-Entropy: Memory-Efficient Loss Computation
  4. Triton Kernel Implementations
  5. LoRA+ Optimizer: Differential Learning Rates
  6. FlashAttention and RoPE Optimizations
  7. FP8 Quantization and Sequence Packing
  8. Experimental Results
  9. Discussion
  10. Conclusion
  11. References
  12. Supplementary Material

Key Result

Proposition 1

The gradient with respect to queries is: where $P = \text{softmax}(QK^T/\sqrt{d})$ is the attention probability matrix and $O$ is the output.

Figures (23)

  • Figure 1: Throughput comparison across frameworks. Chronicals achieves 41,184 tokens/second for full fine-tuning with batch size 16, representing a 3.51x speedup over Unsloth's 11,736 tokens/second under identical conditions with verified gradient flow.
  • Figure 2: Speedup breakdown showing the relative performance gains of Chronicals across different training configurations. The chart demonstrates consistent speedups across full fine-tuning and LoRA training modes.
  • Figure 3: Radar chart comparing Chronicals and Unsloth across multiple dimensions: throughput, memory efficiency, MFU, LoRA speedup, and training correctness. Chronicals outperforms across all metrics.
  • Figure 4: Memory vs throughput scatter plot. Each point represents a framework configuration. Chronicals achieves the optimal trade-off: highest throughput with competitive memory usage.
  • Figure 5: Model FLOPs Utilization (MFU) comparison. Chronicals achieves 39.6% MFU compared to Unsloth's 11.3%, approaching theoretical hardware limits.
  • ...and 18 more figures

Theorems & Definitions (94)

  • Definition 1: Scaled Dot-Product Attention
  • Proposition 1: Attention Gradient
  • Definition 2: Grouped-Query Attention (GQA)
  • Definition 3: SwiGLU Activation
  • Proposition 2: SwiGLU Gradient
  • Definition 4: Root Mean Square Layer Normalization
  • Proposition 3: RMSNorm Backward Pass
  • Definition 5: Cross-Entropy Loss
  • Proposition 4: Cross-Entropy Gradient
  • Definition 6: Smoothed Cross-Entropy
  • ...and 84 more