Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ASDA: Automated Skill Distillation and Adaptation for Financial Reasoning

Tik Yu Yim, Wenting Tan, Sum Yee Chan, Tak-Wah Lam, Siu Ming Yiu

Abstract

Adapting large language models (LLMs) to specialized financial reasoning typically requires expensive fine-tuning that produces model-locked expertise. Training-free alternatives have emerged, yet our experiments show that leading methods (GEPA and ACE) achieve only marginal gains on the FAMMA financial reasoning benchmark, exposing the limits of unstructured text optimization for complex, multi-step domain reasoning. We introduce Automated Skill Distillation and Adaptation (ASDA), a framework that automatically generates structured skill artifacts through iterative error-corrective learning without modifying model weights. A teacher model analyzes a student model's failures on financial reasoning tasks, clusters errors by subfield and error type, and synthesizes skill files containing reasoning procedures, code templates, and worked examples, which are dynamically injected during inference. Evaluated on FAMMA, ASDA achieves up to +17.33% improvement on arithmetic reasoning and +5.95% on non-arithmetic reasoning, substantially outperforming all training-free baselines. The resulting skill artifacts are human-readable, version-controlled, and compatible with the Agent Skills open standard, offering any organization with a labeled domain dataset a practical and auditable path to domain adaptation without weight access or retraining.

ASDA: Automated Skill Distillation and Adaptation for Financial Reasoning

Abstract

Adapting large language models (LLMs) to specialized financial reasoning typically requires expensive fine-tuning that produces model-locked expertise. Training-free alternatives have emerged, yet our experiments show that leading methods (GEPA and ACE) achieve only marginal gains on the FAMMA financial reasoning benchmark, exposing the limits of unstructured text optimization for complex, multi-step domain reasoning. We introduce Automated Skill Distillation and Adaptation (ASDA), a framework that automatically generates structured skill artifacts through iterative error-corrective learning without modifying model weights. A teacher model analyzes a student model's failures on financial reasoning tasks, clusters errors by subfield and error type, and synthesizes skill files containing reasoning procedures, code templates, and worked examples, which are dynamically injected during inference. Evaluated on FAMMA, ASDA achieves up to +17.33% improvement on arithmetic reasoning and +5.95% on non-arithmetic reasoning, substantially outperforming all training-free baselines. The resulting skill artifacts are human-readable, version-controlled, and compatible with the Agent Skills open standard, offering any organization with a labeled domain dataset a practical and auditable path to domain adaptation without weight access or retraining.
Paper Structure (36 sections, 3 figures, 4 tables, 1 algorithm)

This paper contains 36 sections, 3 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. Financial LLM Adaptation
  5. Training-Free Adaptation
  6. Skill-Based Agent Architectures
  7. Method: ASDA Framework
  8. Skills Warm-Up
  9. Failure Analysis and Structured Annotation
  10. Skill Library Organization
  11. Skill Selection and Injection
  12. Dual-Phase Iterative Skill Refinement
  13. Evidence Collection and Attribution
  14. Coverage Phase
  15. Safety Phase
  16. ...and 21 more sections

Figures (3)

  • Figure 1: Overview of the ASDA framework. Phase 1 (warm-up): the teacher model analyzes student failures, producing structured annotations that are clustered by subfield and error type to synthesize an initial skill library $\mathcal{K}_0$. Phase 2 (iterative refinement): the library is refined through two sequential phases, coverage refinement (resolving uncovered failures in $Q^\mathrm{gap}$) followed by safety refinement (suppressing regressions in $Q^-$), with every skill update gated by a correctness threshold. Inference: a selector reads SKILL.md and injects the relevant skill files into the student's prompt.
  • Figure 2: One pattern from an ASDA skill file produced during warm-up from Haiku 3.5 failure analysis by Sonnet 4.5. The full file contains five additional patterns.
  • Figure 3: Baseline vs. skill-augmented output on a FAMMA fixed income question (Haiku 3.5, warm-up). The skill file in Fig. \ref{['fig:skill-example']} provides the domain-specific procedure that corrects the baseline error. The same skill file was credited with 7 additional fixes on related fixed income questions.