Demystifying Domain-adaptive Post-training for Financial LLMs

TL;DR

This work presents FinDaP, a finance-focused framework for domain-adaptive post-training of LLMs, composed of four components: FinCap (target capabilities), FinRec (joint CPT+IT with novel PA signals), FinTrain (curated data), and FinEval (broad evaluation). By applying FinDaP to the Llama3-8B-instruct base, the authors train Llama-Fin, achieving state-of-the-art results on unseen finance tasks and competitive performance on novel challenges, while providing deep analysis of each post-training stage. A key innovation is the Stepwise Corrective Preference and Final Answer Preference used within a GenRM-guided PA pipeline to improve reasoning without excessive data or compute. The framework emphasizes explicit capability definitions, balanced data strategies, and comprehensive evaluation, offering actionable guidance for domain adaptation of finance LLMs and highlighting directions for scaling and multi-modality.

Abstract

Domain-adaptive post-training of large language models (LLMs) has emerged as a promising approach for specialized domains such as medicine and finance. However, significant challenges remain in identifying optimal adaptation criteria and training strategies across varying data and model configurations. To address these challenges, we introduce FINDAP, a systematic and fine-grained investigation into domain-adaptive post-training of LLMs for the finance domain. Our approach consists of four key components: FinCap, which defines the core capabilities required for the target domain; FinRec, an effective training recipe that jointly optimizes continual pre-training and instruction-following, along with a novel preference data distillation method leveraging process signals from a generative reward model; FinTrain, a curated set of training datasets supporting FinRec; and FinEval, a comprehensive evaluation suite aligned with FinCap. The resulting model, Llama-Fin, achieves state-of-the-art performance across a wide range of financial tasks. Our analysis also highlights how each post-training stage contributes to distinct capabilities, uncovering specific challenges and effective solutions, providing valuable insights for domain adaptation of LLMs

Figures (13)

• Figure 1: An overview of our finance-specific post-training framework, FinDaP. It comprises four key components: (1) FinCap, the core expected capabilities, including concepts, reasoning, instruction-following and tasks; (2) FinRec, encompassing both data and model strategies to guide domain-adaptive post-training; (3) FinTrain, which curates training texts and prompts based on the data recipe; and (4) FinEval, a comprehensive evaluation framework designed to assess performance on unseen tasks, categorized into similar and novel, general and domain-specific, and reasoning tasks, using both direct-answer and chain-of-thought (CoT) evaluation methods.
• Figure 2: An overview of the proposed final answer preference (FAP) and stepwise corrective preference (SCP). In FAP, we collect trajectories from the GenRM by evaluating the entire solution. In SCP, we collect trajectories from the GenRM, by identifying and correcting the first erroneous step.
• Figure B.1: Average performance on selected datasets for training Llama3-8b-instruct on our CPT-In, CPT-Gen and CPT-Mix. The Y-axis represents the same performance metrics as those reported in Tables \ref{['tab.similar_results']} and \ref{['tab.novel_results']}. The selected datasets are chosen for illustration purpose based on their ability to illustrate the general trend.
• Figure B.2: Average performance on selected datasets for training Llama3-8b-instruct on our IT-In, IT-Gen and IT-Mix.
• Figure B.3: Average performance on selected datasets for training Llama3-8b-instruct on IT-Mix with full-model finetuning (IT-Mix) and LoRA finetuning (IT-Mix (LoRA)).