THaLLE: Text Hyperlocally Augmented Large Language Extension -- Technical Report

TL;DR

This work tackles the high compute cost of very large LLMs by developing THaLLE, an 8B open-source, Thai-oriented finance-augmented model that passes CFA mock exams. It systematically compares fine-tuning strategies—SFT and DPO—using data augmentation with Zero Shot and Chain-of-Thought prompts, along with LoRA-based parameter-efficient training. The results show THaLLE and other instruction-following open models can achieve CFA passes on internal mocks and the Flare CFA dataset, with SFT and CoT-enhanced data yielding notable gains and DPO offering more stable generalization. The contributions provide cost-effective methodologies for finance-adjacent LLMs and highlight practical evaluation benchmarks, while acknowledging CFA as a proxy necessitating broader real-world validation and Thai-language expansion.

Abstract

Recent advancements in Large Language Models (LLMs) have revealed new capabilities and opportunities across the technological landscape. However, the practicality of very large LLMs is challenged by their high compute cost, which does not justify the benefits given their limited capability compared to humans. While smaller, more practical LLMs have shown potential in financial analysis, though they are not yet fully proficient, as evidenced by their near-passing performance on the Chartered Financial Analyst (CFA) exam. In this work, we present Financial Analyst Extension to our Text Hyperlocally Augmented Large Language Extension (THaLLE), a series of 8B LLMs consistently achieving highest performance on mock CFA exams against models of comparable size. We thoroughly document the fine-tuning techniques used to facilitate future research. Additionally, we introduce the use of Flare CFA, a publicly available dataset for evaluating LLMs as a financial advisor.