ZiGong 1.0: A Large Language Model for Financial Credit

TL;DR

ZiGong introduces a Mistral‑7B–based financial credit LLM augmented with a novel data pruning pipeline and a temporal data distillation method (TracSeq) to mitigate hallucinations and forgetting. By scoring and selecting Top‑K instruct data via a proxy agent and integrating temporal dependencies, ZiGong achieves robust multi-task performance on credit scoring, fraud detection, and related tasks, validated on CALM benchmarks and deployed in a loan process workflow. The hybrid training approach, combining Top‑K data with original data, yields improved robustness and generalization while reducing data requirements. These contributions offer a practical pathway to reliable, finance‑oriented LLM deployments with enhanced risk assessment capabilities.

Abstract

Large Language Models (LLMs) have demonstrated strong performance across various general Natural Language Processing (NLP) tasks. However, their effectiveness in financial credit assessment applications remains suboptimal, primarily due to the specialized financial expertise required for these tasks. To address this limitation, we propose ZiGong, a Mistral-based model enhanced through multi-task supervised fine-tuning. To specifically combat model hallucination in financial contexts, we introduce a novel data pruning methodology. Our approach utilizes a proxy model to score training samples, subsequently combining filtered data with original datasets for model training. This data refinement strategy effectively reduces hallucinations in LLMs while maintaining reliability in downstream financial applications. Experimental results show our method significantly enhances model robustness and prediction accuracy in real-world financial scenarios.

Figures (2)

• Figure 1: Illustrates the entire workflow of ZiGong.
• Figure 2: Illustrates the impact of data pruning on model performance by comparing results across different sample sizes.