LIFT: Interpretable truck driving risk prediction with literature-informed fine-tuned LLMs

TL;DR

This work formulates truck driving risk prediction as a risk probability $P(Risky) = f_\theta(\mathbf{X}_l,\mathbf{X}_s,\mathbf{X}_t)$ and demonstrates a literature-informed fine-tuned LLM (LIFT) that integrates an automated literature knowledge base with an Inference Core. The framework textualizes real-world truck data, fine-tunes an open-source LLM using LoRA, and leverages a literature-derived knowledge base to produce both predictions and human-interpretable explanations. Empirical results show the LIFT LLM achieves higher recall (0.95) and competitive F1 (0.76) than benchmark models, while KeY variable identification aligns with a random forest baseline and permits discovery of significant variable combinations validated by PERMANOVA. The findings highlight the potential of customized knowledge bases and literature-informed prompting to enhance interpretable knowledge discovery and risk management in truck safety, though the study notes challenges such as hallucination and domain-specific generalization for future work.

Abstract

This study proposes an interpretable prediction framework with literature-informed fine-tuned (LIFT) LLMs for truck driving risk prediction. The framework integrates an LLM-driven Inference Core that predicts and explains truck driving risk, a Literature Processing Pipeline that filters and summarizes domain-specific literature into a literature knowledge base, and a Result Evaluator that evaluates the prediction performance as well as the interpretability of the LIFT LLM. After fine-tuning on a real-world truck driving risk dataset, the LIFT LLM achieved accurate risk prediction, outperforming benchmark models by 26.7% in recall and 10.1% in F1-score. Furthermore, guided by the literature knowledge base automatically constructed from 299 domain papers, the LIFT LLM produced variable importance ranking consistent with that derived from the benchmark model, while demonstrating robustness in interpretation results to various data sampling conditions. The LIFT LLM also identified potential risky scenarios by detecting key combination of variables in truck driving risk, which were verified by PERMANOVA tests. Finally, we demonstrated the contribution of the literature knowledge base and the fine-tuning process in the interpretability of the LIFT LLM, and discussed the potential of the LIFT LLM in data-driven knowledge discovery.

Figures (15)

• Figure 1: Knowledge discovery process with LIFT LLMs compared with the paradigm of data-driven knowledge discovery by human researchers.
• Figure 2: The proposed interpretable prediction framework with LIFT LLMs. The framework consists of three components: Literature Processing Pipeline, which constructs a literature knowledge base from a large volume of academic papers; Inference Core, which performs prediction and explanation of truck driving risk based on a fine-tuned LLM, informed by the literature knowledge base; and Result Evaluator, which assesses the model's predictive performance and the quality of its interpretative outputs.
• Figure 3: Prompt template for truck driving risk prediction problem (Task 1).
• Figure 4: Literature Processing Pipeline for variable-related domain knowledge extraction.
• Figure 5: Prompt for literature information extraction.