NSR-Boost: A Neuro-Symbolic Residual Boosting Framework for Industrial Legacy Models

TL;DR

NSR-Boost presents a non-intrusive, neuro-symbolic framework that patches frozen legacy GBDT predictors by learning additive residuals focused on hard regions. It combines LLM-driven symbolic expert generation with bi-level optimization (discrete structure search and continuous parameter tuning) and a context-aware aggregator to safely and efficiently improve performance without retraining the base model. The method achieves state-of-the-art results on public benchmarks and a private financial dataset, demonstrates robust online gains in real deployments, and preserves interpretability through explicit symbolic rules. This approach provides a practical, low-risk path for industrial systems to evolve with long-tail risk capture and low-latency inference.

Abstract

Although the Gradient Boosted Decision Trees (GBDTs) dominate industrial tabular applications, upgrading legacy models in high-concurrency production environments still faces prohibitive retraining costs and systemic risks. To address this problem, we present NSR-Boost, a neuro-symbolic residual boosting framework designed specifically for industrial scenarios. Its core advantage lies in being "non-intrusive". It treats the legacy model as a frozen model and performs targeted repairs on "hard regions" where predictions fail. The framework comprises three key stages: first, finding hard regions through residuals, then generating interpretable experts by generating symbolic code structures using Large Language Model (LLM) and fine-tuning parameters using Bayesian optimization, and finally dynamically integrating experts with legacy model output through a lightweight aggregator. We report on the successful deployment of NSR-Boost within the core financial risk control system at Qfin Holdings. This framework not only significantly outperforms state-of-the-art (SOTA) baselines across six public datasets and one private dataset, more importantly, shows excellent performance gains on real-world online data. In conclusion, it effectively captures long-tail risks missed by traditional models and offers a safe, low-cost evolutionary paradigm for industry.

Figures (11)

• Figure 1: The framework overview of NSR-Boost.
• Figure 2: Generalizability analysis of NSR-Boost across various backbones and datasets.
• Figure 3: Illustration of the explicit risk rules and actionable high-value segments identified by the Symbolic Expert.
• Figure 4: Ablation study of NSR-Boost components on the credit-g dataset.
• Figure 5: Performance evaluation of the Emerging Customer Segment (B-Card) model. The online deployment processes approximately 2.4 million real user samples.