Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Transfer Learning for Loan Recovery Prediction under Distribution Shifts with Heterogeneous Feature Spaces

Christopher Gerling, Hanqiu Peng, Ying Chen, Stefan Lessmann

Abstract

Accurate forecasting of recovery rates (RR) is central to credit risk management and regulatory capital determination. In many loan portfolios, however, RR modeling is constrained by data scarcity arising from infrequent default events. Transfer learning (TL) offers a promising avenue to mitigate this challenge by exploiting information from related but richer source domains, yet its effectiveness critically depends on the presence and strength of distributional shifts, and on potential heterogeneity between source and target feature spaces. This paper introduces FT-MDN-Transformer, a mixture-density tabular Transformer architecture specifically designed for TL in RR forecasting across heterogeneous feature sets. The model produces both loan-level point estimates and portfolio-level predictive distributions, thereby supporting a wide range of practical RR forecasting applications. We evaluate the proposed approach in a controlled Monte Carlo simulation that facilitates systematic variation of covariate, conditional, and label shifts, as well as in a real-world transfer setting using the Global Credit Data (GCD) loan dataset as source and a novel bonds dataset as target. Our results show that FT-MDN-Transformer outperforms baseline models when target-domain data are limited, with particularly pronounced gains under covariate and conditional shifts, while label shift remains challenging. We also observe its probabilistic forecasts to closely track empirical recovery distributions, providing richer information than conventional point-prediction metrics alone. Overall, the findings highlight the potential of distribution-aware TL architectures to improve RR forecasting in data-scarce credit portfolios and offer practical insights for risk managers operating under heterogeneous data environments.

Transfer Learning for Loan Recovery Prediction under Distribution Shifts with Heterogeneous Feature Spaces

Abstract

Accurate forecasting of recovery rates (RR) is central to credit risk management and regulatory capital determination. In many loan portfolios, however, RR modeling is constrained by data scarcity arising from infrequent default events. Transfer learning (TL) offers a promising avenue to mitigate this challenge by exploiting information from related but richer source domains, yet its effectiveness critically depends on the presence and strength of distributional shifts, and on potential heterogeneity between source and target feature spaces. This paper introduces FT-MDN-Transformer, a mixture-density tabular Transformer architecture specifically designed for TL in RR forecasting across heterogeneous feature sets. The model produces both loan-level point estimates and portfolio-level predictive distributions, thereby supporting a wide range of practical RR forecasting applications. We evaluate the proposed approach in a controlled Monte Carlo simulation that facilitates systematic variation of covariate, conditional, and label shifts, as well as in a real-world transfer setting using the Global Credit Data (GCD) loan dataset as source and a novel bonds dataset as target. Our results show that FT-MDN-Transformer outperforms baseline models when target-domain data are limited, with particularly pronounced gains under covariate and conditional shifts, while label shift remains challenging. We also observe its probabilistic forecasts to closely track empirical recovery distributions, providing richer information than conventional point-prediction metrics alone. Overall, the findings highlight the potential of distribution-aware TL architectures to improve RR forecasting in data-scarce credit portfolios and offer practical insights for risk managers operating under heterogeneous data environments.

Paper Structure

This paper contains 44 sections, 15 equations, 15 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Recovery Rate and LGD Modeling
  4. Transfer Learning and Domain Adaptation in Finance
  5. Tabular Transfer, Adaptation, and Robustness Methods
  6. Gap and Our Positioning
  7. Methodology
  8. FT--MDN--Transformer
  9. Data structures and notation
  10. Feature-wise tokenization
  11. Transformer backbone
  12. Mixture-density head
  13. Heterogeneous Schema Adaptation
  14. Token-level mechanism
  15. Two-stage transfer schedule
  16. ...and 29 more sections

Figures (15)

  • Figure 1: Architecture of the FT--MDN--Transformer with distributional RR predictions.
  • Figure 2: Two-stage schema adaptation comparing shared-only and full-source pretraining followed by expansion to the target schema.
  • Figure 3: Evaluation scenarios used throughout, including zero-shot, target baseline, and transfer learning.
  • Figure 4: Comparison of recovery rate distributions and feature space overlap.
  • Figure 5: Correlation heatmaps (31$\times$31) of common macro-financial features in GCD and UP5.
  • ...and 10 more figures