Feature-Function Curvature Analysis: A Geometric Framework for Explaining Differentiable Models

TL;DR

FFCA introduces a geometry-based framework for explaining differentiable models by quantifying each feature's local behavior through a 4D signature: Impact, Volatility, Non-linearity, and Interaction. It couples Static FFCA with Dynamic Archetype Analysis to reveal how features evolve during training, providing evidence of hierarchical learning and enabling early diagnostics for capacity limits and overfitting. The methodology leverages Hessian-based analyses, activation smoothing via Softplus, and a novel eight-archetype taxonomy to translate complex geometry into actionable diagnostics. Empirically, FFCA demonstrates predictive value for feature engineering, early overfitting warnings, and detection of spurious correlations, supporting its practical adoption in model auditing and development.

Abstract

Explainable AI (XAI) is critical for building trust in complex machine learning models, yet mainstream attribution methods often provide an incomplete, static picture of a model's final state. By collapsing a feature's role into a single score, they are confounded by non-linearity and interactions. To address this, we introduce Feature-Function Curvature Analysis (FFCA), a novel framework that analyzes the geometry of a model's learned function. FFCA produces a 4-dimensional signature for each feature, quantifying its: (1) Impact, (2) Volatility, (3) Non-linearity, and (4) Interaction. Crucially, we extend this framework into Dynamic Archetype Analysis, which tracks the evolution of these signatures throughout the training process. This temporal view moves beyond explaining what a model learned to revealing how it learns. We provide the first direct, empirical evidence of hierarchical learning, showing that models consistently learn simple linear effects before complex interactions. Furthermore, this dynamic analysis provides novel, practical diagnostics for identifying insufficient model capacity and predicting the onset of overfitting. Our comprehensive experiments demonstrate that FFCA, through its static and dynamic components, provides the essential geometric context that transforms model explanation from simple quantification to a nuanced, trustworthy analysis of the entire learning process.

Figures (18)

• Figure 1: The learning evolution for the high-capacity model on the Credit Loan dataset. This plot clearly demonstrates hierarchical learning: the impact of the 'simple_driver' (blue line, top-left) rises and stabilizes early. In contrast, the interaction scores of the hidden interactors (red and purple lines, bottom-right) remain near zero before taking off dramatically after epoch 60, corresponding to the model's main performance improvement.
• Figure 2: The final analysis summary for the high-capacity Credit Loan model. (Top-Left) The model performance reaches near-perfect $R^2$. (Top-Right) the rise and plateau of volatility reflects the successful learning of complex functions. (Bottom-Left) The final radar signatures are clear and correctly identify the engineered roles. (Bottom-Right) The interaction heatmap is showing a strong, isolated interaction between 'base_interactor' and 'partner_interactor'.
• Figure 3: Comprehensive analysis of the full model with the interaction term. (Left) Single-score methods ambiguously rank 'credit_history_len' and 'loan_purpose_impact'. (Center) The FFCA radar plot clearly distinguishes their archetypes: 'credit_history_len' is dominated by Interaction (orange), while 'loan_purpose_impact' is driven by Impact (green). (Right) The interaction heatmap confirms the strong pairwise relationship between the two hidden interactors.
• Figure 4: The SHAP importance of 'credit_history_len' collapses from 7.56 to 0.44 when its interaction partner is removed from the system, proving its importance was derived from the interaction that FFCA identified.
• Figure 5: The ideal FFCA signatures for the eight ground-truth archetypes. These plots are generated not from a trained neural network, but from a deterministic mathematical function in which each feature was explicitly engineered to be a perfect exemplar of its archetype. Each radar plot shows the 4D signature for one such feature, visually demonstrating the pure "fingerprint" for each role (e.g., the Simple Workhorse is dominated by Impact, while the Hidden Interactor is defined by Interaction). This provides a clear validation of the FFCA measurement methodology.