Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SCALAR: Self-Calibrating Adaptive Latent Attention Representation Learning

Farwa Abbas, Hussain Ahmad, Claudia Szabo

TL;DR

SCALAR addresses learning from high-dimensional, heterogeneous data by integrating adaptive kernel-based attention with a self-calibration layer and a variational latent space. The model processes feature groups hierarchically to preserve local interactions before integrating global dependencies, and employs a KL-regularized variational framework to enhance generalization, especially with limited data. Empirical results on drug–target interaction and NIR spectroscopy datasets show superior predictive accuracy and interpretable feature importance, validated through SHAP analyses and biologically meaningful feature selection. The approach offers robust cross-domain performance with improved stability under distributional shifts, making it a promising tool for complex multivariate tasks in biology and spectroscopy.

Abstract

High-dimensional, heterogeneous data with complex feature interactions pose significant challenges for traditional predictive modeling approaches. While Projection to Latent Structures (PLS) remains a popular technique, it struggles to model complex non-linear relationships, especially in multivariate systems with high-dimensional correlation structures. This challenge is further compounded by simultaneous interactions across multiple scales, where local processing fails to capture crossgroup dependencies. Additionally, static feature weighting limits adaptability to contextual variations, as it ignores sample-specific relevance. To address these limitations, we propose a novel method that enhances predictive performance through novel architectural innovations. Our architecture introduces an adaptive kernel-based attention mechanism that processes distinct feature groups separately before integration, enabling capture of local patterns while preserving global relationships. Experimental results show substantial improvements in performance metrics, compared to the state-of-the-art methods across diverse datasets.

SCALAR: Self-Calibrating Adaptive Latent Attention Representation Learning

TL;DR

SCALAR addresses learning from high-dimensional, heterogeneous data by integrating adaptive kernel-based attention with a self-calibration layer and a variational latent space. The model processes feature groups hierarchically to preserve local interactions before integrating global dependencies, and employs a KL-regularized variational framework to enhance generalization, especially with limited data. Empirical results on drug–target interaction and NIR spectroscopy datasets show superior predictive accuracy and interpretable feature importance, validated through SHAP analyses and biologically meaningful feature selection. The approach offers robust cross-domain performance with improved stability under distributional shifts, making it a promising tool for complex multivariate tasks in biology and spectroscopy.

Abstract

High-dimensional, heterogeneous data with complex feature interactions pose significant challenges for traditional predictive modeling approaches. While Projection to Latent Structures (PLS) remains a popular technique, it struggles to model complex non-linear relationships, especially in multivariate systems with high-dimensional correlation structures. This challenge is further compounded by simultaneous interactions across multiple scales, where local processing fails to capture crossgroup dependencies. Additionally, static feature weighting limits adaptability to contextual variations, as it ignores sample-specific relevance. To address these limitations, we propose a novel method that enhances predictive performance through novel architectural innovations. Our architecture introduces an adaptive kernel-based attention mechanism that processes distinct feature groups separately before integration, enabling capture of local patterns while preserving global relationships. Experimental results show substantial improvements in performance metrics, compared to the state-of-the-art methods across diverse datasets.
Paper Structure (15 sections, 13 equations, 7 figures, 2 tables, 1 algorithm)

This paper contains 15 sections, 13 equations, 7 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation
  4. Self-Calibration and Variational Processing
  5. Global Attention and Component Weighting
  6. Loss Computation and Feature Importance Score
  7. Experimental Results
  8. Results on Davis Dataset
  9. Results on NIR Spectroscopy for Wheat Dataset
  10. Ablation Study
  11. Self-calibration under domain shifts:
  12. Variational encoding for generalization:
  13. Computational efficiency:
  14. Discussion
  15. Conclusion

Figures (7)

  • Figure 1: Block diagram of our proposed SCALAR model
  • Figure 2: Comparative Analysis for different metric values.
  • Figure 3: Bin-wise RMSE for different methods.
  • Figure 4: Feature importance estimated by kernel based attention
  • Figure 5: Distribution of feature groups in latent space.
  • ...and 2 more figures