Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PiCSRL: Physics-Informed Contextual Spectral Reinforcement Learning

Mitra Nasr Azadani, Syed Usama Imtiaz, Nasrin Alamdari

Abstract

High-dimensional low-sample-size (HDLSS) datasets constrain reliable environmental model development, where labeled data remain sparse. Reinforcement learning (RL)-based adaptive sensing methods can learn optimal sampling policies, yet their application is severely limited in HDLSS contexts. In this work, we present PiCSRL (Physics-Informed Contextual Spectral Reinforcement Learning), where embeddings are designed using domain knowledge and parsed directly into the RL state representation for improved adaptive sensing. We developed an uncertainty-aware belief model that encodes physics-informed features to improve prediction. As a representative example, we evaluated our approach for cyanobacterial gene concentration adaptive sampling task using NASA PACE hyperspectral imagery over Lake Erie. PiCSRL achieves optimal station selection (RMSE = 0.153, 98.4% bloom detection rate, outperforming random (0.296) and UCB (0.178) RMSE baselines, respectively. Our ablation experiments demonstrate that physics-informed features improve test generalization (0.52 R^2, +0.11 over raw bands) in semi-supervised learning. In addition, our scalability test shows that PiCSRL scales effectively to large networks (50 stations, >2M combinations) with significant improvements over baselines (p = 0.002). We posit PiCSRL as a sample-efficient adaptive sensing method across Earth observation domains for improved observation-to-target mapping.

PiCSRL: Physics-Informed Contextual Spectral Reinforcement Learning

Abstract

High-dimensional low-sample-size (HDLSS) datasets constrain reliable environmental model development, where labeled data remain sparse. Reinforcement learning (RL)-based adaptive sensing methods can learn optimal sampling policies, yet their application is severely limited in HDLSS contexts. In this work, we present PiCSRL (Physics-Informed Contextual Spectral Reinforcement Learning), where embeddings are designed using domain knowledge and parsed directly into the RL state representation for improved adaptive sensing. We developed an uncertainty-aware belief model that encodes physics-informed features to improve prediction. As a representative example, we evaluated our approach for cyanobacterial gene concentration adaptive sampling task using NASA PACE hyperspectral imagery over Lake Erie. PiCSRL achieves optimal station selection (RMSE = 0.153, 98.4% bloom detection rate, outperforming random (0.296) and UCB (0.178) RMSE baselines, respectively. Our ablation experiments demonstrate that physics-informed features improve test generalization (0.52 R^2, +0.11 over raw bands) in semi-supervised learning. In addition, our scalability test shows that PiCSRL scales effectively to large networks (50 stations, >2M combinations) with significant improvements over baselines (p = 0.002). We posit PiCSRL as a sample-efficient adaptive sensing method across Earth observation domains for improved observation-to-target mapping.

Paper Structure

This paper contains 13 sections, 6 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Problem Formulation
  4. Semi-Supervised Learning with Physics-Informed Features
  5. Reinforcement Learning in Reduced State Space
  6. Experiments
  7. Experimental Setup
  8. Representation Learning for HDLSS
  9. Adaptive Strategy Performance
  10. Scalability Analysis
  11. Discussion
  12. Conclusion
  13. Acknowledgment

Figures (3)

  • Figure 1: Physics-Informed Contextual Spectral Reinforcement Learning (PiCSRL) framework. Physics-informed bio-optical indices and sparse in-situ observations are integrated through a semi-supervised learning framework and an uncertainty-aware belief model. These uncertainty-aware predictions are then used in a reduced RL state representation to guide adaptive station selection under sampling constraints.
  • Figure 2: Adaptive sampling performance for selecting $K=3$ stations from $N=8$ candidates. Left: Lake-wide reconstruction error (RMSE; mean $\pm 1\sigma$ over 500 episodes), with the optimal exhaustive baseline shown as a dashed line. Right: Bloom detection rate (%), with the 95% operational target and the perfect (100%) reference indicated.
  • Figure 3: Performance comparison of adaptive sampling strategies. Left: Detection accuracy for comparison methods and the proposed PiCSRL method. Right: Corresponding cumulative reward achieved by each strategy. PiCSRL attains the highest detection accuracy and cumulative reward, with statistically significant improvement over baseline methods ($p=0.002$).