Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigation

TL;DR

This paper addresses robust generalization when labeled data are scarce by formulating agnostic active sampling within the Probably Approximately Correct framework to bound generalization error and label complexity under model-agnostic settings. It develops a theoretical backbone linking few-shot and active learning, introduces error disagreement as a central mechanism, and derives both supervised and unsupervised label-complexity bounds that guide low-resource learning. Building on these guarantees, it surveys optimization strategies—ranging from gradient-based and geometry-aware methods to meta-iteration and LLM-powered approaches—and analyzes how these tools operate in domain transfer, reinforcement feedback, and hierarchical structure modeling. The work bridges theory and practice by outlining concrete conditions under which low-resource data can approximate high-resource performance and by profiling practical paradigms for data-efficient AI with broad implications for scalable, resource-constrained learning systems.

Abstract

Learning with high-resource data has demonstrated substantial success in artificial intelligence (AI); however, the costs associated with data annotation and model training remain significant. A fundamental objective of AI research is to achieve robust generalization with limited-resource data. This survey employs agnostic active sampling theory within the Probably Approximately Correct (PAC) framework to analyze the generalization error and label complexity associated with learning from low-resource data in both model-agnostic supervised and unsupervised settings. Based on this analysis, we investigate a suite of optimization strategies tailored for low-resource data learning, including gradient-informed optimization, meta-iteration optimization, geometry-aware optimization, and LLMs-powered optimization. Furthermore, we provide a comprehensive overview of multiple learning paradigms that can benefit from low-resource data, including domain transfer, reinforcement feedback, and hierarchical structure modeling. Finally, we conclude our analysis and investigation by summarizing the key findings and highlighting their implications for learning with low-resource data.

Figures (6)

• Figure 1: The key contents of Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigation.
• Figure 2: The bilevel optimization fashion of meta-learning. During the meta-training phrase, the inner-level optimization is to train the model on different tasks which are sampled from $D^{train}_{source}$ to obtain the optimal model with parameters ${\theta}^{\ast}$, $\mathcal{L}^{task}$ denotes the optimization objective for inner-level optimization; the outer-level optimization aims to obtain a general meta-knowledge ${\phi}^{\ast}$ which be quickly adapted to unseen tasks. $\mathcal{L}^{meta}$ denotes the optimization objective to obtain ${\phi}^{\ast}$.
• Figure 3: Framework of efficient exploration for low-resource domain transfer.
• Figure 4: Reinforcement learning actor-critic strategies involve three components: Environment, Critic (value function), and Actor (policy). The Environment provides states and rewards based on the Actor’s actions; the Actor selects actions from states, while the Critic evaluates state or state–action values. TD error measures the gap between predicted and actual returns, driving updates for both the Critic and the Actor.
• Figure 5: Hierarchical Structure Modeling: Left figure (a) shows that the spatial structure of embodied intelligent agents executing designated tasks in a home scenario can be modeled as hierarchical-structured data, while right figure (b) presents the hierarchical structure mapping in hyperbolic space from a principled perspective.

Theorems & Definitions (12)

• Lemma 1
• Theorem 1
• Theorem 2
• Theorem 3
• Proposition 1
• Proposition 2
• Proposition 3
• Proposition 4
• Proposition 5
• Theorem 4