Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Information Templates: A New Paradigm for Intelligent Active Feature Acquisition

Hung-Tien Huang, Dzung Dinh, Junier B. Oliva

TL;DR

The paper tackles the challenge of costly, instance-adaptive feature acquisition by introducing TAFA, a non-RL, template-based framework that learns a library of informative feature subsets (templates) to guide next-feature decisions. By formulating template discovery as a submodular set-optimization problem and combining mutation-guided greedy search with continuous relaxation, TAFA efficiently narrows the action space while maintaining strong cost-benefit performance. The approach yields an interpretable policy via distillation into step-wise decision trees that expose explicit acquisition rules. Extensive experiments on synthetic and real-world datasets show TAFA outperforms RL-free baselines in accuracy and acquisition efficiency, while delivering substantial speedups and clear interpretability, making it practical for real-time AFA scenarios.

Abstract

Active feature acquisition (AFA) is an instance-adaptive paradigm in which, at inference time, a policy sequentially chooses which features to acquire (at a cost) before predicting. Existing approaches either train reinforcement learning policies, which deal with a difficult MDP, or greedy policies that cannot account for the joint informativeness of features or require knowledge about the underlying data distribution. To overcome this, we propose Template-based AFA (TAFA), a non-greedy framework that learns a small library of feature templates -- sets of features that are jointly informative -- and uses this library of templates to guide the next feature acquisitions. Through identifying feature templates, the proposed framework not only significantly reduces the action space considered by the policy but also alleviates the need to estimate the underlying data distribution. Extensive experiments on synthetic and real-world datasets show that TAFA outperforms the existing state-of-the-art baselines while achieving lower overall acquisition cost and computation.

Information Templates: A New Paradigm for Intelligent Active Feature Acquisition

TL;DR

The paper tackles the challenge of costly, instance-adaptive feature acquisition by introducing TAFA, a non-RL, template-based framework that learns a library of informative feature subsets (templates) to guide next-feature decisions. By formulating template discovery as a submodular set-optimization problem and combining mutation-guided greedy search with continuous relaxation, TAFA efficiently narrows the action space while maintaining strong cost-benefit performance. The approach yields an interpretable policy via distillation into step-wise decision trees that expose explicit acquisition rules. Extensive experiments on synthetic and real-world datasets show TAFA outperforms RL-free baselines in accuracy and acquisition efficiency, while delivering substantial speedups and clear interpretability, making it practical for real-time AFA scenarios.

Abstract

Active feature acquisition (AFA) is an instance-adaptive paradigm in which, at inference time, a policy sequentially chooses which features to acquire (at a cost) before predicting. Existing approaches either train reinforcement learning policies, which deal with a difficult MDP, or greedy policies that cannot account for the joint informativeness of features or require knowledge about the underlying data distribution. To overcome this, we propose Template-based AFA (TAFA), a non-greedy framework that learns a small library of feature templates -- sets of features that are jointly informative -- and uses this library of templates to guide the next feature acquisitions. Through identifying feature templates, the proposed framework not only significantly reduces the action space considered by the policy but also alleviates the need to estimate the underlying data distribution. Extensive experiments on synthetic and real-world datasets show that TAFA outperforms the existing state-of-the-art baselines while achieving lower overall acquisition cost and computation.

Paper Structure

This paper contains 21 sections, 4 theorems, 16 equations, 12 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. AFA Background and Related Works
  3. Methodology
  4. Template Search as Set Optimization
  5. Template-Based Policy
  6. Mutation-Guided Greedy Template Search
  7. Template Search Through Continuous Relaxation
  8. Interpretable Policy and Distillation
  9. Experiments
  10. Benchmark Dataset Evaluations
  11. TAFA Variants Ablation
  12. Interpretability Analysis
  13. Conclusion
  14. Submodularity of Template Set Optimization
  15. Facility Location
  16. ...and 6 more sections

Key Result

Theorem 3.1

(Informal) The template collection objective function $g(\mathcal{B})$ defined in eq:tafa_set_obj_fn is submodular in $\mathcal{B}$.

Figures (12)

  • Figure 1: Example of rolling out an AFA policy. The AFA agent continues gathering information until it has enough information to make a decision about the instance at hand.
  • Figure 2: Our proposed template optimization procedure.
  • Figure 3: Step-wise Ensemble. At each step, non-termination (colored) leaves select features and advance to the next tree level, reusing the same tree structure to build complete acquisition paths that form interpretable rules.
  • Figure 4: Accuracy vs. log-scaled inference time vs. number of features acquired compared to baselines. Tick marks for methods are computed by adjusting respective cost/benefit trade-off hparams.
  • Figure 5: Visualization of feature templates found for the MNIST dataset. Yellow pixels in the "example instances utilizing templates" column represent the pixels that are acquired by the TAFA agent to classify the digits of interest.
  • ...and 7 more figures

Theorems & Definitions (7)

  • Theorem 3.1
  • proof
  • Theorem 3.2
  • proof
  • Theorem 1.1
  • proof
  • Theorem 2.1