Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NOMAD -- Navigating Optimal Model Application to Datastreams

Ashwin Gerard Colaco, Sharad Mehrotra, Michael J De Lucia, Kevin Hamlen, Murat Kantarcioglu, Latifur Khan, Ananthram Swami, Bhavani Thuraisingham

TL;DR

NOMAD tackles real-time multiclass classification during data ingestion by dynamically constructing model chains that balance cost and accuracy. It uses a utility-based greedy policy, belief updates, and a formal chain-safety framework to guarantee $\epsilon$-comparable quality to a high-quality role model, even under distribution drift. The approach supports heterogeneous pretrained models, dependent model structures, and batched inference, delivering 2–6× speedups across eight datasets with substantial throughput gains and robust performance under drift, while incurring minimal overhead. Its adaptive priors via ARIMA and Page-Hinkley enable responsive behavior in non-stationary streams, making NOMAD practical for CPU-only deployments in edge and enterprise environments.

Abstract

NOMAD (Navigating Optimal Model Application for Datastreams) is an intelligent framework for data enrichment during ingestion that optimizes realtime multiclass classification by dynamically constructing model chains, i.e ,sequences of machine learning models with varying cost-quality tradeoffs, selected via a utilitybased criterion. Inspired by predicate ordering techniques from database query processing, NOMAD leverages cheaper models as initial filters, proceeding to more expensive models only when necessary, while guaranteeing classification quality remains comparable to a designated role model through a formal chain safety mechanism. It employs a dynamic belief update strategy to adapt model selection based on per event predictions and shifting data distributions, and extends to scenarios with dependent models such as earlyexit DNNs and stacking ensembles. Evaluation across multiple datasets demonstrates that NOMAD achieves significant computational savings compared to static and naive approaches while maintaining classification quality comparable to that achieved by the most accurate (and often the most expensive) model.

NOMAD -- Navigating Optimal Model Application to Datastreams

TL;DR

NOMAD tackles real-time multiclass classification during data ingestion by dynamically constructing model chains that balance cost and accuracy. It uses a utility-based greedy policy, belief updates, and a formal chain-safety framework to guarantee -comparable quality to a high-quality role model, even under distribution drift. The approach supports heterogeneous pretrained models, dependent model structures, and batched inference, delivering 2–6× speedups across eight datasets with substantial throughput gains and robust performance under drift, while incurring minimal overhead. Its adaptive priors via ARIMA and Page-Hinkley enable responsive behavior in non-stationary streams, making NOMAD practical for CPU-only deployments in edge and enterprise environments.

Abstract

NOMAD (Navigating Optimal Model Application for Datastreams) is an intelligent framework for data enrichment during ingestion that optimizes realtime multiclass classification by dynamically constructing model chains, i.e ,sequences of machine learning models with varying cost-quality tradeoffs, selected via a utilitybased criterion. Inspired by predicate ordering techniques from database query processing, NOMAD leverages cheaper models as initial filters, proceeding to more expensive models only when necessary, while guaranteeing classification quality remains comparable to a designated role model through a formal chain safety mechanism. It employs a dynamic belief update strategy to adapt model selection based on per event predictions and shifting data distributions, and extends to scenarios with dependent models such as earlyexit DNNs and stacking ensembles. Evaluation across multiple datasets demonstrates that NOMAD achieves significant computational savings compared to static and naive approaches while maintaining classification quality comparable to that achieved by the most accurate (and often the most expensive) model.

Paper Structure

This paper contains 114 sections, 2 theorems, 35 equations, 23 figures, 15 tables, 6 algorithms.

Table of Contents

  1. INTRODUCTION
  2. Problem Formalization
  3. Measure of Model Quality
  4. Problem Definition
  5. NOMAD Algorithm
  6. Utility-based Model Selection
  7. Belief Update
  8. CHAIN SAFETY
  9. Computing Passthrough Probability
  10. Global Chain Safety
  11. Checking Global Chain Safety.
  12. Extensions
  13. Adaptive Algorithm
  14. EVALUATION
  15. Experimental Setup
  16. ...and 99 more sections

Key Result

theorem 1

A classification strategy $\mathcal{S}$ constructed to satisfy the global chain safety condition meets the Global $\epsilon$-comparability requirement, if the estimated passthrough probabilities, $\text{Prob}^{est}_{pt}$, are less than or equal to the true probabilities $\text{Prob}^{true}_{pt}$.

Figures (23)

  • Figure 1: NOMAD's Data Ingestion Framework
  • Figure 2: a) Quality comparison at $\epsilon = 0.10$. Error bars show $\pm 1$ std dev. b) NOMAD speedup compared to $S_{max}$ across datasets at $\epsilon = 0.10$.
  • Figure 3: NOMAD Efficiency vs Oracle for varying $\epsilon$ values.
  • Figure 4: a)Throughput analysis on UNSW-NB15. b) Cost impact of distribution shift at 5000 events
  • Figure 5: a)Adaptation benefit increases with drift intensity---at high drift (KL = 0.8), adaptation prevents 30-50% cost increase across datasets. b) Detection threshold $\lambda$ controls detection speed---lower values detect faster but may be sensitive to noise.
  • ...and 18 more figures

Theorems & Definitions (15)

  • Example 2.1
  • definition 1: Role Model
  • definition 2: Class-based $\epsilon$-Comparability
  • definition 3: Global $\epsilon$-Comparability
  • definition 4: Exit Model for a Class
  • definition 5: Exit Classes for a Model
  • definition 6: Classification Strategy
  • definition 7: Realized Model Chain for an Event
  • Example 2.2
  • definition 8: Strategy Cost
  • ...and 5 more