Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Maintaining User Trust Through Multistage Uncertainty Aware Inference

Chandan Agrawal, Ashish Papanai, Jerome White

TL;DR

Maintaining user trust in AI-driven pest management under low-resource conditions is addressed by a multistage uncertainty-aware inference architecture. The approach stacks an on-device detector, a cloud detector, and a human-in-the-loop, with abstention guided by a box-confidence windowing mechanism and evaluated via $MCC$. The system maps pest counts to three action levels and demonstrates end-to-end performance gains and reduced false alarms in field-like deployments, currently active across thousands of cotton farms in India. The work generalizes to other low-resource AI deployments where connectivity and latency constrain direct use of state-of-the-art models.

Abstract

This paper describes and evaluates a multistage approach to AI deployment. Each stage involves a more accurate method of inference, yet engaging each comes with an increasing cost. In outlining the architecture, we present a method for quantifying model uncertainty that facilitates confident deferral decisions. The architecture is currently under active deployment to thousands of cotton farmers across India. The broader idea however is applicable to a growing sector of AI deployments in challenging low resources settings.

Maintaining User Trust Through Multistage Uncertainty Aware Inference

TL;DR

Maintaining user trust in AI-driven pest management under low-resource conditions is addressed by a multistage uncertainty-aware inference architecture. The approach stacks an on-device detector, a cloud detector, and a human-in-the-loop, with abstention guided by a box-confidence windowing mechanism and evaluated via . The system maps pest counts to three action levels and demonstrates end-to-end performance gains and reduced false alarms in field-like deployments, currently active across thousands of cotton farms in India. The work generalizes to other low-resource AI deployments where connectivity and latency constrain direct use of state-of-the-art models.

Abstract

This paper describes and evaluates a multistage approach to AI deployment. Each stage involves a more accurate method of inference, yet engaging each comes with an increasing cost. In outlining the architecture, we present a method for quantifying model uncertainty that facilitates confident deferral decisions. The architecture is currently under active deployment to thousands of cotton farmers across India. The broader idea however is applicable to a growing sector of AI deployments in challenging low resources settings.
Paper Structure (13 sections, 4 figures)

This paper contains 13 sections, 4 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Setup
  4. Pest detection in low resource settings
  5. Exposing optimal inference
  6. A multistage modeling approach
  7. Methodology
  8. Object detection for count classification
  9. Box confidence windowing
  10. Results
  11. End-to-end performance
  12. Comparative performance
  13. Conclusion and Future Work

Figures (4)

  • Figure 1: Proposed multistage architecture. Images are first passed to a small model on the phone. If that model is uncertain about its predictions, the images are sent to a larger model in cloud. If the cloud model is uncertain, a human expert is engaged. If the phone or cloud model is confident in its estimate, the pipeline is halted and a recommendation is made.
  • Figure 2: Visualization of the dynamics between the lower and upper box confidence thresholds in the phone model.
  • Figure 3: Performance of the end-to-end system at varying abstention levels. Cells represent MCC for that phone-cloud abstention combination.
  • Figure 4: Fraction of erroneous spray recommendations at varying abstention levels. "Combined" represents our end-to-end system.