Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bridging the Perception Gap: A Lightweight Coarse-to-Fine Architecture for Edge Audio Systems

Hengfan Zhang, Yueqian Lin, Hai Helen Li, Yiran Chen

TL;DR

The paper tackles the perceptual gap in edge audio understanding, where single-pass Audio-LLMs struggle with multi-step reasoning under tight resource and privacy constraints. It introduces CoFi-Agent, a coarse-to-fine, hybrid edge-cloud architecture that performs a fast local perception and conditionally triggers lightweight cloud-guided refinement using on-device tools like temporal re-listening and local ASR while keeping raw audio on-device. On MMAR (N=1,000), it raises accuracy from 27.20% to 53.60% and reduces unnecessary cloud offloading by gating about 62% of samples, demonstrating a favorable accuracy-latency trade-off. This approach demonstrates practical edge deployment viability by maximizing informational value from compact evidence while preserving acoustic privacy.

Abstract

Deploying Audio-Language Models (Audio-LLMs) on edge infrastructure exposes a persistent tension between perception depth and computational efficiency. Lightweight local models tend to produce passive perception - generic summaries that miss the subtle evidence required for multi-step audio reasoning - while indiscriminate cloud offloading incurs unacceptable latency, bandwidth cost, and privacy risk. We propose CoFi-Agent (Tool-Augmented Coarse-to-Fine Agent), a hybrid architecture targeting edge servers and gateways. It performs fast local perception and triggers conditional forensic refinement only when uncertainty is detected. CoFi-Agent runs an initial single-pass on a local 7B Audio-LLM, then a cloud controller gates difficult cases and issues lightweight plans for on-device tools such as temporal re-listening and local ASR. On the MMAR benchmark, CoFi-Agent improves accuracy from 27.20% to 53.60%, while achieving a better accuracy-efficiency trade-off than an always-on investigation pipeline. Overall, CoFi-Agent bridges the perception gap via tool-enabled, conditional edge-cloud collaboration under practical system constraints.

Bridging the Perception Gap: A Lightweight Coarse-to-Fine Architecture for Edge Audio Systems

TL;DR

The paper tackles the perceptual gap in edge audio understanding, where single-pass Audio-LLMs struggle with multi-step reasoning under tight resource and privacy constraints. It introduces CoFi-Agent, a coarse-to-fine, hybrid edge-cloud architecture that performs a fast local perception and conditionally triggers lightweight cloud-guided refinement using on-device tools like temporal re-listening and local ASR while keeping raw audio on-device. On MMAR (N=1,000), it raises accuracy from 27.20% to 53.60% and reduces unnecessary cloud offloading by gating about 62% of samples, demonstrating a favorable accuracy-latency trade-off. This approach demonstrates practical edge deployment viability by maximizing informational value from compact evidence while preserving acoustic privacy.

Abstract

Deploying Audio-Language Models (Audio-LLMs) on edge infrastructure exposes a persistent tension between perception depth and computational efficiency. Lightweight local models tend to produce passive perception - generic summaries that miss the subtle evidence required for multi-step audio reasoning - while indiscriminate cloud offloading incurs unacceptable latency, bandwidth cost, and privacy risk. We propose CoFi-Agent (Tool-Augmented Coarse-to-Fine Agent), a hybrid architecture targeting edge servers and gateways. It performs fast local perception and triggers conditional forensic refinement only when uncertainty is detected. CoFi-Agent runs an initial single-pass on a local 7B Audio-LLM, then a cloud controller gates difficult cases and issues lightweight plans for on-device tools such as temporal re-listening and local ASR. On the MMAR benchmark, CoFi-Agent improves accuracy from 27.20% to 53.60%, while achieving a better accuracy-efficiency trade-off than an always-on investigation pipeline. Overall, CoFi-Agent bridges the perception gap via tool-enabled, conditional edge-cloud collaboration under practical system constraints.
Paper Structure (27 sections, 5 equations, 3 figures, 2 tables)

This paper contains 27 sections, 5 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Audio-Language Models(Audio-LLMs)
  4. Agentiv AI and Tool Use
  5. Efficient Edge Intelligence
  6. Methodology
  7. Stage 0: Edge Coarse Perception
  8. Adaptive Confidence Gate
  9. Stage 1: Cloud-Guided Refinement Planning
  10. Stage 2: On-Device Tool Execution
  11. Tool: Temporal Re-listening
  12. Tool: On-Device ASR (Whisper)
  13. Evidence Integration and Verdict
  14. Privacy and Bandwidth Efficiency
  15. Experimental Evaluation
  16. ...and 12 more sections

Figures (3)

  • Figure 1: Accuracy--efficiency trade-off on MMAR (N=1,000). Adaptive gating achieves higher accuracy and lower average latency than always-on investigation.
  • Figure 2: CoFi-Agent overview. A local coarse perception model answers easy queries via a Fast Path. A cloud confidence gate escalates uncertain cases and emits lightweight refinement plans for on-device tools (temporal re-listening and ASR). Raw audio remains on-device; only compact evidence (e.g., transcripts, tool summaries) is shared for cloud reasoning.
  • Figure 3: Distribution of inference paths under CoFi-Agent adaptive gating on MMAR (N=1,000).