Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

InterventionLens: A Multi-Agent Framework for Detecting ASD Intervention Strategies in Parent-Child Shared Reading

Xiao Wang, Lu Dong, Ifeoma Nwogu, Srirangaraj Setlur, Venu Govindaraju

Abstract

Home-based interventions like parent-child shared reading provide a cost-effective approach for supporting children with autism spectrum disorder (ASD). However, analyzing caregiver intervention strategies in naturalistic home interactions typically relies on expert annotation, which is costly, time-intensive, and difficult to scale. To address this challenge, we propose InterventionLens, an end-to-end multi-agent system for automatically detecting and temporally segmenting caregiver intervention strategies from shared reading videos. Without task-specific model training or fine-tuning, InterventionLens uses a collaborative multi-agent architecture to integrate multimodal interaction content and perform fine-grained strategy analysis. Experiments on the ASD-HI dataset show that InterventionLens achieves an overall F1 score of 79.44\%, outperforming the baseline by 19.72\%. These results suggest that InterventionLens is a promising system for analyzing caregiver intervention strategies in home-based ASD shared reading settings. Additional resources will be released on the project page.

InterventionLens: A Multi-Agent Framework for Detecting ASD Intervention Strategies in Parent-Child Shared Reading

Abstract

Home-based interventions like parent-child shared reading provide a cost-effective approach for supporting children with autism spectrum disorder (ASD). However, analyzing caregiver intervention strategies in naturalistic home interactions typically relies on expert annotation, which is costly, time-intensive, and difficult to scale. To address this challenge, we propose InterventionLens, an end-to-end multi-agent system for automatically detecting and temporally segmenting caregiver intervention strategies from shared reading videos. Without task-specific model training or fine-tuning, InterventionLens uses a collaborative multi-agent architecture to integrate multimodal interaction content and perform fine-grained strategy analysis. Experiments on the ASD-HI dataset show that InterventionLens achieves an overall F1 score of 79.44\%, outperforming the baseline by 19.72\%. These results suggest that InterventionLens is a promising system for analyzing caregiver intervention strategies in home-based ASD shared reading settings. Additional resources will be released on the project page.
Paper Structure (15 sections, 5 equations, 1 figure, 4 tables)

This paper contains 15 sections, 5 equations, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation
  4. Method
  5. Perception Layer
  6. Intervention Detection & Segmentation Layer
  7. Progressive Knowledge Refinement for SSEs
  8. Experiments
  9. Experimental Setup
  10. Main Results
  11. Ablation Study
  12. Ablation on the Book Modeling Agent (BMA).
  13. Error Analysis
  14. Conclusion and Limitations
  15. Ethical Considerations

Figures (1)

  • Figure 1: Overview of InterventionLens. The framework consists of a Perception Layer and an Intervention Detection & Segmentation Layer (IDS). The Perception Layer first converts the raw shared book reading video into a time-aligned multimodal transcript and structured book context. The IDS Layer then proposes candidate intervention segments and routes them to strategy-specific experts, each of which performs strategy-specific verification and boundary refinement to produce the final temporally localized intervention segments.