HCQA-1.5 @ Ego4D EgoSchema Challenge 2025

TL;DR

This work targets reliable long-form egocentric video question answering on the EgoSchema benchmark. It extends the HCQA framework with a two-stage approach: first, a multi-source LLM ensemble (Gemini-1.5-Pro, GPT-4.1, Qwen2.5) produces diverse predictions with a confidence-based filter; second, low-confidence cases undergo fine-grained reasoning via vision-based analysis (45 frames processed by Qwen2.5-VL-72B) and thought-based textual reasoning (using DeepSeek-R1). The method achieves 77% accuracy, ranking third on EgoSchema, and ablations show gains from both ensemble and refinement stages (full pipeline reaching 77.3%). The work demonstrates improved reliability in challenging egocentric QA and provides a public codebase for reproduction, signaling practical impact for robust video-language reasoning in first-person perception contexts.

Abstract

In this report, we present the method that achieves third place for Ego4D EgoSchema Challenge in CVPR 2025. To improve the reliability of answer prediction in egocentric video question answering, we propose an effective extension to the previously proposed HCQA framework. Our approach introduces a multi-source aggregation strategy to generate diverse predictions, followed by a confidence-based filtering mechanism that selects high-confidence answers directly. For low-confidence cases, we incorporate a fine-grained reasoning module that performs additional visual and contextual analysis to refine the predictions. Evaluated on the EgoSchema blind test set, our method achieves 77% accuracy on over 5,000 human-curated multiple-choice questions, outperforming last year's winning solution and the majority of participating teams. Our code will be added at https://github.com/Hyu-Zhang/HCQA.

Figures (2)

• Figure 1: An illustration of two-stage decision-making process.
• Figure 2: One failed example of our framework on EgoSchema subset.