ReCCur: A Recursive Corner-Case Curation Framework for Robust Vision-Language Understanding in Open and Edge Scenarios

TL;DR

ReCCur (Recursive Corner-Case Curation), a low-compute framework that converts noisy web imagery into auditable fine-grained labels via a multi-agent recursive pipeline, is presented.

Abstract

Corner cases are rare or extreme scenarios that drive real-world failures, but they are difficult to curate at scale: web data are noisy, labels are brittle, and edge deployments preclude large retraining. We present ReCCur (Recursive Corner-Case Curation), a low-compute framework that converts noisy web imagery into auditable fine-grained labels via a multi-agent recursive pipeline. First, large-scale data acquisition and filtering expands a domain vocabulary with a vision-language model (VLM), crawls the web, and enforces tri-modal (image, description, keyword) consistency with light human spot checks to yield refined candidates. Next, mixture-of-experts knowledge distillation uses complementary encoders (e.g., CLIP, DINOv2, BEiT) for kNN voting with dual-confidence activation and uncertainty sampling, converging to a high-precision set. Finally, region-evidence VLM adversarial labeling pairs a proposer (multi-granularity regions and semantic cues) with a validator (global and local chained consistency) to produce explainable labels and close the loop. On realistic corner-case scenarios (e.g., flooded-car inspection), ReCCur runs on consumer-grade GPUs, steadily improves purity and separability, and requires minimal human supervision, providing a practical substrate for downstream training and evaluation under resource constraints. Code and dataset will be released.

Figures (15)

• Figure 1: Overview of ReCCur. Multimodal acquisition/filtering, MoE labeling (with confidence + uncertainty), and region-evidence VLM progressively purify data and refine labels.
• Figure 2: Overall framework of Large-Scale Data Acquisition and Filtering. LLM-expanded keywords guide crawling; the Human Supervision Loop refines prompts, adds descriptors, and categorizes samples into save/mixed/delete; vision–language clustering prunes noise to produce a pseudo-labeled, high-quality dataset.
• Figure 3: Overall Structure of Mixture-of-Experts Knowledge Distillation. Human-labeled images are embedded into a vector index. MoE predicts $\hat{y}$ with topic/label confidences; uncertainty sampling escalates low-confidence cases to annotators, and confidence activation decodes the final decision.
• Figure 4: Architecture of the VLM Proposer. The Proposer outputs Multi-Head-Region grids, isolates the subject and subregions, and annotates each with semantic cues $\boldsymbol{\Phi}$.
• Figure 5: Architecture of the VLM Validator. The validator performs global inference, fuses evidence via chained reasoning, and outputs $\hat{y}_{\text{sem}}$.