Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CPJ: Explainable Agricultural Pest Diagnosis via Caption-Prompt-Judge with LLM-Judged Refinement

Wentao Zhang, Tao Fang, Lina Lu, Lifei Wang, Weihe Zhong

TL;DR

CPJ introduces a training-free, caption-grounded VQA framework for explainable agricultural pest diagnosis. It integrates generative multi-angle captions, task-specific VQA, and LLM-based judge selection to produce dual outputs—disease recognition and management knowledge—with transparent reasoning. On CDDMBench, CPJ achieves substantial gains, e.g., +22.7pp in disease classification and +19.5 points in QA when using GPT-5-mini captions, without any fine-tuning, demonstrating robustness to domain shifts and improved interpretability. The approach yields evidence-based, human-readable diagnostics and is openly available for replication and deployment in field settings.

Abstract

Accurate and interpretable crop disease diagnosis is essential for agricultural decision-making, yet existing methods often rely on costly supervised fine-tuning and perform poorly under domain shifts. We propose Caption--Prompt--Judge (CPJ), a training-free few-shot framework that enhances Agri-Pest VQA through structured, interpretable image captions. CPJ employs large vision-language models to generate multi-angle captions, refined iteratively via an LLM-as-Judge module, which then inform a dual-answer VQA process for both recognition and management responses. Evaluated on CDDMBench, CPJ significantly improves performance: using GPT-5-mini captions, GPT-5-Nano achieves \textbf{+22.7} pp in disease classification and \textbf{+19.5} points in QA score over no-caption baselines. The framework provides transparent, evidence-based reasoning, advancing robust and explainable agricultural diagnosis without fine-tuning. Our code and data are publicly available at: https://github.com/CPJ-Agricultural/CPJ-Agricultural-Diagnosis.

CPJ: Explainable Agricultural Pest Diagnosis via Caption-Prompt-Judge with LLM-Judged Refinement

TL;DR

CPJ introduces a training-free, caption-grounded VQA framework for explainable agricultural pest diagnosis. It integrates generative multi-angle captions, task-specific VQA, and LLM-based judge selection to produce dual outputs—disease recognition and management knowledge—with transparent reasoning. On CDDMBench, CPJ achieves substantial gains, e.g., +22.7pp in disease classification and +19.5 points in QA when using GPT-5-mini captions, without any fine-tuning, demonstrating robustness to domain shifts and improved interpretability. The approach yields evidence-based, human-readable diagnostics and is openly available for replication and deployment in field settings.

Abstract

Accurate and interpretable crop disease diagnosis is essential for agricultural decision-making, yet existing methods often rely on costly supervised fine-tuning and perform poorly under domain shifts. We propose Caption--Prompt--Judge (CPJ), a training-free few-shot framework that enhances Agri-Pest VQA through structured, interpretable image captions. CPJ employs large vision-language models to generate multi-angle captions, refined iteratively via an LLM-as-Judge module, which then inform a dual-answer VQA process for both recognition and management responses. Evaluated on CDDMBench, CPJ significantly improves performance: using GPT-5-mini captions, GPT-5-Nano achieves \textbf{+22.7} pp in disease classification and \textbf{+19.5} points in QA score over no-caption baselines. The framework provides transparent, evidence-based reasoning, advancing robust and explainable agricultural diagnosis without fine-tuning. Our code and data are publicly available at: https://github.com/CPJ-Agricultural/CPJ-Agricultural-Diagnosis.
Paper Structure (17 sections, 5 equations, 2 figures, 2 tables)

This paper contains 17 sections, 5 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. Agricultural Vision-Language Models
  4. Explainable AI in Vision-Language Models
  5. METHODOLOGY
  6. Image Interpretability and Caption Enhancement
  7. Explanational caption-Optimized VQA
  8. Crop Disease Classification Task
  9. Crop Disease Knowledge Question Answering Task
  10. LLM-as-a-Judge Answer Selection
  11. Experiments and Analysis
  12. Dataset and Evaluation
  13. Model Selection and Settings
  14. Main Results
  15. Ablation Study
  16. ...and 2 more sections

Figures (2)

  • Figure 1: Overview of the "CPJ" pipeline for explainable Agri-Pest VQA, featuring three cohesive stages: (i) Generative Explanational Captioning: A large vision-language model analyzes input crop disease images to generate multi-angle descriptive captions detailing plant morphology, disease symptoms, severity levels, and diagnostic uncertainties. These captions undergo iterative refinement via LLM-as-a-Judge evaluation to ensure accuracy, completeness, and neutrality. (ii) Task-Specific Prompt-Based VQA Generation: Using the refined captions alongside original images and few-shot exemplars, the VQA model generates dual complementary answers addressing both disease recognition (crop type, disease identification, visual features) and actionable management guidance (treatment methods, prevention strategies). (iii) LLM-as-a-Judge Answer Selection: A stronger LLM evaluates both candidate answers against reference answers using multi-dimensional criteria (factual accuracy, completeness, specificity, practical relevance), selecting the superior response to deliver reliable, interpretable diagnostic recommendations.
  • Figure 2: Performance trends in the ablation study using Qwen2.5-VL-72B-Instruct with Explanational Captions (EC). The abbreviations are as follows: EC for Explanational Caption, CC for Crop Classification, DC for Disease Classification, CDK for Crop Disease Knowledge, Qwen for Qwen-VL-Chat, and GPT for GPT-5-Nano.