Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Large Reasoning Models for Agriculture

Hossein Zaremehrjerdi, Shreyan Ganguly, Ashlyn Rairdin, Elizabeth Tranel, Benjamin Feuer, Juan Ignacio Di Salvo, Srikanth Panthulugiri, Hernan Torres Pacin, Victoria Moser, Sarah Jones, Joscif G Raigne, Yanben Shen, Heidi M. Dornath, Aditya Balu, Adarsh Krishnamurthy, Asheesh K Singh, Arti Singh, Baskar Ganapathysubramanian, Chinmay Hegde, Soumik Sarkar

TL;DR

This paper introduces AgThoughts, a large, expert-in-the-loop repository of 44.6K agricultural Q&A pairs with reasoning traces, and AgReason, a 100-question open-ended benchmark designed to evaluate context-rich agronomic reasoning. It demonstrates that large reasoning models outperform standard LLMs on agricultural reasoning tasks, yet performance remains limited (best baseline ~36% accuracy), motivating the development of AgThinker, a family of small, domain-adapted models trained with LoRA on consumer hardware. The authors propose an LLM-as-Judge evaluation framework and show that extensive expert curation and reasoning traces can unlock domain-specific reasoning in LLMs, while also revealing substantial room for improvement and future work in multimodal and expanded geographic coverage. Overall, the work provides a concrete, expert-driven path for advancing agricultural decision-support AI with specialized datasets, benchmarks, and lightweight models capable of real-world deployment.

Abstract

Agricultural decision-making involves complex, context-specific reasoning, where choices about crops, practices, and interventions depend heavily on geographic, climatic, and economic conditions. Traditional large language models (LLMs) often fall short in navigating this nuanced problem due to limited reasoning capacity. We hypothesize that recent advances in large reasoning models (LRMs) can better handle such structured, domain-specific inference. To investigate this, we introduce AgReason, the first expert-curated open-ended science benchmark with 100 questions for agricultural reasoning. Evaluations across thirteen open-source and proprietary models reveal that LRMs outperform conventional ones, though notable challenges persist, with the strongest Gemini-based baseline achieving 36% accuracy. We also present AgThoughts, a large-scale dataset of 44.6K question-answer pairs generated with human oversight and equipped with synthetically generated reasoning traces. Using AgThoughts, we develop AgThinker, a suite of small reasoning models that can be run on consumer-grade GPUs, and show that our dataset can be effective in unlocking agricultural reasoning abilities in LLMs. Our project page is here: https://baskargroup.github.io/Ag_reasoning/

Towards Large Reasoning Models for Agriculture

TL;DR

This paper introduces AgThoughts, a large, expert-in-the-loop repository of 44.6K agricultural Q&A pairs with reasoning traces, and AgReason, a 100-question open-ended benchmark designed to evaluate context-rich agronomic reasoning. It demonstrates that large reasoning models outperform standard LLMs on agricultural reasoning tasks, yet performance remains limited (best baseline ~36% accuracy), motivating the development of AgThinker, a family of small, domain-adapted models trained with LoRA on consumer hardware. The authors propose an LLM-as-Judge evaluation framework and show that extensive expert curation and reasoning traces can unlock domain-specific reasoning in LLMs, while also revealing substantial room for improvement and future work in multimodal and expanded geographic coverage. Overall, the work provides a concrete, expert-driven path for advancing agricultural decision-support AI with specialized datasets, benchmarks, and lightweight models capable of real-world deployment.

Abstract

Agricultural decision-making involves complex, context-specific reasoning, where choices about crops, practices, and interventions depend heavily on geographic, climatic, and economic conditions. Traditional large language models (LLMs) often fall short in navigating this nuanced problem due to limited reasoning capacity. We hypothesize that recent advances in large reasoning models (LRMs) can better handle such structured, domain-specific inference. To investigate this, we introduce AgReason, the first expert-curated open-ended science benchmark with 100 questions for agricultural reasoning. Evaluations across thirteen open-source and proprietary models reveal that LRMs outperform conventional ones, though notable challenges persist, with the strongest Gemini-based baseline achieving 36% accuracy. We also present AgThoughts, a large-scale dataset of 44.6K question-answer pairs generated with human oversight and equipped with synthetically generated reasoning traces. Using AgThoughts, we develop AgThinker, a suite of small reasoning models that can be run on consumer-grade GPUs, and show that our dataset can be effective in unlocking agricultural reasoning abilities in LLMs. Our project page is here: https://baskargroup.github.io/Ag_reasoning/

Paper Structure

This paper contains 31 sections, 1 equation, 9 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Data generation and curation pipeline
  4. Question generation workflow
  5. Response generation and curation
  6. Dataset, benchmark, and reasoning models
  7. The AgThoughts Dataset.
  8. The AgReason Benchmark.
  9. LLM-as-Judge design for benchmark evaluation.
  10. AgThinker models.
  11. Experimental results
  12. Benchmark results
  13. Category-based results
  14. Impact of supervised fine-tuning on model performance
  15. Discussion
  16. ...and 16 more sections

Figures (9)

  • Figure 1: Workflow for the development of AgThoughts and AgReason: (1) Base templates are expanded into detailed question--answer pairs using LLMs; (2) Expert feedback on 200 sampled examples identifies common issues; (3) Expert and LLM-based feedback is used to iteratively filter and finalize 44.6k Q&A pairs; (4) The AgReason Benchmark evaluates candidate LLM performance using 100 questions with expert-curated gold-standard answers using LLM-as-judge.
  • Figure 2: Ten expert-defined agronomic question categories used in AgThoughts and AgReason.
  • Figure 3: Percentage of questions with F1 scores above varying thresholds.
  • Figure 4: Question‑level pass rate ($F_1 \ge 0.8$) for the base and AgThinker models.
  • Figure 5: Performance of models across each question category. Category-level pass rates (F$_1 \geq$ 0.8) highlight the superior performance of reasoning models.
  • ...and 4 more figures