EcoThink: A Green Adaptive Inference Framework for Sustainable and Accessible Agents

Abstract

As the Web transitions from static retrieval to generative interaction, the escalating environmental footprint of Large Language Models (LLMs) presents a critical sustainability challenge. Current paradigms indiscriminately apply computation-intensive strategies like Chain-of-Thought (CoT) to billions of daily queries, causing LLM overthinking, a redundancy that amplifies carbon emissions and operational barriers. This inefficiency directly undermines UN Sustainable Development Goals 13 (Climate Action) and 10 (Reduced Inequalities) by hindering equitable AI access in resource-constrained regions. To address this, we introduce EcoThink, an energy-aware adaptive inference framework designed to reconcile high-performance AI intelligence with environmental responsibility. EcoThink employs a lightweight, distillation-based router to dynamically assess query complexity, skipping unnecessary reasoning for factoid retrieval while reserving deep computation for complex logic. Extensive evaluations across 9 diverse benchmarks demonstrate that EcoThink reduces inference energy by 40.4% on average (up to 81.9% for web knowledge retrieval) without statistically significant performance loss. By mitigating algorithmic waste, EcoThink offers a scalable path toward a sustainable, inclusive, and energy-efficient generative AI Agent.

Figures (5)

• Figure 1: Quantitative comparison of inference carbon emissions. We benchmark the proposed EcoThink against state-of-the-art proprietary models and open-source baselines.
• Figure 2: The EcoThink framework for energy-aware adaptive inference. The top panel illustrates the high-level architecture where a decision router dynamically directs queries to either a low-energy path or a computation-intensive path. The bottom panel details the workflow of hybrid retrieval for the Green Path and an adaptive CoT mechanism for the Deep Path.
• Figure 3: Sensitivity analysis of the router threshold $\gamma$ illustrating the trade-off between model accuracy and energy efficiency. The dual-axis plot shows the average accuracy (red line, left axis) and energy saving percentage (blue line, right axis) as a function of $\gamma$. The vertical dashed line indicates the empirically determined optimal threshold ($\gamma=0.5$), achieving a balanced performance of 89.6% accuracy with 41.9% energy savings.
• Figure 4: Energy Consumption Breakdown Analysis. Normalizing the Standard CoT baseline to 100%, this stacked bar chart illustrates the distribution of computational energy expenditure. EcoThink achieves a $\sim$40% total saving not by magic, but by structurally replacing the majority of expensive "Deep Reasoning Compute" (Red) with highly efficient "Green Path Compute" (Green). The overhead introduced by the routing mechanism (Gray) is negligible compared to the resulting energy gains.
• Figure 5: A Simple case of LLM overthinking.