Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

One Search Fits All: Pareto-Optimal Eco-Friendly Model Selection

Filippo Betello, Antonio Purificato, Vittoria Vineis, Gabriele Tolomei, Fabrizio Silvestri

TL;DR

This work tackles the environmental impact of AI by addressing energy costs during model selection. It proposes GREEN, an inference-time system that predicts cross-domain model configurations via a learned predictor and selects Pareto-optimal options balancing validation performance and energy consumption, across computer vision, NLP, and recommendation systems. Central contributions include the EcoTaskSet knowledge base, the Set-Based Order Value Alignment (SOVA) ranking metric, and the two-stage Pareto-front-based optimization with online updates, all demonstrated through extensive experiments showing energy-efficient configurations with competitive performance. The approach enables scalable, domain-agnostic, and rapid model selection that can adapt to user preferences and various infrastructures, contributing to sustainable AI practices.

Abstract

The environmental impact of Artificial Intelligence (AI) is emerging as a significant global concern, particularly regarding model training. In this paper, we introduce GREEN (Guided Recommendations of Energy-Efficient Networks), a novel, inference-time approach for recommending Pareto-optimal AI model configurations that optimize validation performance and energy consumption across diverse AI domains and tasks. Our approach directly addresses the limitations of current eco-efficient neural architecture search methods, which are often restricted to specific architectures or tasks. Central to this work is EcoTaskSet, a dataset comprising training dynamics from over 1767 experiments across computer vision, natural language processing, and recommendation systems using both widely used and cutting-edge architectures. Leveraging this dataset and a prediction model, our approach demonstrates effectiveness in selecting the best model configuration based on user preferences. Experimental results show that our method successfully identifies energy-efficient configurations while ensuring competitive performance.

One Search Fits All: Pareto-Optimal Eco-Friendly Model Selection

TL;DR

This work tackles the environmental impact of AI by addressing energy costs during model selection. It proposes GREEN, an inference-time system that predicts cross-domain model configurations via a learned predictor and selects Pareto-optimal options balancing validation performance and energy consumption, across computer vision, NLP, and recommendation systems. Central contributions include the EcoTaskSet knowledge base, the Set-Based Order Value Alignment (SOVA) ranking metric, and the two-stage Pareto-front-based optimization with online updates, all demonstrated through extensive experiments showing energy-efficient configurations with competitive performance. The approach enables scalable, domain-agnostic, and rapid model selection that can adapt to user preferences and various infrastructures, contributing to sustainable AI practices.

Abstract

The environmental impact of Artificial Intelligence (AI) is emerging as a significant global concern, particularly regarding model training. In this paper, we introduce GREEN (Guided Recommendations of Energy-Efficient Networks), a novel, inference-time approach for recommending Pareto-optimal AI model configurations that optimize validation performance and energy consumption across diverse AI domains and tasks. Our approach directly addresses the limitations of current eco-efficient neural architecture search methods, which are often restricted to specific architectures or tasks. Central to this work is EcoTaskSet, a dataset comprising training dynamics from over 1767 experiments across computer vision, natural language processing, and recommendation systems using both widely used and cutting-edge architectures. Leveraging this dataset and a prediction model, our approach demonstrates effectiveness in selecting the best model configuration based on user preferences. Experimental results show that our method successfully identifies energy-efficient configurations while ensuring competitive performance.
Paper Structure (50 sections, 2 theorems, 27 equations, 5 figures, 12 tables)

This paper contains 50 sections, 2 theorems, 27 equations, 5 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Exisiting benchmarks.
  4. Eco-Aware NAS methods.
  5. GREEN
  6. Theoretical Foundations
  7. Inputs
  8. Predictive Model Learning
  9. Multi-Objective Optimization and Ranking for Best Model Selection
  10. Pareto Frontier Identification
  11. Preference-Based Filtering and Ranking
  12. Online Updates
  13. EcoTaskSet
  14. Experiments
  15. Experimental Setup
  16. ...and 35 more sections

Key Result

Lemma B.1

Figures (5)

  • Figure 1: An overview of GREEN. It takes as input features from EcoTaskSet. Then GREEN identifies energy-efficient configurations while maintaining competitive performance metrics. The output is a set of Pareto-optimal model configurations, which can be ranked according to user preferences to suggest the single best model configuration for a specific dataset, task, and computational infrastructure.
  • Figure 2: Mean and standard deviation of SOVA@k across test datasets at varying $\omega_A$. $\omega_A$ represents the weight assigned to the validation accuracy target relative to the energy target $(\omega_A +\omega_E)=1$.
  • Figure 3: Comparison of True and Predicted Pareto Fronts on CIFAR-10. Pareto-optimal configurations are shown in the normalized Validation Accuracy vs. Energy space. Each subfigure corresponds to a different percentage of discarded test data (0%, 30%, 70%), while the predictor is trained with the same seed (42) in all cases. Gray dots represent all configurations evaluated with true target values. Blue markers show the True Pareto front, orange markers the Predicted Pareto front based on the predicted value of the objectives and green markers denote Predicted Pareto configurations re-evaluated with true values. Both true and predicted fronts include only configurations achieving at least 0.9 validation accuracy, filtered based on their respective values. The x-axis (Energy) is limited to the normalized range [0.00, 0.20], and the y-axis (Validation Accuracy) spans [0.6, 1.0] to enhance clarity.
  • Figure 4: Comparison of True and Predicted Pareto Fronts on Rotten_tomatoes. Pareto-optimal configurations are shown in the normalized Validation Accuracy vs. Energy space. Each subfigure corresponds to a different percentage of discarded test data (0%, 30%, 70%), while the predictor is trained with the same seed (42) in all cases. Gray dots represent all configurations evaluated with true target values. Blue markers show the True Pareto front, orange markers the Predicted Pareto front based on the predicted value of the objectives and green markers denote Predicted Pareto configurations re-evaluated with true values. Both true and predicted fronts include only configurations achieving at least 0.45 validation accuracy, filtered based on their respective values. The x-axis (Energy) is limited to the normalized range [0.00, 0.20], and the y-axis (Validation Accuracy) spans [0.4, 1.0] to enhance clarity.
  • Figure 5: Comparison of True and Predicted Pareto Fronts on FS-TKY. Pareto-optimal configurations are shown in the normalized Validation Accuracy vs. Energy space. Each subfigure corresponds to a different percentage of discarded test data (0%, 30%, 70%), while the predictor is trained with the same seed (42) in all cases. Gray dots represent all configurations evaluated with true target values. Blue markers show the True Pareto front, orange markers the Predicted Pareto front based on the predicted value of the objectives and green markers denote Predicted Pareto configurations re-evaluated with true values. Both true and predicted fronts include only configurations achieving at least 0.9 validation accuracy, filtered based on their respective values. The x-axis (Energy) is limited to the normalized range [0.00, 0.20], and the y-axis (Validation Accuracy) spans [0.80, 1.0] to enhance clarity.

Theorems & Definitions (8)

  • Remark 3.2
  • Definition 5.1
  • Lemma B.1
  • proof
  • Lemma B.2
  • proof
  • Definition B.3: SOVA@k with Ties in Ranks
  • proof