Benchmarking News Recommendation in the Era of Green AI
Qijiong Liu, Jieming Zhu, Quanyu Dai, Xiao-Ming Wu
TL;DR
This work tackles the lack of standardized benchmarks for news recommender systems and their environmental footprint by introducing GreenRec, the first Green AI benchmark for news recommendation. It evaluates 30 baselines across end-to-end and only-encode-once (OLEO) paradigms on MIND-small and MIND-large datasets, integrating sustainability metrics CO2E and ApC (ApC = ((AUC - 50) / CO2E) × 100). Results show that OLEO delivers competitive accuracy relative to state-of-the-art end-to-end PLM-NR methods while achieving substantial reductions in carbon emissions, with up to 2992% improvement in the sustainability metric. The benchmark provides a practical, domain-specific framework for eco-friendly news recommendation research and encourages broader adoption of OLEO in future work.
Abstract
Over recent years, news recommender systems have gained significant attention in both academia and industry, emphasizing the need for a standardized benchmark to evaluate and compare the performance of these systems. Concurrently, Green AI advocates for reducing the energy consumption and environmental impact of machine learning. To address these concerns, we introduce the first Green AI benchmarking framework for news recommendation, known as GreenRec, and propose a metric for assessing the tradeoff between recommendation accuracy and efficiency. Our benchmark encompasses 30 base models and their variants, covering traditional end-to-end training paradigms as well as our proposed efficient only-encode-once (OLEO) paradigm. Through experiments consuming 2000 GPU hours, we observe that the OLEO paradigm achieves competitive accuracy compared to state-of-the-art end-to-end paradigms and delivers up to a 2992\% improvement in sustainability metrics.