Latenrgy: Model Agnostic Latency and Energy Consumption Prediction for Binary Classifiers

TL;DR

This paper tackles inference-time compute overhead under Responsible AI guardrails by developing a model-agnostic latency-energy framework. It introduces a general predictor $O=f(A,D,G)$ that combines classifier type $A$, dataset characteristics $D$, and guardrails $G$ to produce latency $L$ and energy $E$ through two explicit prediction equations, enabling cross-model benchmarking and practical deployment planning. By fusing theoretical variables with guardrail costs, the work provides a foundational approach to balancing computational efficiency with ethical robustness and outlines clear steps for empirical validation. The framework aims to inform both researchers and practitioners on how to design scalable, responsible binary classifiers for resource-constrained environments.

Abstract

Machine learning systems increasingly drive innovation across scientific fields and industry, yet challenges in compute overhead, specifically during inference, limit their scalability and sustainability. Responsible AI guardrails, essential for ensuring fairness, transparency, and privacy, further exacerbate these computational demands. This study addresses critical gaps in the literature, chiefly the lack of generalized predictive techniques for latency and energy consumption, limited cross-comparisons of classifiers, and unquantified impacts of RAI guardrails on inference performance. Using Theory Construction Methodology, this work constructed a model-agnostic theoretical framework for predicting latency and energy consumption in binary classification models during inference. The framework synthesizes classifier characteristics, dataset properties, and RAI guardrails into a unified analytical instrument. Two predictive equations are derived that capture the interplay between these factors while offering generalizability across diverse classifiers. The proposed framework provides foundational insights for designing efficient, responsible ML systems. It enables researchers to benchmark and optimize inference performance and assists practitioners in deploying scalable solutions. Finally, this work establishes a theoretical foundation for balancing computational efficiency with ethical AI principles, paving the way for future empirical validation and broader applications.