Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Biases in Edge Language Models: Detection, Analysis, and Mitigation

Vinamra Sharma, Danilo Pietro Pau, José Cano

TL;DR

This work evaluates text-based bias across cloud, desktop, and edge deployments of LLMs under repeated prompts, revealing that edge-optimized models exhibit higher bias. It leverages pruning and INT8 quantization on Llama-2 7B for Raspberry Pi 4 and contrasts results with cloud (GPT-4o-mini, Gemini-1.5-flash, Grok-beta) and desktop (Gemma2, Mistral) baselines. A context-aware, layer-wise feedback loop is proposed to mitigate bias during inference without retraining, achieving a reported $79.28%$ bias reduction at the cost of increased memory and latency. The findings underscore the importance of bias monitoring in edge AI and motivate development of memory-efficient mitigation techniques for resource-constrained deployments.

Abstract

The integration of large language models (LLMs) on low-power edge devices such as Raspberry Pi, known as edge language models (ELMs), has introduced opportunities for more personalized, secure, and low-latency language intelligence that is accessible to all. However, the resource constraints inherent in edge devices and the lack of robust ethical safeguards in language models raise significant concerns about fairness, accountability, and transparency in model output generation. This paper conducts a comparative analysis of text-based bias across language model deployments on edge, cloud, and desktop environments, aiming to evaluate how deployment settings influence model fairness. Specifically, we examined an optimized Llama-2 model running on a Raspberry Pi 4; GPT 4o-mini, Gemini-1.5-flash, and Grok-beta models running on cloud servers; and Gemma2 and Mistral models running on a MacOS desktop machine. Our results demonstrate that Llama-2 running on Raspberry Pi 4 is 43.23% and 21.89% more prone to showing bias over time compared to models running on the desktop and cloud-based environments. We also propose the implementation of a feedback loop, a mechanism that iteratively adjusts model behavior based on previous outputs, where predefined constraint weights are applied layer-by-layer during inference, allowing the model to correct bias patterns, resulting in 79.28% reduction in model bias.

Biases in Edge Language Models: Detection, Analysis, and Mitigation

TL;DR

This work evaluates text-based bias across cloud, desktop, and edge deployments of LLMs under repeated prompts, revealing that edge-optimized models exhibit higher bias. It leverages pruning and INT8 quantization on Llama-2 7B for Raspberry Pi 4 and contrasts results with cloud (GPT-4o-mini, Gemini-1.5-flash, Grok-beta) and desktop (Gemma2, Mistral) baselines. A context-aware, layer-wise feedback loop is proposed to mitigate bias during inference without retraining, achieving a reported bias reduction at the cost of increased memory and latency. The findings underscore the importance of bias monitoring in edge AI and motivate development of memory-efficient mitigation techniques for resource-constrained deployments.

Abstract

The integration of large language models (LLMs) on low-power edge devices such as Raspberry Pi, known as edge language models (ELMs), has introduced opportunities for more personalized, secure, and low-latency language intelligence that is accessible to all. However, the resource constraints inherent in edge devices and the lack of robust ethical safeguards in language models raise significant concerns about fairness, accountability, and transparency in model output generation. This paper conducts a comparative analysis of text-based bias across language model deployments on edge, cloud, and desktop environments, aiming to evaluate how deployment settings influence model fairness. Specifically, we examined an optimized Llama-2 model running on a Raspberry Pi 4; GPT 4o-mini, Gemini-1.5-flash, and Grok-beta models running on cloud servers; and Gemma2 and Mistral models running on a MacOS desktop machine. Our results demonstrate that Llama-2 running on Raspberry Pi 4 is 43.23% and 21.89% more prone to showing bias over time compared to models running on the desktop and cloud-based environments. We also propose the implementation of a feedback loop, a mechanism that iteratively adjusts model behavior based on previous outputs, where predefined constraint weights are applied layer-by-layer during inference, allowing the model to correct bias patterns, resulting in 79.28% reduction in model bias.

Paper Structure

This paper contains 16 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLMs at the Edge
  4. LLMs and bias
  5. Methodology
  6. Edge LLMs setup and optimization
  7. Pruning
  8. Quantization ($INT8$)
  9. Feedback loop
  10. Cloud LLMs Setup
  11. Desktop LLMs Setup
  12. Evaluation
  13. Output comparison between various deployments
  14. Impact of feedback loop on ELM implementation
  15. Discussion
  16. ...and 1 more sections

Figures (3)

  • Figure 1: High-level LLM model API interaction architecture on a Google Cloud Platform (GCP) hosted environment.
  • Figure 2: Average decision distribution of the Grok-beta, Gemini-1.5-flash, and gpt-4o-mini models running in a cloud environment compared with the Llama 2.0 ($INT8$) model running on a Raspberry Pi.
  • Figure 3: Average decision distribution of the gemma2 and mistral models running on a Mac computer compared with the Llama 2.0 ($INT8$) model running on a Raspberry Pi.