Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM BiasScope: A Real-Time Bias Analysis Platform for Comparative LLM Evaluation

Himel Ghosh, Nick Elias Werner

Abstract

As large language models (LLMs) are deployed widely, detecting and understanding bias in their outputs is critical. We present LLM BiasScope, a web application for side-by-side comparison of LLM outputs with real-time bias analysis. The system supports multiple providers (Google Gemini, DeepSeek, MiniMax, Mistral, Meituan, Meta Llama) and enables researchers and practitioners to compare models on the same prompts while analyzing bias patterns. LLM BiasScope uses a two-stage bias detection pipeline: sentence-level bias detection followed by bias type classification for biased sentences. The analysis runs automatically on both user prompts and model responses, providing statistics, visualizations, and detailed breakdowns of bias types. The interface displays two models side-by-side with synchronized streaming responses, per-model bias summaries, and a comparison view highlighting differences in bias distributions. The system is built on Next.js with React, integrates Hugging Face inference endpoints for bias detection, and uses the Vercel AI SDK for multi-provider LLM access. Features include real-time streaming, export to JSON/PDF, and interactive visualizations (bar charts, radar charts) for bias analysis. LLM BiasScope is available as an open-source web application, providing a practical tool for bias evaluation and comparative analysis of LLM behaviour.

LLM BiasScope: A Real-Time Bias Analysis Platform for Comparative LLM Evaluation

Abstract

As large language models (LLMs) are deployed widely, detecting and understanding bias in their outputs is critical. We present LLM BiasScope, a web application for side-by-side comparison of LLM outputs with real-time bias analysis. The system supports multiple providers (Google Gemini, DeepSeek, MiniMax, Mistral, Meituan, Meta Llama) and enables researchers and practitioners to compare models on the same prompts while analyzing bias patterns. LLM BiasScope uses a two-stage bias detection pipeline: sentence-level bias detection followed by bias type classification for biased sentences. The analysis runs automatically on both user prompts and model responses, providing statistics, visualizations, and detailed breakdowns of bias types. The interface displays two models side-by-side with synchronized streaming responses, per-model bias summaries, and a comparison view highlighting differences in bias distributions. The system is built on Next.js with React, integrates Hugging Face inference endpoints for bias detection, and uses the Vercel AI SDK for multi-provider LLM access. Features include real-time streaming, export to JSON/PDF, and interactive visualizations (bar charts, radar charts) for bias analysis. LLM BiasScope is available as an open-source web application, providing a practical tool for bias evaluation and comparative analysis of LLM behaviour.
Paper Structure (25 sections, 11 figures, 3 tables)

This paper contains 25 sections, 11 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. System Architecture
  4. Frontend Architecture
  5. Backend Architecture
  6. Data Flow
  7. User Interaction Flow
  8. Bias Analysis Flow
  9. Comparison Flow
  10. Evaluation
  11. Bias Detection Model Evaluations
  12. Datasets
  13. Evaluation Methodology
  14. Bias Score Extraction and Aggregation.
  15. Results
  16. ...and 10 more sections

Figures (11)

  • Figure 1: LLM BiasScope Application Home Page.
  • Figure 2: System architecture. The system uses a three-layer client–server design: (1) a React/Next.js frontend with dual chat panels for parallel LLM comparison and bias visualizations; (2) Next.js API routes handling model inference and the two-stage bias analysis pipeline; and (3) external services, including multi-provider LLMs via the Vercel AI Gateway and Hugging Face endpoints for bias detection and classification. Arrows show the flow from user input to model outputs, bias analysis, and visual comparison.
  • Figure 3: Comparison of Metrics for several bias detection models. The accuracy and F1 Score of the bias-detector show better performance compared to others.
  • Figure 4: Bias Classification: examples of some bias types as detected by the system.
  • Figure 5: Model Comparison: Bias-Types distribution from Model responses.
  • ...and 6 more figures