Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bielik Guard: Efficient Polish Language Safety Classifiers for LLM Content Moderation

Krzysztof Wróbel, Jan Maria Kowalski, Jerzy Surma, Igor Ciuciura, Maciej Szymański

TL;DR

Polish safety tooling for LLMs is undersupplied, motivating Bielik Guard, a pair of compact RoBERTa-based safety classifiers trained on a community-annotated Polish dataset across five categories. The 0.5B variant achieves best overall discrimination (F1 micro 0.791, macro 0.785), while the 0.1B variant delivers exceptional efficiency with high precision (77.65%) and very low false positives (0.63%) on real prompts. The approach leverages language-specific data, a five-category taxonomy, and threshold calibration to outperform larger multilingual systems such as HerBERT-PL-Guard and Llama Guard in Polish contexts. The models are publicly available and deployed, illustrating practical impact for safety tooling in low-resource languages and offering a blueprint for community-driven data collection in other languages.

Abstract

As Large Language Models (LLMs) become increasingly deployed in Polish language applications, the need for efficient and accurate content safety classifiers has become paramount. We present Bielik Guard, a family of compact Polish language safety classifiers comprising two model variants: a 0.1B parameter model based on MMLW-RoBERTa-base and a 0.5B parameter model based on PKOBP/polish-roberta-8k. Fine-tuned on a community-annotated dataset of 6,885 Polish texts, these models classify content across five safety categories: Hate/Aggression, Vulgarities, Sexual Content, Crime, and Self-Harm. Our evaluation demonstrates that both models achieve strong performance on multiple benchmarks. The 0.5B variant offers the best overall discrimination capability with F1 scores of 0.791 (micro) and 0.785 (macro) on the test set, while the 0.1B variant demonstrates exceptional efficiency. Notably, Bielik Guard 0.1B v1.1 achieves superior precision (77.65\%) and very low false positive rate (0.63\%) on real user prompts, outperforming HerBERT-PL-Guard (31.55\% precision, 4.70\% FPR) despite identical model size. The models are publicly available and designed to provide appropriate responses rather than simple content blocking, particularly for sensitive categories like self-harm.

Bielik Guard: Efficient Polish Language Safety Classifiers for LLM Content Moderation

TL;DR

Polish safety tooling for LLMs is undersupplied, motivating Bielik Guard, a pair of compact RoBERTa-based safety classifiers trained on a community-annotated Polish dataset across five categories. The 0.5B variant achieves best overall discrimination (F1 micro 0.791, macro 0.785), while the 0.1B variant delivers exceptional efficiency with high precision (77.65%) and very low false positives (0.63%) on real prompts. The approach leverages language-specific data, a five-category taxonomy, and threshold calibration to outperform larger multilingual systems such as HerBERT-PL-Guard and Llama Guard in Polish contexts. The models are publicly available and deployed, illustrating practical impact for safety tooling in low-resource languages and offering a blueprint for community-driven data collection in other languages.

Abstract

As Large Language Models (LLMs) become increasingly deployed in Polish language applications, the need for efficient and accurate content safety classifiers has become paramount. We present Bielik Guard, a family of compact Polish language safety classifiers comprising two model variants: a 0.1B parameter model based on MMLW-RoBERTa-base and a 0.5B parameter model based on PKOBP/polish-roberta-8k. Fine-tuned on a community-annotated dataset of 6,885 Polish texts, these models classify content across five safety categories: Hate/Aggression, Vulgarities, Sexual Content, Crime, and Self-Harm. Our evaluation demonstrates that both models achieve strong performance on multiple benchmarks. The 0.5B variant offers the best overall discrimination capability with F1 scores of 0.791 (micro) and 0.785 (macro) on the test set, while the 0.1B variant demonstrates exceptional efficiency. Notably, Bielik Guard 0.1B v1.1 achieves superior precision (77.65\%) and very low false positive rate (0.63\%) on real user prompts, outperforming HerBERT-PL-Guard (31.55\% precision, 4.70\% FPR) despite identical model size. The models are publicly available and designed to provide appropriate responses rather than simple content blocking, particularly for sensitive categories like self-harm.
Paper Structure (27 sections, 1 figure, 5 tables)

This paper contains 27 sections, 1 figure, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM-based Safety Classifiers
  4. Polish Language Models and Safety
  5. Community-based Annotation
  6. Bielik Guard: Model Architecture and Training
  7. Safety Taxonomy
  8. Model Architecture
  9. Training Data and Methodology
  10. Data Collection
  11. Training Splits and Evaluation Strategy
  12. Data Distribution
  13. Quality Control
  14. Training Procedure
  15. Model Versions
  16. ...and 12 more sections

Figures (1)

  • Figure 1: Comparison of safety classifiers on Polish user prompts. Higher Precision is better, lower FPR is better. Bielik Guard 0.1B v1.1 (124M) outperforms all compared models including larger multilingual alternatives.