From Pre-labeling to Production: Engineering Lessons from a Machine Learning Pipeline in the Public Sector

TL;DR

This study investigates the engineering, governance, and organizational challenges of deploying ML components in a large public-sector civic platform (Brasil Participativo). Using a retrospective mixed-methods design, it analyzes how architectural choices such as LLM-based pre-labeling, class balancing, and modular routing affect reliability, traceability, and cost. Key findings show that LLM pre-labeling accelerates development but requires ongoing human validation for trust, synthetic data offers limited long-tail gains, and dual-classifier routing can improve training metrics at the cost of operational complexity. The work argues that successful public-sector ML depends more on transparent, reproducible data infrastructures and institutional governance than on modeling breakthroughs, calling for governance-aware MLOps and robust data provenance as civic infrastructure.

Abstract

Machine learning is increasingly being embedded into government digital platforms, but public-sector constraints make it difficult to build ML systems that are accurate, auditable, and operationally sustainable. In practice, teams face not only technical issues like extreme class imbalance and data drift, but also organizational barriers such as bureaucratic data access, lack of versioned datasets, and incomplete governance over provenance and monitoring. Our study of the Brasil Participativo (BP) platform shows that common engineering choices -- like using LLMs for pre-labeling, splitting models into routed classifiers, and generating synthetic data -- can speed development but also introduce new traceability, reliability, and cost risks if not paired with disciplined data governance and human validation. This means that, in the public sector, responsible ML is not just a modeling problem but an institutional engineering problem, and ML pipelines must be treated as civic infrastructure. Ultimately, this study shows that the success of machine learning in the public sector will depend less on breakthroughs in model accuracy and more on the ability of institutions to engineer transparent, reproducible, and accountable data infrastructures that citizens can trust.

Figures (2)

• Figure 1: ML system development workflow amershi_software_2019giray2021softwareengineeringperspectiveengineeringpaleyes2022challenges
• Figure 2: Class distribution with mean and $\pm \mathrm{CV}\cdot u$ reference lines.