Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI for Regulatory Affairs: Balancing Accuracy, Interpretability, and Computational Cost in Medical Device Classification

Yu Han, Aaron Ceross, Jeroen H. M. Bergmann

TL;DR

The paper tackles the challenge of deploying AI for medical device regulatory classification by benchmarking rule-based, traditional ML, deep learning, and LLM approaches on Chinese device descriptions from the NMPA. It assesses models along three axes—classification accuracy, interpretability, and computational cost—and reports that CNNs and XGBoost offer the strongest balance, while LLMs provide interpretability at the expense of accuracy and speed. The study also analyzes interpretability mechanisms, highlights per-class performance differences, and argues for hybrid workflows that combine fast, transparent models with high-accuracy ones, augmented by expert review and scenario-based testing. The result is a practical, regulator-focused framework for selecting and validating AI tools in regulatory affairs, with proposed guidelines for explanation utility, risk-based testing, and governance to enable trustworthy adoption in real-world regulatory pipelines.

Abstract

Regulatory affairs, which sits at the intersection of medicine and law, can benefit significantly from AI-enabled automation. Classification task is the initial step in which manufacturers position their products to regulatory authorities, and it plays a critical role in determining market access, regulatory scrutiny, and ultimately, patient safety. In this study, we investigate a broad range of AI models -- including traditional machine learning (ML) algorithms, deep learning architectures, and large language models -- using a regulatory dataset of medical device descriptions. We evaluate each model along three key dimensions: accuracy, interpretability, and computational cost.

AI for Regulatory Affairs: Balancing Accuracy, Interpretability, and Computational Cost in Medical Device Classification

TL;DR

The paper tackles the challenge of deploying AI for medical device regulatory classification by benchmarking rule-based, traditional ML, deep learning, and LLM approaches on Chinese device descriptions from the NMPA. It assesses models along three axes—classification accuracy, interpretability, and computational cost—and reports that CNNs and XGBoost offer the strongest balance, while LLMs provide interpretability at the expense of accuracy and speed. The study also analyzes interpretability mechanisms, highlights per-class performance differences, and argues for hybrid workflows that combine fast, transparent models with high-accuracy ones, augmented by expert review and scenario-based testing. The result is a practical, regulator-focused framework for selecting and validating AI tools in regulatory affairs, with proposed guidelines for explanation utility, risk-based testing, and governance to enable trustworthy adoption in real-world regulatory pipelines.

Abstract

Regulatory affairs, which sits at the intersection of medicine and law, can benefit significantly from AI-enabled automation. Classification task is the initial step in which manufacturers position their products to regulatory authorities, and it plays a critical role in determining market access, regulatory scrutiny, and ultimately, patient safety. In this study, we investigate a broad range of AI models -- including traditional machine learning (ML) algorithms, deep learning architectures, and large language models -- using a regulatory dataset of medical device descriptions. We evaluate each model along three key dimensions: accuracy, interpretability, and computational cost.

Paper Structure

This paper contains 39 sections, 7 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Rule-Based Systems
  4. Traditional Machine Learning
  5. Deep Learning Models
  6. Transformer-Based Models and LLMs
  7. Model Interpretability in Regulatory Contexts and Computational Cost
  8. Trade-off
  9. Methodology
  10. Dataset and Preprocessing
  11. Text Representation
  12. Train–Validation–Test Split
  13. Classifiers and Training Setup
  14. Traditional Machine Learning Models.
  15. Deep Learning Models.
  16. ...and 24 more sections

Figures (7)

  • Figure 1: Text process, Model Training and Evaluation Pipeline
  • Figure 2: Comparative interpretability visualizations across models: (a) SVM with LIME, (b) Logistic Regression with Anchor explanations, and (c) RNN with Integrated Gradients.
  • Figure 3: TreeSHAP-based explanations for BERT embeddings using ensemble models.
  • Figure 4: LLM interpretability
  • Figure 5: Comparison of Models Based on Accuracy, Interpretability, and Cost
  • ...and 2 more figures