Evaluating Few-Shot Pill Recognition Under Visual Domain Shift

TL;DR

Investigation of few-shot pill recognition from a deployment-oriented perspective indicates that semantic pill recognition adapts rapidly with few-shot supervision, with classification performance reaching saturation even with a single labeled example, however, stress testing under overlapping and occluded conditions demonstrates a marked decline in localization and recall.

Abstract

Adverse drug events are a significant source of preventable harm, which has led to the development of automated pill recognition systems to enhance medication safety. Real-world deployment of these systems is hindered by visually complex conditions, including cluttered scenes, overlapping pills, reflections, and diverse acquisition environments. This study investigates few-shot pill recognition from a deployment-oriented perspective, prioritizing generalization under realistic cross-dataset domain shifts over architectural innovation. A two-stage object detection framework is employed, involving base training followed by few-shot fine-tuning. Models are adapted to novel pill classes using one, five, or ten labeled examples per class and are evaluated on a separate deployment dataset featuring multi-object, cluttered scenes. The evaluation focuses on classification-centric and error-based metrics to address heterogeneous annotation strategies. Findings indicate that semantic pill recognition adapts rapidly with few-shot supervision, with classification performance reaching saturation even with a single labeled example. However, stress testing under overlapping and occluded conditions demonstrates a marked decline in localization and recall, despite robust semantic classification. Models trained on visually realistic, multi-pill data consistently exhibit greater robustness in low-shot scenarios, underscoring the importance of training data realism and the diagnostic utility of few-shot fine-tuning for deployment readiness.

Figures (4)

• Figure 1: Example images from the MEDISEG and CURE datasets. The MEDISEG image (left) contains multiple pill instances within a container, while the CURE image (right) depicts a single pill captured under controlled conditions.
• Figure 2: Representative examples from the novel deployment dataset used for few-shot adaptation and evaluation. Images depict multiple pill instances with varying degrees of overlap, occlusion, and interference from pill containers and surrounding objects, reflecting realistic medication handling conditions.
• Figure 3: Evolution of classification-centric and loss-based metrics during few-shot fine-tuning for CURE- and MEDISEG-trained models under 1-shot, 5-shot, and 10-shot configurations. Shown are foreground classification accuracy, foreground false negative rate, classification loss, and total detection loss over 2,000 fine-tuning iterations.
• Figure 4: Qualitative comparison of few-shot detection results for models base-trained on MEDISEG and CURE under 1-shot, 5-shot, and 10-shot fine-tuning. Each row shows a different test image, and each column corresponds to a specific base-training dataset and supervision level. Predicted bounding boxes illustrate detection and classification behavior under clutter, overlap, and occlusion.