Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PicoSAM3: Real-Time In-Sensor Region-of-Interest Segmentation

Pietro Bonazzi, Nicola Farronato, Stefan Zihlmann, Haotong Qin, Michele Magno

Abstract

Real-time, on-device segmentation is critical for latency-sensitive and privacy-aware applications such as smart glasses and Internet-of-Things devices. We introduce PicoSAM3, a lightweight promptable visual segmentation model optimized for edge and in-sensor execution, including deployment on the Sony IMX500 vision sensor. PicoSAM3 has 1.3 M parameters and combines a dense CNN architecture with region of interest prompt encoding, Efficient Channel Attention, and knowledge distillation from SAM2 and SAM3. On COCO and LVIS, PicoSAM3 achieves 65.45% and 64.01% mIoU, respectively, outperforming existing SAM-based and edge-oriented baselines at similar or lower complexity. The INT8 quantized model preserves accuracy with negligible degradation while enabling real-time in-sensor inference at 11.82 ms latency on the IMX500, fully complying with its memory and operator constraints. Ablation studies show that distillation from large SAM models yields up to +14.5% mIoU improvement over supervised training and demonstrate that high-quality, spatially flexible promptable segmentation is feasible directly at the sensor level.

PicoSAM3: Real-Time In-Sensor Region-of-Interest Segmentation

Abstract

Real-time, on-device segmentation is critical for latency-sensitive and privacy-aware applications such as smart glasses and Internet-of-Things devices. We introduce PicoSAM3, a lightweight promptable visual segmentation model optimized for edge and in-sensor execution, including deployment on the Sony IMX500 vision sensor. PicoSAM3 has 1.3 M parameters and combines a dense CNN architecture with region of interest prompt encoding, Efficient Channel Attention, and knowledge distillation from SAM2 and SAM3. On COCO and LVIS, PicoSAM3 achieves 65.45% and 64.01% mIoU, respectively, outperforming existing SAM-based and edge-oriented baselines at similar or lower complexity. The INT8 quantized model preserves accuracy with negligible degradation while enabling real-time in-sensor inference at 11.82 ms latency on the IMX500, fully complying with its memory and operator constraints. Ablation studies show that distillation from large SAM models yields up to +14.5% mIoU improvement over supervised training and demonstrate that high-quality, spatially flexible promptable segmentation is feasible directly at the sensor level.
Paper Structure (14 sections, 9 equations, 5 figures, 2 tables)

This paper contains 14 sections, 9 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Hardware Setup
  4. PicoSAM3
  5. Implicit Prompt Encoding via Centered Cropping
  6. Model Architecture
  7. SAM3 Distillation
  8. Training and Inference Pipeline
  9. Experimental Results
  10. Model Performance
  11. Ablations Studies
  12. Qualitative Results
  13. Conclusion & Discussion
  14. Reproducibility

Figures (5)

  • Figure 1: Hardware setup: PicoSAM3 runs directly in the Sony IMX500 camera, the ARM Cortex A76 is only used to deploy the model.
  • Figure 2: Overview of the PicoSAM3 architecture and knowledge distillation pipeline.
  • Figure 3: Segmentation precision (mAP and mIoU) vs. model size (log scale) per model family.
  • Figure 4: Qualitative comparison of each model's mask inference.
  • Figure :