A Point-Neighborhood Learning Framework for Nasal Endoscope Image Segmentation

TL;DR

This work presents Point-Neighborhood Learning (PNL) to address nasal endoscope lesion segmentation under weak supervision by transforming sparse point annotations into informative point-neighborhoods. It introduces Point-Neighborhood Supervision (PNS), Pseudo-label Scoring Mechanism (PSM), and dual data-augmentation strategies (PNMxp and PVRMxp) within a Mean Teacher framework to generate reliable pseudo-labels and robust representations. PNL achieves state-of-the-art results on the NPC-LES dataset and demonstrates strong generalization to small-object colonoscopic datasets, reducing annotation burden while maintaining high accuracy. The approach offers practical impact by enabling high-quality segmentation with minimal pixel-level labeling, suitable for deployment in real-world endoscopic workflows.

Abstract

Lesion segmentation on nasal endoscopic images is challenging due to its complex lesion features. Fully-supervised deep learning methods achieve promising performance with pixel-level annotations but impose a significant annotation burden on experts. Although weakly supervised or semi-supervised methods can reduce the labelling burden, their performance is still limited. Some weakly semi-supervised methods employ a novel annotation strategy that labels weak single-point annotations for the entire training set while providing pixel-level annotations for a small subset of the data. However, the relevant weakly semi-supervised methods only mine the limited information of the point itself, while ignoring its label property and surrounding reliable information. This paper proposes a simple yet efficient weakly semi-supervised method called the Point-Neighborhood Learning (PNL) framework. PNL incorporates the surrounding area of the point, referred to as the point-neighborhood, into the learning process. In PNL, we propose a point-neighborhood supervision loss and a pseudo-label scoring mechanism to explicitly guide the model's training. Meanwhile, we proposed a more reliable data augmentation scheme. The proposed method significantly improves performance without increasing the parameters of the segmentation neural network. Extensive experiments on the NPC-LES dataset demonstrate that PNL outperforms existing methods by a significant margin. Additional validation on colonoscopic polyp segmentation datasets confirms the generalizability of the proposed PNL.

Figures (12)

• Figure 1: Comparison between the existing image segmentation methods (a) and our proposed method (b) for point annotation. Methods (a) use point coordinates as input, while our method (b) mines the points' neighborhoods for data augmentation, supervision, and low-quality pseudo-label suppression. (c) shows a visual comparison of the test results for different methods.
• Figure 2: Framework of Our Method. Leveraging point-neighborhood transformations, we propose strategies for two critical training stages: data augmentation and supervision. In the data augmentation, Point-Neighborhood Mixup (PNMxp) and Particular-Value Randomly Mixup (PVRMxp) enhance data diversity, then improving the model generalization. In the supervision, Point-Neighborhood Supervision (PNS) provides local supervision for point-neighborhood regions. Additionally, the Pseudo-Label Scoring Mechanism (PSM) scores and suppresses low-confidence pseudo-labels.
• Figure 3: The illustration of PNS loss. PNS loss measures the prediction accuracy inside $\mathcal{N}$ but ignores those pixels outside of $\mathcal{N}$.
• Figure 4: The illustration of Point-Neighborhood Mixup (PNMxp).
• Figure 5: The illustration of Particular-Value Randomly Mixup (PVRMxp).