An Effective Flow-based Method for Positive-Unlabeled Learning: 2-HNC

TL;DR

This work tackles Positive-Unlabeled learning by introducing $2$-HNC, a two-stage method that exploits pairwise sample similarities and Hochbaum's Normalized Cut via parametric minimum cuts on a similarity graph. Stage 1 employs $HNC-$ with only the positive seeds to generate a nested partition sequence, enabling a ranking of unlabeled samples by their likelihood of being negative. Stage 2 selects a set of likely-negative unlabeled samples from stage 1 and applies $HNC+$ with $L^+$ and $L^N$ to produce a second partition sequence; the final prediction chooses the partition whose positive fraction closely matches the prior $\pi$. Empirically, $2$-HNC is competitive with or superior to state-of-the-art PU methods on synthetic and real datasets, and the ranking mechanism suggests broader applicability to active learning and outlier detection, underpinned by a new Double Intra-similarity Theorem showing the polynomial solvability of this clustering formulation.

Abstract

In many scenarios of binary classification, only positive instances are provided in the training data, leaving the rest of the data unlabeled. This setup, known as positive-unlabeled (PU) learning, is addressed here with a network flow-based method which utilizes pairwise similarities between samples. The method we propose here, 2-HNC, leverages Hochbaum's Normalized Cut (HNC) and the set of solutions it provides by solving a parametric minimum cut problem. The set of solutions, that are nested partitions of the samples into two sets, correspond to varying tradeoff values between the two goals: high intra-similarity inside the sets and low inter-similarity between the two sets. This nested sequence is utilized here to deliver a ranking of unlabeled samples by their likelihood of being negative. Building on this insight, our method, 2-HNC, proceeds in two stages. The first stage generates this ranking without assuming any negative labels, using a problem formulation that is constrained only on positive labeled samples. The second stage augments the positive set with likely-negative samples and recomputes the classification. The final label prediction selects among all generated partitions in both stages, the one that delivers a positive class proportion, closest to a prior estimate of this quantity, which is assumed to be given. Extensive experiments across synthetic and real datasets show that 2-HNC yields strong performance and often surpasses existing state-of-the-art algorithms.

Figures (5)

• Figure 1: Nested cut partitions of a parametric flow graph in which the capacities of source and sink adjacent arcs are non-increasing and non-decreasing functions of the parameter $\lambda$. Sink sets (in yellow) of smaller $\lambda$ are nested in those of larger $\lambda$.
• Figure 2: Associated graphs with HNC+ and HNC- formulations, when labeled samples from both classes are given. Nodes in the middle, outside the blue and yellow shaded areas, correspond to unlabeled samples in $U = V \backslash (L^+ \cup L^-)$.
• Figure 3: Graphs on which we solve HNC+ and HNC- as minimum cut problems, in PU learning where negative labeled samples are not provided.
• Figure 4: Histograms of accuracy improvement yielded by 2-HNC over (a) uPU, (b) nnPU, and (c) PUET, on 2160 synthetic datasets.
• Figure 5: Histograms of balanced accuracy improvement yielded by 2-HNC over: (a) uPU, (b) nnPU, and (c) PUET, on 2160 synthetic datasets.

Theorems & Definitions (5)

• Lemma 3.1: Nested Cut Property
• Theorem 3.2: Double Intra-similarity Theorem
• proof
• Lemma 4.1
• proof