Bipartite Mode Matching for Vision Training Set Search from a Hierarchical Data Server

TL;DR

This work tackles domain shift when on-the-fly data annotation is impractical by constructing target-tailored training sets from a large data server. It introduces a hierarchical data server and a bipartite mode matching (BMM) algorithm to align target domain modes with server modes, selecting a one-to-one subset of server modes that minimizes target risk. The approach leverages balanced hierarchical clustering and Fréchet Inception Distance–based matching solved by the Hungarian algorithm, and demonstrates improved domain alignment and higher accuracy on object re-ID and detection tasks, including benefits when combined with unsupervised domain-adaptation methods. The results suggest a strong, data-centric complement to existing model-centric UDA techniques, with robust performance across diverse data servers and target domains.

Abstract

We explore a situation in which the target domain is accessible, but real-time data annotation is not feasible. Instead, we would like to construct an alternative training set from a large-scale data server so that a competitive model can be obtained. For this problem, because the target domain usually exhibits distinct modes (i.e., semantic clusters representing data distribution), if the training set does not contain these target modes, the model performance would be compromised. While prior existing works improve algorithms iteratively, our research explores the often-overlooked potential of optimizing the structure of the data server. Inspired by the hierarchical nature of web search engines, we introduce a hierarchical data server, together with a bipartite mode matching algorithm (BMM) to align source and target modes. For each target mode, we look in the server data tree for the best mode match, which might be large or small in size. Through bipartite matching, we aim for all target modes to be optimally matched with source modes in a one-on-one fashion. Compared with existing training set search algorithms, we show that the matched server modes constitute training sets that have consistently smaller domain gaps with the target domain across object re-identification (re-ID) and detection tasks. Consequently, models trained on our searched training sets have higher accuracy than those trained otherwise. BMM allows data-centric unsupervised domain adaptation (UDA) orthogonal to existing model-centric UDA methods. By combining the BMM with existing UDA methods like pseudo-labeling, further improvement is observed.

Figures (7)

• Figure 1: Motivation illustration. Our research explores the often-overlooked potential of optimizing the structure of the data server. To explain, when aligning target modes with server modes, it is often challenging (a) due to granularity mismatches. In this paper, we propose a hierarchical server design (b) that allows target modes to match source modes at varying granularities, resulting in more effective and precise alignment (c).
• Figure 2: Workflow of BMM. (Top left): For a given target, we extract modes using flat clustering lloyd1982leastmacqueen1967some. (Bottom left): For our data server, we extract modes using hierarchical clustering mullner2011modern. For modes existing in both target and the data server, we align them using the bipartite graph matching algorithm, i.e., Hungarian algorithm kuhn1955hungarianmunkres1957algorithms. For source modes aligning to the target, we select them to form our searched training set. The searched training set can be further pruned and then used for model training.
• Figure 3: Mode matching examples (Left) and composition of the searched training set (Right). (Left): We show matched data server modes (images) resembling target modes (images). Four sub-figures (A) and (B) show mode matching examples for person re-ID, and vehicle detection respectively. For example, (B) shows a successful mode matching a dark environment. It is such successful mode matching that ensures the searched training set has a similar distribution to the target, and thus high training efficacy. (Right:) The pie charts on the right illustrate the proportions of the searched training set.
• Figure 4: The comparison of source server flat clustering and source hierarchical clustering. We show the impact of the number of source and target clusters on model performance. The target cluster number is fixed to 20 when varying source cluster numbers, and the source cluster number is fixed to 128 when varying target cluster numbers. Source flat clustering requires hyperparameter tuning for the "sweet point" (shown in top left) and top right, while hierarchical clustering does not (shown in bottom left). Furthermore, bottom right shows $L>20$ and bottomleft shows $>200$ source clusters (i.e., depth > 8) suffice for high accuracy.
• Figure 5: Comparison to the balanced K-means. Settings are the same as Tab. \ref{['tab:snp_to_othermethods_reid']} with 5% pruning rate targeting AlicePerson.