Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TTVD: Towards a Geometric Framework for Test-Time Adaptation Based on Voronoi Diagram

Mingxi Lei, Chunwei Ma, Meng Ding, Yufan Zhou, Ziyun Huang, Jinhui Xu

TL;DR

This work reframes test-time adaptation as a geometric space-partitioning problem using Voronoi-based diagrams. It introduces Cluster-induced Voronoi Diagram (CIVD) to allow cluster-to-point influence and Power Diagram (PD) to enable flexible, weighted boundaries, forming the TTVD framework. TTVD demonstrates improved classification accuracy and calibration on robustness benchmarks such as CIFAR-10-C, CIFAR-100-C, ImageNet-C, and ImageNet-R, outperforming several state-of-the-art TTA methods and showing resilience to small batch sizes and label shifts. The approach offers interpretable boundaries and robust sample handling, with practical implications for deploying reliable models in real-world, distribution-shifted environments.

Abstract

Deep learning models often struggle with generalization when deploying on real-world data, due to the common distributional shift to the training data. Test-time adaptation (TTA) is an emerging scheme used at inference time to address this issue. In TTA, models are adapted online at the same time when making predictions to test data. Neighbor-based approaches have gained attention recently, where prototype embeddings provide location information to alleviate the feature shift between training and testing data. However, due to their inherit limitation of simplicity, they often struggle to learn useful patterns and encounter performance degradation. To confront this challenge, we study the TTA problem from a geometric point of view. We first reveal that the underlying structure of neighbor-based methods aligns with the Voronoi Diagram, a classical computational geometry model for space partitioning. Building on this observation, we propose the Test-Time adjustment by Voronoi Diagram guidance (TTVD), a novel framework that leverages the benefits of this geometric property. Specifically, we explore two key structures: 1) Cluster-induced Voronoi Diagram (CIVD): This integrates the joint contribution of self-supervision and entropy-based methods to provide richer information. 2) Power Diagram (PD): A generalized version of the Voronoi Diagram that refines partitions by assigning weights to each Voronoi cell. Our experiments under rigid, peer-reviewed settings on CIFAR-10-C, CIFAR-100-C, ImageNet-C, and ImageNet-R shows that TTVD achieves remarkable improvements compared to state-of-the-art methods. Moreover, extensive experimental results also explore the effects of batch size and class imbalance, which are two scenarios commonly encountered in real-world applications. These analyses further validate the robustness and adaptability of our proposed framework.

TTVD: Towards a Geometric Framework for Test-Time Adaptation Based on Voronoi Diagram

TL;DR

This work reframes test-time adaptation as a geometric space-partitioning problem using Voronoi-based diagrams. It introduces Cluster-induced Voronoi Diagram (CIVD) to allow cluster-to-point influence and Power Diagram (PD) to enable flexible, weighted boundaries, forming the TTVD framework. TTVD demonstrates improved classification accuracy and calibration on robustness benchmarks such as CIFAR-10-C, CIFAR-100-C, ImageNet-C, and ImageNet-R, outperforming several state-of-the-art TTA methods and showing resilience to small batch sizes and label shifts. The approach offers interpretable boundaries and robust sample handling, with practical implications for deploying reliable models in real-world, distribution-shifted environments.

Abstract

Deep learning models often struggle with generalization when deploying on real-world data, due to the common distributional shift to the training data. Test-time adaptation (TTA) is an emerging scheme used at inference time to address this issue. In TTA, models are adapted online at the same time when making predictions to test data. Neighbor-based approaches have gained attention recently, where prototype embeddings provide location information to alleviate the feature shift between training and testing data. However, due to their inherit limitation of simplicity, they often struggle to learn useful patterns and encounter performance degradation. To confront this challenge, we study the TTA problem from a geometric point of view. We first reveal that the underlying structure of neighbor-based methods aligns with the Voronoi Diagram, a classical computational geometry model for space partitioning. Building on this observation, we propose the Test-Time adjustment by Voronoi Diagram guidance (TTVD), a novel framework that leverages the benefits of this geometric property. Specifically, we explore two key structures: 1) Cluster-induced Voronoi Diagram (CIVD): This integrates the joint contribution of self-supervision and entropy-based methods to provide richer information. 2) Power Diagram (PD): A generalized version of the Voronoi Diagram that refines partitions by assigning weights to each Voronoi cell. Our experiments under rigid, peer-reviewed settings on CIFAR-10-C, CIFAR-100-C, ImageNet-C, and ImageNet-R shows that TTVD achieves remarkable improvements compared to state-of-the-art methods. Moreover, extensive experimental results also explore the effects of batch size and class imbalance, which are two scenarios commonly encountered in real-world applications. These analyses further validate the robustness and adaptability of our proposed framework.

Paper Structure

This paper contains 19 sections, 1 theorem, 6 equations, 7 figures, 12 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Voronoi Diagram: foundational geometric structure for Neighbor-based Test-time Adaptation
  5. Cluster-induced Voronoi Diagram: Multi-site Influences Mechanism Improves Robustness
  6. Power Diagram: Identifying Noisy Samples by Flexible Boundaries
  7. Experiments
  8. Experiment Setup
  9. Experiment Results
  10. Conclusion
  11. Appendix
  12. Sample analysis
  13. Additional Tables
  14. Demonstrative illustration of MNIST-C Dataset in $\mathbb{R}^2$
  15. Hyperparameter settings in the Experiment
  16. ...and 4 more sections

Key Result

Lemma 3.1

A logistic regression model parameterized by $W^{K\times\ell}$ and $b^K$ partitions the feature space $\mathbb{R}^\ell$ into a $K$-cell Power Diagram with $\mu_k =\frac{1}{2} W^{k\times\ell}$ and $v_k^2 =b^k+\frac{1}{4}\left\|W^{k\times\ell}\right\|_2^2$.

Figures (7)

  • Figure 1: (A) Visualization of space partitions induced by Voronoi Diagram, Power Diagram and Augmented Voronoi Diagram (by self-supervision) on MNIST-C mu2019mnist (digit "0" $\sim$ "2" only, gaussian-noise-corrupted) in $\mathbb{R}^2$. (B) Visualization of adaptation performance on MNIST-C using T3A NEURIPS2021_1415fe9f, Tent wang2021tent and VD and Augmented VD with joint influence. See \ref{['sec:appendix_mnist']} for details.
  • Figure 2: Noisy sample filtering by diagram subtraction. (a) Entropy landscape of MNIST. Loss value quickly shrinks once a sample leave the boundaries. (b) Multi-site provides more reliable samples. The solid and dash line are boundaries given by PD and VD, respectively. Reliable samples can be identified by subtracting Voronoi cells, marked in deeper colors.
  • Figure 3: (Left) Illustrations on differences between VD, CIVD and CIPD. (Right) Illustrations on Test-time adaptations by Voronoi Diagram(s) guidance.
  • Figure 4: Comparison on the Adaptation Curves on different noise perturbations in ImageNet-C. Error (%)$\downarrow$ is calculated over all retrospective test samples. The first four types of perturbations in the dataset are presented above.
  • Figure 5: A misclassified "bike" sample corrected by CIVD. The x-axis denotes the index of classes and y-axis denotes the distance to their corresponding Voronoi sites. The green lines indicate the ground-true label and the red lines indicate the predicted label.
  • ...and 2 more figures

Theorems & Definitions (5)

  • Definition 3.1: Voronoi Diagram
  • Definition 3.2: Cluster-induced Voronoi Diagram focsdoi:10.1137/15M1044874huang2021influence
  • Definition 3.3: Power Diagram doi:10.1137/0216006
  • Lemma 3.1: ma2022fewshotma2023progressive
  • Definition 3.4: Cluster-induced Power Diagram