Wireless Dataset Similarity: Measuring Distances in Supervised and Unsupervised Machine Learning

TL;DR

This work tackles predicting cross-dataset transferability for wireless ML by proposing a task- and model-aware framework to quantify dataset similarity. It leverages latent-space projections (notably UMAP) to define distances using Euclidean and Wasserstein metrics, augmented with a label-aware extension for supervised tasks. Across unsupervised CSI compression and supervised beam prediction, the proposed distances show strong correlations with transfer performance (e.g., Pearson > $0.85$ in some setups) and outperform traditional baselines, enabling data selection, augmentation, and deployment decisions. The approach is complemented by an open-source implementation and provides guidance on when to use which distance type and how label information can enhance discrimination.

Abstract

This paper introduces a task- and model-aware framework for measuring similarity between wireless datasets, enabling applications such as dataset selection/augmentation, simulation-to-real (sim2real) comparison, task-specific synthetic data generation, and informing decisions on model training/adaptation to new deployments. We evaluate candidate dataset distance metrics by how well they predict cross-dataset transferability: if two datasets have a small distance, a model trained on one should perform well on the other. We apply the framework on an unsupervised task, channel state information (CSI) compression, using autoencoders. Using metrics based on UMAP embeddings, combined with Wasserstein and Euclidean distances, we achieve Pearson correlations exceeding 0.85 between dataset distances and train-on-one/test-on-another task performance. We also apply the framework to a supervised beam prediction in the downlink using convolutional neural networks. For this task, we derive a label-aware distance by integrating supervised UMAP and penalties for dataset imbalance. Across both tasks, the resulting distances outperform traditional baselines and consistently exhibit stronger correlations with model transferability, supporting task-relevant comparisons between wireless datasets.

Figures (8)

• Figure 1: Example applications enabled by dataset distance computation.
• Figure 2: Classes of distances that can be applied between datasets.
• Figure 3: Framework for evaluating a dataset distance metric for a given task and model: if two datasets are close according to the metric, then a model trained on one should achieve high performance when tested on the other. The higher the correlation between distances computed with a given metric and model performance on a given task, the more suitable the metric is for measuring dataset similarity in that task.
• Figure 4: Architecture of the model used in the unsupervised CSI compression task. The model is heavily inspired in the CSINet+ 8972904.
• Figure 5: Real (left) and rendered (right) top view the ASU campus DeepMIMO dataset. The base station is showed in both figures. It should be noted that buildings and other scenario assets are 3D, and their heights matter significantly for roof diffractions. The mesh represented in the synthetic counterpart represents the received power when applying a standard DFT codebook at the base station.