Transfer Learning Under High-Dimensional Network Convolutional Regression Model
Liyuan Wang, Jiachen Chen, Kathryn L. Lunetta, Danyang Huang, Huimin Cheng, Debarghya Mukherjee
TL;DR
The work addresses transfer learning for high-dimensional, networked data by introducing a network-convolutional regression (NCR) framework that aggregates neighbor information via the normalized adjacency $A^*=A/\sqrt{(n-1)p}$ and regresses the outcome on both self-features $X$ and neighborhood-aggregated features. It develops two transfer-learning schemes: Oracle Trans-NCR (with known transferable sources) and Trans-NCR (with data-driven source selection via source-discrepancy measures and Q-aggregation), along with a source-detection mechanism to mitigate negative transfer. The authors establish theoretical guarantees under Erdős–Rényi graphs, showing that transfer improves convergence when informative sources exist, and validate the approach through extensive simulations and a Sina Weibo real-data application, demonstrating improved prediction with limited target labels. This framework provides a principled method for transferring knowledge in structured, dependent, high-dimensional settings and points to future work on SBM/graphon extensions, higher-order convolutions, and multiplex networks, broadening applicability to complex networked data.
Abstract
Transfer learning enhances model performance by utilizing knowledge from related domains, particularly when labeled data is scarce. While existing research addresses transfer learning under various distribution shifts in independent settings, handling dependencies in networked data remains challenging. To address this challenge, we propose a high-dimensional transfer learning framework based on network convolutional regression (NCR), inspired by the success of graph convolutional networks (GCNs). The NCR model incorporates random network structure by allowing each node's response to depend on its features and the aggregated features of its neighbors, capturing local dependencies effectively. Our methodology includes a two-step transfer learning algorithm that addresses domain shift between source and target networks, along with a source detection mechanism to identify informative domains. Theoretically, we analyze the lasso estimator in the context of a random graph based on the Erdos-Renyi model assumption, demonstrating that transfer learning improves convergence rates when informative sources are present. Empirical evaluations, including simulations and a real-world application using Sina Weibo data, demonstrate substantial improvements in prediction accuracy, particularly when labeled data in the target domain is limited.