Federated Low-Rank Tensor Estimation for Multimodal Image Reconstruction

TL;DR

This work tackles multimodal image reconstruction in a federated setting by formulating constrained low-rank tensor estimation with Tucker decomposition. It introduces joint factorization and randomized sketching to enable server aggregation without reconstructing full tensors, while supporting heterogeneous Tucker ranks $(r_1,\dots,r_d)$. The proposed CompJF and CompRandJF methods demonstrate superior reconstruction quality under noise and undersampling and achieve better communication compression than baselines such as FIRM and FullDecomp, including comparisons to Top-$k$ and CSR encodings. The framework holds promise for scalable, privacy-preserving multimodal inverse problems with reduced communication overhead and robust performance across diverse data ranks.

Abstract

Low-rank tensor estimation offers a powerful approach to addressing high-dimensional data challenges and can substantially improve solutions to ill-posed inverse problems, such as image reconstruction under noisy or undersampled conditions. Meanwhile, tensor decomposition has gained prominence in federated learning (FL) due to its effectiveness in exploiting latent space structure and its capacity to enhance communication efficiency. In this paper, we present a federated image reconstruction method that applies Tucker decomposition, incorporating joint factorization and randomized sketching to manage large-scale, multimodal data. Our approach avoids reconstructing full-size tensors and supports heterogeneous ranks, allowing clients to select personalized decomposition ranks based on prior knowledge or communication capacity. Numerical results demonstrate that our method achieves superior reconstruction quality and communication compression compared to existing approaches, thereby highlighting its potential for multimodal inverse problems in the FL setting.

Figures (6)

• Figure 1: Evaluating the performance of various methods given the strongest noise level ($\sigma=0.1$). For CompJF, CompRandJF and FullDecomp, we present the results for Tucker rank $r=100$, the largest value that guarantee communication compression. The results for other ranks $r \geq 40$ is similar.
• Figure 2: Ground-truth and reconstructed images from various methods for client $N$ with XRT imaging modality. The images are reconstructed from noisy data ($\sigma=0.1$). For FullDecomp, CompJF, and CompRandJF, we use Tucker rank $r=100$. The reconstructed images correspond to the highest SSIM values.
• Figure 3: Performance of CompJF and CompRandJF for varying Tucker decomposition ranks and varying level of noise. (a) Includes the highest SSIM value, averaged across clients, obtained by the approaches. (b) Includes the average SSIM obtained when the discrepancy principle is satisfied. With early stopping condition, all three approaches terminate at the same epoch.
• Figure 4: Performance of our methods, CompJF and CompRandJF, Top-k and CSR encoding in balancing reconstruction quality with communication compression. a) GCE is computed using the SSIM values when the early stopping is satisfied and is averaged across clients. b) For CompJF and CompRandJF, the Tucker rank is $r=40$ and for Top-$k$, we use $k=30$. With this choice of hyperparameters, our methods and Top-$k$ have close percentage of compression in each communication round ($30\%$ and $34\%$).
• Figure 5: Performance of FullDecomp, CompJF and CompRandJF in heterogeneous rank setting for noisy data ($\sigma=0.1$). We conduct 10 independent simulations and plot the mean with one standard deviation.

Theorems & Definitions (6)

• Lemma 2.1
• Lemma 3.1
• proof
• Remark
• Proposition 1
• proof