Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Laplacian-LoRA: Delaying Oversmoothing in Deep GCNs via Spectral Low-Rank Adaptation

Sai Vamsi Alisetti

TL;DR

The paper identifies oversmoothing in deep GCNs as an operator-level contraction in the Laplacian spectrum and proposes Laplacian-LoRA, a low-rank spectral correction that weakens per-layer contraction while preserving stability. By introducing a depth-annealed, spectrally anchored residual in the Laplacian eigenspace, the method yields an effective filter $g_{eff}(\lambda;\alpha_\ell)=(1-\lambda)(2-\alpha_\ell \theta(\lambda))$, delaying representational collapse without eliminating it. Across five benchmarks, Laplacian-LoRA extends the effective network depth and improves accuracy at intermediate depths, with spectral diagnostics showing a smooth, bounded correction that preserves high-frequency energy better than standard GCNs. These findings treat oversmoothing as a depth-dependent spectral phenomenon and demonstrate that principled spectral design can meaningfully modulate it. The approach offers a principled, interpretable avenue to enhance deep GCNs, albeit with reliance on spectral decompositions that may impact scalability.

Abstract

Oversmoothing is a fundamental limitation of deep graph convolutional networks (GCNs), causing node representations to collapse as depth increases. While many prior approaches mitigate this effect through architectural modifications or residual mechanisms, the underlying spectral cause of oversmoothing is often left implicit. We propose Laplacian-LoRA, a simple and interpretable low-rank spectral adaptation of standard GCNs. Rather than redesigning message passing, Laplacian-LoRA introduces a learnable, spectrally anchored correction to the fixed Laplacian propagation operator, selectively weakening contraction while preserving stability and the low-pass inductive bias. Across multiple benchmark datasets and depths, Laplacian-LoRA consistently delays the onset of oversmoothing, extending the effective depth of GCNs by up to a factor of two. Embedding variance diagnostics confirm that these gains arise from delayed representational collapse, while learned spectral analysis demonstrates that the correction is smooth, bounded, and well behaved. Our results show that oversmoothing is a depth-dependent spectral phenomenon that can be systematically delayed through modest, low-rank adaptation of the graph propagation operator.

Laplacian-LoRA: Delaying Oversmoothing in Deep GCNs via Spectral Low-Rank Adaptation

TL;DR

The paper identifies oversmoothing in deep GCNs as an operator-level contraction in the Laplacian spectrum and proposes Laplacian-LoRA, a low-rank spectral correction that weakens per-layer contraction while preserving stability. By introducing a depth-annealed, spectrally anchored residual in the Laplacian eigenspace, the method yields an effective filter , delaying representational collapse without eliminating it. Across five benchmarks, Laplacian-LoRA extends the effective network depth and improves accuracy at intermediate depths, with spectral diagnostics showing a smooth, bounded correction that preserves high-frequency energy better than standard GCNs. These findings treat oversmoothing as a depth-dependent spectral phenomenon and demonstrate that principled spectral design can meaningfully modulate it. The approach offers a principled, interpretable avenue to enhance deep GCNs, albeit with reliance on spectral decompositions that may impact scalability.

Abstract

Oversmoothing is a fundamental limitation of deep graph convolutional networks (GCNs), causing node representations to collapse as depth increases. While many prior approaches mitigate this effect through architectural modifications or residual mechanisms, the underlying spectral cause of oversmoothing is often left implicit. We propose Laplacian-LoRA, a simple and interpretable low-rank spectral adaptation of standard GCNs. Rather than redesigning message passing, Laplacian-LoRA introduces a learnable, spectrally anchored correction to the fixed Laplacian propagation operator, selectively weakening contraction while preserving stability and the low-pass inductive bias. Across multiple benchmark datasets and depths, Laplacian-LoRA consistently delays the onset of oversmoothing, extending the effective depth of GCNs by up to a factor of two. Embedding variance diagnostics confirm that these gains arise from delayed representational collapse, while learned spectral analysis demonstrates that the correction is smooth, bounded, and well behaved. Our results show that oversmoothing is a depth-dependent spectral phenomenon that can be systematically delayed through modest, low-rank adaptation of the graph propagation operator.
Paper Structure (14 sections, 14 equations, 3 figures)

This paper contains 14 sections, 14 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Methodology
  3. Spectral View of Oversmoothing
  4. Weakening Contraction Without Instability
  5. Laplacian-LoRA Operator
  6. Implementation Details
  7. Results
  8. Accuracy as a Function of Depth
  9. Oversmoothing Diagnostics
  10. Learned Spectral Response
  11. Propagation Spectrum.
  12. Depth-wise Spectral Contraction.
  13. Frequency-wise Energy Retention.
  14. Conclusion

Figures (3)

  • Figure 1: Test accuracy versus depth $L$ for standard GCNs and Laplacian-LoRA across five benchmark datasets. Laplacian-LoRA consistently mitigates over-smoothing and depth degradation.
  • Figure 2: Spectral analysis of Laplacian-LoRA.Top row (Cora):Left: propagation eigenvalues $\mu(\lambda)$ as a function of Laplacian eigenvalue $\lambda$. Center: depth-wise spectral contraction ratio $\left(|\mu_2|/|\mu_1|\right)^L$. Right: retained spectral energy after $L{=}16$ layers. Bottom row (CoauthorCS): corresponding spectral diagnostics on a larger graph.
  • Figure 3: Embedding variance as a function of depth $L$ for standard GCNs and Laplacian-LoRA.