Post-TIPS Prediction via Multimodal Interaction: A Multi-Center Dataset and Framework for Survival, Complication, and Portal Pressure Assessment

TL;DR

This work tackles variable post-TIPS prognosis by introducing MultiTIPS, the first public multi-center dataset for TIPS prognosis and a multimodal framework that fuses preoperative CT radiomics, deep learning features from portal vein segmentation, and clinical data. The framework combines a dual-option portal vein segmentation module (semi-supervised DSDA-MT and foundation-model MedSAM2), MGRA-POD-CGPE multimodal interaction, and a staged training strategy for simultaneous survival, PPG, and OHE prediction. Key contributions include improved segmentation robustness under limited annotations, equitable cross-modal representation learning, and superior cross-domain generalization with rich interpretability, demonstrated on internal and external cohorts. The public dataset and code enable reproducibility and further research toward clinically impactful, multimodal prognostic tools in TIPS. Overall, the method achieves state-of-the-art performance across the three prognostic tasks and provides a practical, interpretable framework for clinical decision support in portal hypertension management.

Abstract

Transjugular intrahepatic portosystemic shunt (TIPS) is an established procedure for portal hypertension, but provides variable survival outcomes and frequent overt hepatic encephalopathy (OHE), indicating the necessity of accurate preoperative prognostic modeling. Current studies typically build machine learning models from preoperative CT images or clinical characteristics, but face three key challenges: (1) labor-intensive region-of-interest (ROI) annotation, (2) poor reliability and generalizability of unimodal methods, and (3) incomplete assessment from single-endpoint prediction. Moreover, the lack of publicly accessible datasets constrains research in this field. Therefore, we present MultiTIPS, the first public multi-center dataset for TIPS prognosis, and propose a novel multimodal prognostic framework based on it. The framework comprises three core modules: (1) dual-option segmentation, which integrates semi-supervised and foundation model-based pipelines to achieve robust ROI segmentation with limited annotations and facilitate subsequent feature extraction; (2) multimodal interaction, where three techniques, multi-grained radiomics attention (MGRA), progressive orthogonal disentanglement (POD), and clinically guided prognostic enhancement (CGPE), are introduced to enable cross-modal feature interaction and complementary representation integration, thus improving model accuracy and robustness; and (3) multi-task prediction, where a staged training strategy is used to perform stable optimization of survival, portal pressure gradient (PPG), and OHE prediction for comprehensive prognostic assessment. Extensive experiments on MultiTIPS demonstrate the superiority of the proposed method over state-of-the-art approaches, along with strong cross-domain generalization and interpretability, indicating its promise for clinical application. The dataset and code are available.

Figures (14)

• Figure 1: The flowcharts of (a) preoperative CT-based methods, (b) structured clinical parameter-driven methods, and (c) our multimodal framework for TIPS prognosis. SSS: Semi-Supervised Segmentation. DL Feat.: Deep Learning Features. RA Feat.: Radiomics Features. Cli. Feat.: Clinical Features. MGRA: Multi-Grained Radiomics Attention. POD: Progressive Orthogonal Disentanglement. CGPE: Clinically Guided Prognostic Enhancement. Gen. Feat.: Generalized Features.
• Figure 3: Illustrative examples from two patients in the MultiTIPS dataset. Columns (a)--(d) show annotations in axial, sagittal, coronal, and 3D views, respectively. Red, green, blue, and yellow represent the central portal vein, peripheral portal vein, inferior vena cava, and liver.
• Figure 4: Dual-option pipeline for portal vein segmentation, including (a) a semi-supervised approach (DSDA-MT) and (b) a foundation model-based approach (MedSAM2).
• Figure 5: Overview of the proposed multimodal prediction architecture, which consists of two components: multimodal interaction and multi-task prediction. The former includes (1) a multi-grained radiomics attention (MGRA) mechanism to capture comprehensive deep learning features guided by hierarchical radiomics features; (2) a progressive orthogonal disentanglement (POD) strategy to reduce their redundancy and enhance complementary patterns; and (3) a clinically guided prognostic enhancement (CGPE) module to obtain a generalized and discriminative unified representation across the three modalities. Then, this representation serves three TIPS prognostic tasks involving survival analysis, PPG assessment and OHE prediction. OT.: OT-based Co-attention. GAP: Global Attention Pooling.
• Figure 6: Example of the hierarchical structure of radiomics features. $\mathcal{F}$: Filter Class. $\mathcal{S}$: Feature Class. $\mathcal{A}$: Attribute. Level $\mathrm{I}$ represents 1,595 individual $1\times1$ features, each defined by a specific ($\mathcal{F}$, $\mathcal{S}$, $\mathcal{A}$) combination. Level $\mathrm{II}$ groups features by joint filter-feature categories ($\mathcal{F}\_\mathcal{S}, N_g = 103$). Level $\mathrm{III}$ further aggregates Level $\mathrm{II}$ into higher-level categories based on either $\mathcal{F}$ ($N_{filt} = 17$) or $\mathcal{S}$ ($N_{feat} = 7$).