Journal of Shandong University (Health Sciences)

Theoretical and methodological framework for multimodal big data cohort design based on AI language representation

XUE Fuzhong

Journal of Shandong University (Health Sciences). 2025, 63(8): 1-16. doi:10.6040/j.issn.1671-7554.0.2025.0568

Abstract ( 218 )

PDF (17983KB) ( 72 )

Save

References | Related Articles | Metrics

This paper proposes a theoretical and methodological framework for multimodal cohort design based on artificial intelligence(AI)language representation, breaking through the conventional paradigm of traditional epidemiological cohort studies and establishing a novel model for language-based multimodal integration. The framework integrates heterogeneous medical data—such as health records, electronic medical records, medical imaging, and genomic information—into a unified low-dimensional embedding space using Transformer-based models. Centered on a three-layer architecture of “Digital Omics-Digital Biomarkers-Digital Phenotypes”, it introduces key methods including embedding vector generation, causal inference, and multimodal data fusion. The study innovatively defines the PICLS criteria for digital biomarkers: predictability, interpretability, computability, latent-variable structure, and stability. On this basis, digital phenotypes are further required to meet the endpoints criterion, forming the PICLSE criteria to ensure their clinical utility in disease prediction and intervention. Technically, the paper details the entire process of embedding generation, data encoding/decoding, database construction, and biomarker extraction. A case study on scarlet fever surveillance demonstrates the practical application of the proposed multimodal embedded cohort in clinical screening and intelligent early warning. This framework offers a novel paradigm for epidemiological cohort research and provides methodological support for advancing precision medicine and smart public health.

Multimodal medical data fusion technology and its application

YANG Fan

Journal of Shandong University (Health Sciences). 2025, 63(8): 17-40. doi:10.6040/j.issn.1671-7554.0.2025.0512

Abstract ( 641 )

PDF (17179KB) ( 110 )

Save

References | Related Articles | Metrics

With the explosive growth of multi-source medical data such as bio-multi-omics, medical imaging, and electronic health records, a single modality is unable to characterize the biological heterogeneity of complex diseases. Multimodal medical data fusion technology provides new possibilities for disease prediction and treatment by integrating heterogeneous information at the feature level, representation level, and decision level. This study systematically reviews the progress of fusion methodologies based on deep learning and statistical modeling in recent years, including end-to-end frameworks driven by Transformer and graph neural networks, explicit probabilistic inference supported by Bayesian and latent factor models, and new theoretical perspectives such as information bottlenecks and commonality-specificity decomposition to enhance representation effectiveness. In view of cross-modal heterogeneity and high-dimensional sparsity, this paper summarizes three types of fusion strategies, namely early, mid-, and late-stage, as well as training paradigms such as collaborative training and multi-view alignment, and discusses the role of attention mechanisms in capturing complementary information. Further combined with application cases such as cancer prognosis, biomarker discovery, drug response prediction, and clinical decision support, this paper explains the advantages and challenges of fusion models in improving prediction performance, enhancing interpretability, and fitting clinical workflows. This paper proposes future research directions for clinical implementation: building a secure and compliant federal data lake, developing a causal explainable fusion framework, and strengthening deep coupling with medical care processes to achieve a closed-loop transformation from multimodal data to precision diagnosis and treatment.

Blood glucose concentration prediction method for type 1 diabetes mellitus based on multi-modal cross-attention mechanism fusion

WANG Mengxing, XUE Fuzhong, YANG Fan

Journal of Shandong University (Health Sciences). 2025, 63(8): 41-50. doi:10.6040/j.issn.1671-7554.0.2025.0511

Abstract ( 262 )

PDF (4779KB) ( 63 )

Save

References | Related Articles | Metrics

Objective To develop a blood glucose concentration prediction model for patients with type 1 diabetes mellitus(T1DM)by integrating multimodal information from flash glucose monitoring(FGM)data and structured electronic health records(EHR), so as to address the limitations of traditional unimodal models in capturing complex glucose fluctuation patterns and provide data support for personalized glycemic control strategies and early risk warning in clinical practice. Methods Based on the T1DiabetesGranada dataset, the study integrated multimodal features including FGM data, biochemical test indicators, demographic information, and diagnostic codes to construct a multimodal temporal prediction model, XCLA-Net. The model employed a one-dimensional convolutional neural network(1D-CNN)to extract short-term dynamic features of the glucose sequence, combined a long short-term memory(LSTM)network to capture long-term temporal dependencies, incorporated a cross-attention mechanism for multimodal semantic alignment, and introduced a self-normalizing neural network(SNN)to enhance the stability of the fused representations. Results XCLA-Net significantly outperformed multiple baseline models in terms of error metrics such as root mean square error(RMSE), mean absolute error(MAE), and mean absolute percentage error(MAPE). The MAPE values for 1-hour and 3-hour prediction tasks were 19.64% and 37.81%, respectively, indicating strong predictive accuracy across different temporal scales. Clarke error grid analysis showed that the majority of prediction points fell within Zone A, reflecting good clinical consistency. Ablation experiments confirm the critical roles of the cross-attention mechanism, 1D-CNN, and LSTM in enhancing model performance. Conclusion The proposed XCLA-Net model effectively improves the accuracy and stability of blood glucose prediction through multimodal data fusion and temporal modeling. It demonstrates favorable clinical interpretability and practical value, providing reliable support for personalized glycemic management and early risk prediction in patients with T1DM.

Cancer subtype clustering via multimodal decoupled contrastive learning

ZHANG Runze, XUE Fuzhong, YANG Fan

Journal of Shandong University (Health Sciences). 2025, 63(8): 51-60. doi:10.6040/j.issn.1671-7554.0.2025.0510

Abstract ( 203 )

PDF (4230KB) ( 90 )

Save

References | Related Articles | Metrics

Objective To propose a cancer subtype clustering model that integrates graph convolutional networks, self-attention mechanisms, and decoupled contrastive learning, based on multi-omics data from five cancer types in the cancer genome atlas(TCGA). Methods The model took four types of omics data from five cancer types in the TCGA database as input. For each omics type, it constructed a sample-wise relational graph and employed a graph convolutional network(GCN)to extract intra-omics structural information, thereby better preserving inter-sample feature differences. The features from different omics were concatenated and further fused through an attention mechanism, which automatically learned the relative importance and complementary relationships among omics modalities. Finally, a decoupled contrastive learning strategy was applied, and different augmented views of the same sample were used for unsupervised training, guiding the model to identify potential cancer subtypes in the absence of ground-truth labels. Results The model demonstrated good clustering performance across five cancer datasets, effectively dividing samples into distinct subtypes. In survival analysis, the survival curves of different subtypes showed significant separation, indicating that the identified subtypes were associated with different prognoses. Some subtypes also exhibited strong differentiation in clinical characteristics. Compared with several existing methods, the proposed model achieved favorable results on multiple evaluation metrics, yielding more stable clustering outcomes and demonstrating stronger biological interpretability. Conclusion This study proposes a cancer subtype clustering model that effectively integrates multi-omics data through the synergistic use of GCN, self-attention mechanisms, and contrastive learning. The model significantly improves the accuracy and clinical interpretability of cancer subtype clustering, offering a new perspective for cancer heterogeneity research and contributing to the development of personalized treatment strategies in precision medicine.

Causality extraction algorithm of medical text based on BERT and graph attention network

LIU Weilong, WANG Ding, ZHAO Chao, WANG Ning, ZHANG Xu, SU Ping, SONG Shudian, ZHANG Na, CHI Weiwei

Journal of Shandong University (Health Sciences). 2025, 63(8): 61-68. doi:10.6040/j.issn.1671-7554.0.2025.0152

Abstract ( 192 )

PDF (3437KB) ( 35 )

Save

References | Related Articles | Metrics

Objective To propose an algorithm capable of effectively extracting causal relationships to improve the accuracy of medical text processing. Methods The study proposed a bidirectional encoder representations from Transformers(BERT)-causal graph attention networks(CGAT)algorithm based on BERT and graph attention network. First, a causal relationship graph was constructed, and the BERT model was fine-tuned on medical texts to obtain optimized entity embeddings. Subsequently, a knowledge fusion channel integrated textual encoding information with causal structures, which were then fed into the graph attention network. A multi-head attention mechanism was employed to process information from different subspaces in parallel, enhancing the ability to capture complex semantic relationships. Finally, a dual-channel decoding layer was adopted to simultaneously extract entities and their causal relationships. Results Experiments on the self-built diabetes causal entity dataset showed that the model employing the BERT-CGAT algorithm had an improvement of 0.65% and 16.73% in precision rate(99.74%)and recall rate(81.04%)compared with the traditional BiLSTM-CRF baseline, and the F1 value were 80.83%. Conclusion The BERT-CGAT algorithm effectively enhances the accuracy of causal relationship extraction from medical texts by combining BERTs semantic feature extraction capability with the relational modeling advantages of graph neural networks, thereby validating the efficacy of the proposed method.

Application of metabolomic risk score in predicting cardiovascular outcomes in patients with type 2 diabetes mellitus

SHEN Lujia, LU Tianwei, GONG Weiming, ZHAO Yansong, WANG Shukang, YUAN Zhongshang

Journal of Shandong University (Health Sciences). 2025, 63(8): 69-78. doi:10.6040/j.issn.1671-7554.0.2025.0119

Abstract ( 207 )

PDF (3352KB) ( 51 )

Save

References | Related Articles | Metrics

Objective To identify metabolites associated with myocardial infarction, heart failure, and ischemic stroke respectively in a population with type 2 diabetes mellitus, thereby constructing a metabolomic risk score and evaluating whether incorporation of the metabolomic risk score into a traditional clinical model improves the models predictive accuracy. Methods Using data from the UK Biobank cohort, a multivariate Cox proportional risk regression model was applied to screen for metabolites associated with myocardial infarction, heart failure, and ischemic stroke, respectively, and a time-truncated sensitivity analysis was performed. Subsequently, metabolomic risk scores were constructed based on the LightGBM algorithm, and finally the scores were incorporated into traditional clinical models for model evaluation. Results Multivariate Cox proportional risk regression models and time-truncated sensitivity analyses identified a total of 119, 77, and 12 metabolites associated with myocardial infarction, heart failure, and ischemic stroke, respectively, and these metabolites were then used to construct metabolomic risk scores. Incorporation of the constructed metabolomic risk scores into traditional clinical models significantly improved model prediction performance, with model AUCs improving to 0.804, 0.900, and 0.844 for the three diseases respectively, and the improvements were 0.145, 0.198, and 0.188, compared to utilizing only traditional clinical models. In addition, the sensitivity and specificity results showed that the three models had high prediction accuracy(sensitivity: 0.706, 0.804, 0.801; specificity: 0.763, 0.861, 0.722), and the calibration curves and decision curves likewise showed that the models had good prediction performance. Conclusion Incorporation of metabolomic risk scores into traditional clinical models can significantly improve the predictive accuracy of cardiovascular disease in the type 2 diabetes mellitus population.

Multi-cancer risk prediction model based on multi-modal data fusion

LI Qian, YANG Fan, XUE Fuzhong

Journal of Shandong University (Health Sciences). 2025, 63(8): 79-85. doi:10.6040/j.issn.1671-7554.0.2025.0118

Abstract ( 276 )

PDF (6742KB) ( 145 )

Save

References | Related Articles | Metrics

Objective To develop a multi-cancer risk prediction model using data from 15 common cancers in the UK Biobank, employing a multi-modal data fusion approach, so as to explore the application of genomic and clinical data in cancer risk prediction, with the goal of enhancing early cancer detection accuracy and providing valuable insights for personalized medicine. Methods The rigorous quality control was performed to the data. High-dimensional genomic data were then transformed into image representations and processed using convolutional neural networks, while clinical data were modeled using multi-layer perceptron. An attention mechanism was incorporated to perform weighted fusion of features from both genomic and clinical modalities, aiming to optimize predictive performance. Results The integration of genomic and clinical data through a multi-modal fusion model resulted in a significant improvement in cancer prediction accuracy. Features extracted by convolutional neural networks from genomic data and by multi-layer perceptron from clinical data effectively augmented the predictive capability of the model, enhancing both the accuracy and robustness of the predictions. Conclusion This study introduces a novel multi-cancer risk prediction framework that integrates genomic and clinical data. The application of multi-modal deep learning techniques, including convolutional neural networks, multi-layer perceptrons, and attention mechanisms, significantly enhances early cancer prediction accuracy. The findings provide robust early support for cancer diagnosis and personalized treatment strategies, demonstrating the potential of multi-modal approaches in precision oncology.

Predicting ICU sepsis mortality using self-attention mechanism

LI Xiaoqi, LIU Peili, CHENG Hong, ZHAO Yanyan

Journal of Shandong University (Health Sciences). 2025, 63(8): 86-93. doi:10.6040/j.issn.1671-7554.0.2025.0117

Abstract ( 198 )

PDF (2006KB) ( 143 )

Save

References | Related Articles | Metrics

Objective To predict sepsis mortality in Intensive Care Unit(ICU)using a self-attention mechanism model. Methods Sepsis patients who meet the Sepsis-3 criteria were selected from the MIMIC-IV database. Multiple logistic regression analysis was performed to assess the impact of ethnicity on sepsis mortality. To further validate this relationship, parallel predictive models(with and without ethnicity)were constructed and their performance metrics were compared. The dataset was split into training and validation sets at a 1∶1 ratio. A binary cross-entropy loss function and the Adam optimizer were used to train the model for 1,000 iterations on the training set, and the models performance was evaluated on the validation set. Performance metrics included the area under the curve(AUC)of receiver operating characteristic(ROC)and accuracy. Results A total of 16,521 sepsis patients were included. The multiple logistic regression analysis showed no significant relationship between ethnicity and mortality, so ethnicity was not included in the subsequent model. Both models(with/without ethnicity)achieved identical AUC(0.82)and accuracy(0.88)on the validation set, outperforming traditional scoring systems(e.g., OASIS AUC: 0.70; LODS AUC: 0.74; SAPSII AUC: 0.75). Conclusion Ethnicity has no significant effect on the prediction of mortality in patients with sepsis. The prediction model built on the self-attention mechanism significantly improves the prediction performance of ICU sepsis mortality, outperforming traditional scoring systems.

A Bayesian network-based risk prediction study of stroke in patients with type 2 diabetes mellitus

CHEN Yingying, WANG Lu, HU Xifeng, ZHU Gaopei, XUE Fuzhong

Journal of Shandong University (Health Sciences). 2025, 63(8): 94-102. doi:10.6040/j.issn.1671-7554.0.2024.1217

Abstract ( 245 )

PDF (3817KB) ( 68 )

Save

References | Related Articles | Metrics

Objective To construct a simple, economical and clinically applicable prediction model for T2DM complicated stroke, so as to accurately predict the risk of diabetes mellitus and stroke and obtain relevant risk factors. Methods Based on the data of Cheeloo Lifespan Electronic Health Research Data-library(CHeeloo LEAD), univariate Cox regression analysis was used to screen the risk factors associated with diabetes mellitus complicated with stroke, and the combined Cox and Bayesian network models were used to construct a risk prediction model, then the prediction performance of the model was evaluated from two aspects of identification and calibration. Results A total of 15,528 diabetic patients in CHeeloo LEAD database from January 1, 2015 to October 31, 2017 were included in this study, and 2,552 cases of stroke occurred from January 1, 2015 to January 1, 2023. Sixty-seven potential risk factors related to stroke were screened out by univariate analysis and used to construct Bayesian networks, and 4 independent risk factors, including age, transient and sudden-onset diseases, circulatory system diseases, and sequelae of cerebrovascular diseases, were screened by multivariate Cox regression model. Combined with the Cox model and Bayesian network model, the prediction model of T2DM complicated stroke was constructed to predict the risk of stroke in individuals in 3 years, and the AUC of the training set and test set were 0.814 and 0.816, respectively, exhibiting the basically consistent results. Conclusion Age, transient and sudden-onset diseases, circulatory diseases, and sequelae of cerebrovascular diseases are important risk factors for the increased risk of stroke in patients with T2DM. In clinical practice, attention should be paid to the occurrence of brain lesions in patients with T2DM. The identification of relevant risk factors and the strengthening of monitoring and management should be carried out to reduce the incidence of stroke and improve the prognosis of patients.

Risk of recurrent acute coronary syndrome associated with coronary revascularization

XU Ruize, WANG Jinlan, LUO Qingxin, XU Zhaoke, LYU Mingyue, ZHANG Shuo, YAN Luning, HU Xifeng, ZHAO Qingbo, ZHU Gaopei, LI Lei, XUE Fuzhong

Journal of Shandong University (Health Sciences). 2025, 63(8): 103-110. doi:10.6040/j.issn.1671-7554.0.2024.1358

Abstract ( 186 )

PDF (1341KB) ( 36 )

Save

References | Related Articles | Metrics

Objective To investigate the risk of recurrent acute coronary syndrome(ACS)in patients who had undergone coronary revascularization for ACS. Methods A real-world study was conducted using data from patients diagnosed with ACS in the Cheeloo Lifespan Electronic Health Research Data-library(Cheeloo LEAD), employing a cohort design based on propensity score matching(PSM). Patients who underwent coronary revascularization were classified as the exposure group, while those who did not undergo the procedure and received only drug therapy were assigned to the control group. L1-regularized PSM was used to control for confounding bias. Descriptive analyses were performed using the covariate description table and the description table of incidence density before and after matching. Kaplan-Meier(KM)survival curves were plotted, and the average causal effect(ACE)was estimated using the Cox proportional hazards regression model, and sensitivity analyses and subgroup analyses were also conducted. Results After L1-regularized PSM, the majority of standardized differences for covariates were below 0.1, indicating substantially improved post-matching balance and comparability between the groups. The P-values for covariate differences increased significantly after matching, further supporting the effectiveness of the matching procedure. Among patients with recurrent ACS, the incidence density in males was slightly higher than that in females both before and after matching, and the ages at onset were predominantly between 75 and 80 years. After matching, the five-year survival rate in the exposure group was significantly lower than that in the control group ［0.46(95%CI: 0.42-0.51)vs. 0.57(95%CI: 0.54-0.60)］, and the difference in the KM survival curves between the two groups was statistically significant(P<0.05). The Cox proportional hazards regression model showed that coronary revascularization was associated with an increased risk of recurrent ACS(HR=1.38, 95%CI: 1.19-1.61, P<0.05). The Schoenfeld residual test indicated that coronary revascularization met the proportional hazards assumption(χ2=3.53, P>0.05). The results of the sensitivity analyses and the subgroup analysis of percutaneous coronary intervention(PCI)were consistent with those of the primary analysis, whereas the association between coronary artery bypass grafting(CABG)and an increased risk of recurrent ACS was not statistically significant(P>0.05). Conclusion Patients with ACS who undergo coronary revascularization may have a risk of recurrent ACS. In clinical practice, clinicians should consider patients individual risk factors and potential long-term outcomes, weigh the risks and benefits of coronary revascularization, optimize treatment strategies, and improve the long-term prognosis of patients.

Application of functional brain network analysis in aphasia: insights into neuropathological mechanisms, clinical diagnosis, and assessment of therapeutic outcome

LUO Qi, WANG Xia, JIANG Meng

Journal of Shandong University (Health Sciences). 2025, 63(8): 111-126. doi:10.6040/j.issn.1671-7554.0.2024.1456

Abstract ( 185 )

PDF (3297KB) ( 34 )

Save

References | Related Articles | Metrics

Functional brain network analysis has emerged as a powerful approach to study aphasia by characterizing the synchronization patterns and connectivity profiles of neural activity across distributed brain regions. Utilizing advanced non-invasive neuroimaging(e.g. functional magnetic resonance imaging, fMRI)and electrophysiological(e.g. electroencephalography, EEG)techniques, functional networks are constructed to identify abnormal functional connectivity and dynamic network reorganization in aphasic patients. The present systematic review critically evaluates the application of functional brain network analysis in aphasia research, with particular emphasis on elucidation of neuropathological mechanisms, subtype classification, severity stratification, and assessment of therapeutic outcome. Through integrative analysis of both static and dynamic network properties across multiple neuroimaging modalities, we identify consistent patterns of network dysfunction in aphasia, including topological perturbations, functional connectivity reconfigurations, frequency-band-specific oscillations, and compensatory network reorganization. Specifically, this analytical framework demonstrates significant clinical utility by improving the sensitivity and specificity of conventional diagnostic protocols, providing quantitative biomarkers for grading disease severity, and enabling objective assessment of treatment-induced neuroplastic changes. By synthesizing current evidence, this review aims to advance the understanding for future investigations of the neural substrates of aphasia and to inform the development of personalized treatment interventions.

Table of Content