多模态模型在肾脏病领域的应用

doi:10.6040/j.issn.1671-7554.0.2025.0094

摘要/Abstract

摘要： 肾脏是维持人体健康的重要器官,其功能障碍相关疾病由多种病因引发,依据疾病发展过程,通常分为急性肾损伤和慢性肾脏病。临床上通过实验室检查、影像学和肾组织活检等方法发现。在人工智能迅速发展的今天,基于多模态模型的人工智能是一个新兴发展的专业领域,也是一个高效分析和挖掘数据的方法,为肾脏病个体化精确诊治提供了可能性。近年来,基于多模态模型的人工智能技术已被广泛应用于急性肾损伤、慢性肾脏病、血液透析、肾移植和肾脏肿瘤等多种临床场景,为疾病的精准管理治疗提供了支持。本文就多模态模型方法论、多模态模型在肾脏病领域的应用、当前面临的挑战及未来展望进行综述,旨在为多模态模型在肾脏病领域的进一步推广应用提供参考,并揭示其在临床实践应用中的巨大优势和潜力。

关键词: 人工智能, 大语言模型, 多模态模型, 急性肾损伤, 慢性肾脏病

Abstract: Kidneys are vital organs for maintaining human health, and diseases associated with their dysfunction are caused by a variety of aetiological factors. Based on the progression of the disease, they are usually categorised as acute kidney injury and chronic kidney disease. It is detected clinically by laboratory tests, imaging and renal tissue biopsy. In the rapid development of artificial intelligence(AI), multimodal model-based AI is an emerging and evolving field of expertise and an efficient way to analyse and mine data, which providing the possibility of individualised and precise diagnosis and treatment of kidney disease. In recent years, multimodal model-based AI techniques have been widely used in a variety of clinical scenarios, including acute kidney injury, chronic kidney disease, hamodialysis, kidney transplantation, and renal tumour, to support precise management and treatment of diseases. In this paper, we provide an overview of multimodal modeling methodologies, applications of multimodal models in the field of renal diseases, current challenges and future perspectives, with the aim of providing references for the further popularization and application of multimodal models in the field of renal diseases, as well as revealing their great advantages and potentials in the application of clinical practice.

Key words: Artificial intelligence, Large language model, Multimodal models, Acute kidney injury, Chronic kidney disease

中图分类号:

R319

武琪琪,成淼淼,肖晓燕. 多模态模型在肾脏病领域的应用[J]. 山东大学学报 (医学版), 2025, 63(10): 117-124.

WU Qiqi, CHENG Miaomiao, XIAO Xiaoyan. Multimodal models in the field of kidney disease[J]. Journal of Shandong University (Health Sciences), 2025, 63(10): 117-124.

参考文献

[1] Ge XY, Lan ZK, Lan QQ, et al. Diagnostic accuracy of ultrasound-based multimodal radiomics modeling for fibrosis detection in chronic kidney disease[J]. Eur Radiol, 2023, 33(4): 2386-2398.
[2] Franke M, Kramarczyk A, Taylan C, et al. Ultrasound-guided percutaneous renal biopsy in 295 children and adolescents: role of ultrasound and analysis of complications[J]. PloS One, 2014, 9(12): e114737. doi:10.1371/journal.pone.0114737
[3] Halimi JM, Gatault P, Longuet H, et al. Major bleeding and risk of death after percutaneous native kidney biopsies: a French nationwide cohort study[J]. Clin J Am Soc Nephrol, 2020, 15(11): 1587-1594.
[4] Occhipinti A, Verma S, Doan LMT, et al. Mechanism-aware and multimodal AI: beyond model-agnostic interpretation[J]. Trends Cell Biol, 2024, 34(2): 85-89.
[5] 岳潭. 基于多模态学习的图文信息融合方法的研究及应用[D]. 北京:北京邮电大学,2024.
[6] 陈晋音, 席昌坤, 郑海斌, 等. 多模态大语言模型的安全性研究综述[J]. 计算机科学, 2025, 52(7): 315-341.
[7] Uhm KH, Jung SW, Hong SH, et al. Lesion-aware cross-phase attention network for renal tumor subtype classification on multi-phase CT scans[J]. Comput Biol Med, 2024, 178: 108746.doi:10.1016/j.compbiomed.2024.108746
[8] Forbes AN, Xu D, Cohen S, et al. Discovery of therapeutic targets in cancer using chromatin accessibility and transcriptomic data[J]. Cell Syst, 2024, 15(9): 824-837.
[9] Cheng L, Huang Q, Zhu Z, et al. MoAGL-SA: a multi-omics adaptive integration method with graph learning and self attention for cancer subtype classification[J]. BMC Bioinformatics, 2024, 25(1): 364. doi:10.1186/s12859-024-05989-y
[10] Tian H, He X, Yang K, et al. DAPNet: multi-view graph contrastive network incorporating disease clinical and molecular associations for disease progression prediction[J]. BMC Med Inform Decis Mak, 2024, 24(1): 345. doi: 10.1186/s12911-024-02756-0
[11] Liu R, Wang Q, Zhang X. Identification of prognostic coagulation-related signatures in clear cell renal cell carcinoma through integrated multi-omics analysis and machine learning[J]. Comput Biol Med, 2024, 168: 107779. doi: 10.1016/j.compbiomed.2023.107779
[12] Pan KH, Yao F, Hong WF, et al. Multimodal radiomics based on ¹⁸F-Prostate-specific membrane antigen-1007 PET/CT and multiparametric MRI for prostate cancer extracapsular extension prediction[J].Br J Radiol, 2024, 97(1154): 408-414.
[13] Huang KB, Gui CP, Xu YZ, et al. A multi-classifier system integrated by clinico-histology-genomic analysis for predicting recurrence of papillary renal cell carcinoma[J]. Nat Commun, 2024, 15(1): 6215.doi: 10.1038/s41467-024-50369-y
[14] Wang Z, Xue Y, Liu Z, et al. AI fusion of multisource data identifies key features of vitiligo[J]. Sci Rep, 2024, 14(1): 24278. doi: 10.1038/s41598-024-75062-4
[15] Zhu L, Huang R, Zhou Z, et al. Prediction of renal function 1 year after transplantation using machine learning methods based on ultrasound radiomics combined with clinical and imaging features[J]. Ultrason Imaging, 2023, 45(2): 85-96.
[16] Leube J, Horn M, Hartrampf PE, et al. PSMA-PET improves deep learning-based automated CT kidney segmentation[J]. Z Fur Med Phys, 2023, 34(2): 231-241.
[17] Shehata M, Shalaby A, Switala AE, et al. A multimodal computer-aided diagnostic system for precise identification of renal allograft rejection: Preliminary results[J]. Med phys, 2020, 47(6): 2427-2440.
[18] Wang X, Cai S, Wang H, et al. Deep-learning-based renal artery stenosis diagnosis via multimodal fusion[J]. J App Clin Med Phys, 2024, 25(3): e14298. doi: 10.1002/acm2.14298
[19] Jeon SK, Joo I, Park J, et al. Fully-automated multi-organ segmentation tool applicable to both non-contrast and post-contrast abdominal CT: deep learning algorithm developed using dual-energy CT images[J]. Sci Rep, 2024, 14(1): 4378. doi: 10.1038/s41598-024-55137-y
[20] 唐玉宁, 潘天岳, 董智慧等. 深度学习在主动脉影像自动分割中的研究进展[J]. 山东大学学报(医学版), 2024, 62(9): 66-73. TANG Yuning, PAN Tianyue, DONG Zhihui, et al. Research progress of deep learning in automatic segmentation of aortic images[J]. Journal of Shandong University(Health Sciences), 2024, 62(9): 66-73.
[21] Bifarin OO, Gaul DA, Sah S, et al. Machine learning-enabled renal cell carcinoma status prediction using multiplatform urine-based metabolomics[J]. J Proteom Res, 2021, 20(7): 3629-3641.
[22] Chen Z, Wang Y, Ying MTC, et al. Interpretable machine learning model integrating clinical and elastosonographic features to detect renal fibrosis in Asian patients with chronic kidney disease[J]. J Nephrol, 2024, 37(4): 1027-1039.
[23] Xiong H, Wei B, Huang Y, et al. A novel approach to the cause of death identification-multi-strategy integration of multi-organ FTIR spectroscopy information using machine learning[J]. Talanta, 2025, 282: 127040. doi: 10.1016/j.talanta.2024.127040
[24] Handa K, Sasaki S, Asano S, et al. Prediction of inhibitory activity against the MATE1 transporter via combined fingerprint- and physics-based machine learning models[J]. J Chem Inf and Model, 2024, 64(18): 7068-7076.
[25] Chen L, Ren Y, Yuan Y, et al. Multi-parametric MRI-based machine learning model for prediction of patholo-gical grade of renal injury in a rat kidney cold ischemia-reperfusion injury model[J]. BMC Med Imaging, 2024, 24: 188. doi: 10.1186/s12880-024-01320-6
[26] Ding JL, Xing ZY, Jiang ZX, et al. Evaluation of renal dysfunction using texture analysis based on DWI, BOLD, and susceptibility-weighted imaging[J]. Eur Radiol, 2019, 29(5): 2293-2301.
[27] Oetjen J, Aichler M, Trede D, et al. MRI-compatible pipeline for three-dimensional MALDI imaging mass spectrometry using PAXgene fixation[J]. J Proteom, 2013, 90(2): 52-60.
[28] Guo SZ, Chen HJ, Sheng XY, et al. Cross-modal transfer learning based on an improved CycleGAN model for accurate kidney segmentation in ultrasound images[J]. Ultrasound Med Biol, 2024, 50(11): 1638-1645.
[29] Liu K, Yuan BR, Zhang XZ, et al. Characterizing the temporal changes in association between modifiable risk factors and acute kidney injury with multi-view analysis[J]. Int J Med Inform, 2022, 163: 104785. doi: 10.1016/j.ijmedinf.2022.104785
[30] Grams ME, Sang Y, Coresh J, et al. Acute kidney injury after major surgery: a retrospective analysis of veterans health administration data[J]. Am J Kidney Dis, 2016, 67(6): 872-880.
[31] Hofer IS, Kupina M, Laddaran L, et al. Integration of feature vectors from raw laboratory, medication and procedure names improves the precision and recall of models to predict postoperative mortality and acute kidney injury[J]. Sci Rep, 2022, 12(1): 10254. doi: 10.1038/s41598-022-13879-7
[32] Weaver JK, Milford K, Rickard M, et al. Deep learning imaging features derived from kidney ultrasounds predict chronic kidney disease progression in children with posterior urethral valves[J]. Pediatr Nephrol, 2023, 38(3): 839-846.
[33] 张炯. 功能磁共振对慢性肾脏病患者肾脏纤维化状态的评估及多模态模型的建立[D]. 上海: 中国人民解放军海军军医大学,2022.
[34] Qin X, Liu X, Xia L, et al. Multimodal ultrasound deep learning to detect fibrosis in early chronic kidney disease[J]. Ren Fail, 2024, 46(2): 2417740. doi: 10.1080/0886022X.2024.2417740
[35] Lee SM, Kang M, Byeon K, et al. Machine learning-aided chronic kidney disease diagnosis based on ultrasound imaging integrated with computer-extracted measurable features[J]. J Digit Imag, 2022, 35(5): 1091-1100.
[36] Si S, Liu H, Xu L, et al. Identification of novel therapeutic targets for chronic kidney disease and kidney function by integrating multi-omics proteome with transcriptome[J]. Genome Med, 2024, 16(1): 84.doi: 10.1186/s13073-024-01356-x
[37] Qin X, Xia L, Ma Q, et al. Development of a novel combined nomogram model integrating deep learning radiomics to diagnose IgA nephropathy clinically[J]. Ren Fail, 2023, 45(2): 2271104. doi: 10.1080/0886022X.2023.2271104
[38] Chen T, Chen T, Xu W, et al. Development and external validation of a multidimensional deep learning model to dynamically predict kidney outcomes in IgA nephropathy[J]. Clin J Am Soc Nephrol, 2024, 19(7): 898-907.
[39] 杨玲. 联合影像组学及肾脏多模态磁共振成像技术建立IgA肾病患者病理分级影像预测模型及类固醇激素疗效预测模型的临床研究[D]. 成都: 四川大学, 2022.
[40] Chen W, Zhang L, Cai G, et al. Machine learning-based multimodal MRI texture analysis for assessing renal function and fibrosis in diabetic nephropathy: a retrospective study[J]. Front Endocrinol, 2023, 14: 1050078. doi: 10.3389/fendo.2023.1050078
[41] He XZ, Hu ZX, Dev H, et al. Test retest reproducibility of organ volume measurements in ADPKD using 3D multimodality deep learning[J]. Acad Radiol, 2024, 31(3): 889-899.
[42] Yun D, Yang HL, Kim SG, et al. Real-time dual prediction of intradialytic hypotension and hypertension using an explainable deep learning model[J]. Sci Rep, 2023, 13: 18054. doi: 10.1038/s41598-023-45282-1
[43] Shehata M, Ghazal M, Khalifeh HA, et al. A deep learning-based cad system for renal allograft assessment: diffusion, bold, and clinical biomarkers [J]. Proc Int Conf Image Proc, 2020, 2020: 355-359. doi: 10.1109/ICIP40778.2020.9190818
[44] Siegerist F, Hay E, Dikou JS, et al. ScoMorphoFISH: a deep learning enabled toolbox for single-cell single-mRNA quantification and correlative(ultra-)morphometry[J]. J Cell Mol Med, 2022, 26(12): 3513-3526.
[45] Mazin A, Hawkins SH, Stringfield O, et al. Identification of sarcomatoid differentiation in renal cell carcinoma by machine learning on multiparametric MRI[J]. Sci Rep, 2021, 11(1): 3785. doi: 10.1038/s41598-021-83271-4
[46] Prade VM, Sun N, Shen J, et al. The synergism of spatial metabolomics and morphometry improves machine learning-based renal tumour subtype classification[J]. Clin Transl Med, 2022, 12(2): e666. doi: 10.1002/ctm2.666
[47] Durán I, Castellano D, Puente J, et al. Exploring the synergistic effects of cabozantinib and a programmed cell death protein 1 inhibitor in metastatic renal cell carcinoma with machine learning[J]. Oncotarget, 2022, 13: 237-256. doi: 10.18632/oncotarget.28183
[48] Cheng G, Zhou Z, Li S, et al. Integration of proteomics and transcriptomics to construct a prognostic signature of renal clear cell carcinoma[J]. Int J Med Sci, 2024, 21(11): 2215-2232.
[49] Zhu YF, Liu ML, Zheng WT, et al. Predictive model of CK7 expression in patients with clear cell renal cell carcinoma by combined multimodal ultrasound diagnostic techniques: a retrospective study[J]. Ultrasound Med Biol, 2024, 50(4): 520-527.
[50] Mahootiha M, Ali Qadir H, Bergsland J, et al. Multimodal deep learning for personalized renal cell carcinoma prognosis: Integrating CT imaging and clinical data[J]. Comput Meth Programs Biomed, 2024, 244: 107978. doi: 10.1016/j.cmpb.2023.107978
[51] England JR, Cheng PM. Artificial intelligence for medical image analysis: a guide for authors and reviewers[J]. AJR Am J roentgenol, 2019, 212(3): 513-519.
[52] Jee J, Fong C, Pichotta K, et al. Automated real-world data integration improves cancer outcome prediction[J]. Nature, 2024, 636(8043): 728-736.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed