Journal of Shandong University (Health Sciences) ›› 2020, Vol. 58 ›› Issue (11): 17-23.doi: 10.6040/j.issn.1671-7554.0.2020.1227

• Special topic on new progress in ophthalmic artificial intelligence • Previous Articles     Next Articles

New developments in ophthalmic AI algorithms

Zongyuan GE*(),Wanji HE,Lie JU,Xuan YAO,Lin WANG,Yelin HUANG,Zhiwen YANG,Jianhao XIONG,Yining BAO,Ming LI,Bing ZHANG,Xin ZHAO   

  1. Monash e-Research Center and Faculty of Engineering, Monash University, Melbourne 3800, Victoria, Australia
  • Received:2020-08-29 Online:2020-11-10 Published:2020-11-04
  • Contact: Zongyuan GE E-mail:zongyuan.ge@monash.edu

RichHTML

56

PDF (PC)

1372

Abstract:

Deep neural networks, combined with high quality annotated medical data and low-cost GPU devices, have been successfully implemented in the field of ophthalmology. Impressive grounding outcomes have occurred in both in-hospital and out-hospital scenarios. Some of the published results demonstrated AI could achieve better diagnosis performance than general practice in the diagnosis of certain retinal diseases. In this article, we will discuss how classification, detection, segmentation and domain adaptation play their roles in AI ophthalmology. We will also discuss the limitations of the current state-of-the-art (SOTA) algorithms, hoping to provide prevision for future research in this field.

Key words: Deep neural networks, Retinal disease diagnosis, Artificial intelligence, Computer vision

CLC Number: 

  • R770.4
1 路寻. 卓越的人工智能科学家——马文·明斯基[J]. 自然辩证法通讯, 2010, 32 (2): 104- 111, 128.
2 Govindaiah A, Smith RT, Bhuiyan A. A new and improved method for automated screening of age-related macular degeneration using ensemble deep neural networks[C]. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2018: 702-705.
3 Russakoff DB , Lamin A , Oakley JD , et al. Deep learning for prediction of amd progression: A pilot study[J]. Invest Ophthalmol Vis Sci, 2019, 60 (2): 712- 722.
doi: 10.1167/iovs.18-25325
4 Ting DSW , Cheung CY , Lim G , et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes[J]. JAMA, 2017, 318 (22): 2211- 2223.
doi: 10.1001/jama.2017.18152
5 Brown JM , Campbell JP , Beers A , et al. Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks[J]. JAMA Ophthalmol, 2018, 136 (7): 803- 810.
doi: 10.1001/jamaophthalmol.2018.1934
6 Kesim C , Tas AY , Karslıoglu MZ , et al. Validation results of a deep learning algorithm for detection of diabetic retinopathy with lesion localization from retinal fundus photographs[J]. Invest Ophthalmol Vis Sci, 2020, 62 (7): 1626.
7 Lakra A, Tripathi P, Keshari R, et al. Segdensenet: Iris segmentation for pre and post cataract surgery[J/OL]. arXiv, 2018: 1801.10100[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2018arXiv180110100L.
8 Dong Y, Zhang Q, Qiao Z, et al. Classification of cataract fundus image based on deep learning[C]. Proceedings of IEEE International Conference on Imaging Systems and Techniques, 2017: 1-5.
9 Varadarajan AV , Poplin R , Blumer K , et al. Deep learning for predicting refractive error from retinal fundus images[J]. Invest Ophthalmol Vis Sci, 2018, 59 (7): 2861- 2868.
doi: 10.1167/iovs.18-23887
10 Yau JW , Rogers SL , Kawasaki R , et al. Global prevalence and major risk factors of diabetic retinopathy[J]. Diabetes Care, 2012, 35 (3): 556- 564.
11 Ortiz A , Munilla J , Gorriz JM , et al. Ensembles of deep learning architectures for the early diagnosis of the alzheimer's disease[J]. Int J Neural Syst, 2016, 26 (7): 1650025.
doi: 10.1142/S0129065716500258
12 Gulshan V , Peng L , Coram M , et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs[J]. JAMA, 2016, 316 (22): 2402- 2410.
doi: 10.1001/jama.2016.17216
13 Diaz-Pinto A , Morales S , Naranjo V , et al. Cnns for automatic glaucoma assessment using fundus images: An extensive validation[J]. Biomed Eng Online, 2019, 18 (1): 29.
doi: 10.1186/s12938-019-0649-y
14 Wang X, Ju L, Zhao X, et al. Retinal abnormalities recognition using regional multitask learning[C]. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2019: 30-38.
15 Ruder S. An overview of multi-task learning in deep neural networks[J/OL]. arXiv, 2017: 1706.05098[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2017arXiv170605098R.
16 Ravindran S . How artificial intelligence is helping to prevent blindness[J]. Nature, 2019, 568 (7751): 6.
17 Nair A , Merkel S , Straub J , et al. Real-time pupil detection for fundus imager using single-shot detector and hard- negative training[J]. Invest Ophthalmol Vis Sci, 2020, 61 (9): PB00110- PB00110.
18 Mitra A , Banerjee PS , Roy S , et al. The region of interest localization for glaucoma analysis from retinal fundus image using deep learning[J]. Comput Methods Programs Biomed, 2018, 165 (2018): 25- 35.
19 Selvaraju RR, Das A, Vedantam R, et al. Grad-cam: Why did you say that?[J/OL]. arXiv, 2016: 1611.07450.[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2016arXiv161107450S.
20 Haloi M. Improved microaneurysm detection using deep neural networks[J/OL]. arXiv, 2015: 1505.04424[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2015arXiv-150504424H.
21 Khojasteh P , Passos Junior LA , Carvalho T , et al. Exudate detection in fundus images using deeply-learnable features[J]. Comput Biol Med, 2019, 104 (2019): 62- 69.
22 Mitani A , Huang A , Venugopalan S , et al. Detection of anaemia from retinal fundus images via deep learning[J]. Nat Biomed Eng, 2020, 4 (1): 18- 27.
23 Quellec G , Lamard M , Conze PH , et al. Automatic detection of multiple pathologies in fundus photographs[J]. Invest Ophthalmol Vis Sci, 2020, 61 (7): 1641- 6141.
24 Fang L , Cunefare D , Wang C , et al. Automatic segmentation of nine retinal layer boundaries in oct images of non-exudative amd patients using deep learning and graph search[J]. Biomed Opt Express, 2017, 8 (5): 2732- 2744.
doi: 10.1364/BOE.8.002732
25 De Fauw J , Ledsam JR , Romera-Paredes B , et al. Clinically applicable deep learning for diagnosis and referral in retinal disease[J]. Nat Med, 2018, 24 (9): 1342- 1350.
doi: 10.1038/s41591-018-0107-6
26 Ma W, Yu S, Ma K, et al. Multi-task neural networks with spatial activation for retinal vessel segmentation and artery/vein classification[C]. Proceedings of the Medical Image Computing and Computer Assisted Intervention-MICCAI 2019, Cham, F, 2019: 769-778.
27 Sevastopolsky A . Optic disc and cup segmentation methods for glaucoma detection with modification of u-net convolutional neural network[J]. Pattern Recog& Ima Anal, 2017, 27 (3): 618- 624.
28 Sarhan MH, Albarqouni S, Yigitsoy M, et al. Multi-scale microaneurysms segmentation using embedding triplet loss[C]. Proceedings of the Medical Image Computing and Computer Assisted Intervention-MICCAI 2019, Cham, F, 2019: 174-182.
29 Fu H , Cheng J , Xu Y , et al. Joint optic disc and cup segmentation based on multi-label deep network and polar transformation[J]. IEEE Trans Med Imaging, 2018, 37 (7): 1597- 1605.
doi: 10.1109/TMI.2018.2791488
30 Shelhamer E , Long J , Darrell T . Fully convolutional networks for semantic segmentation[J]. IEEE Trans Pattern Anal Mach Intell, 2017, 39 (4): 640- 651.
doi: 10.1109/TPAMI.2016.2572683
31 Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[J/OL]. arXiv, 2015: 1505.04597[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2015arXiv150504597R.
32 Chen LC, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[J/OL]. arXiv, 2018: 1802.02611[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2018arXiv180202611C.
33 Yan X, Jiang W, Shi Y, et al. Ms-nas: Multi-scale neural architecture search for medical image segmentation[J/OL]. arXiv, 2020: 2007.06151[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2020arXiv200706-151Y.
34 Jiang Y, Tan N. Retinal vessel segmentation based on conditional deep convolutional generative adversarial networks[J/OL]. arXiv, 2018: 1805.04224[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2018arXiv180504-224J.
35 Redko I, Habrard A, Morvant E, et al. 2 - domain adaptation problem[M]// Redko I, Habrard A, Morvant E, et al. Advances in domain adaptation theory. Amsterdam: Elsevier, 2019: 21-36.
36 Ma Y, Lao S, Takikawa E, et al. Discriminant analysis in correlation similarity measure space[C]. Proceedings of the 24th international conference on Machine learning, 2007: 577-584.
37 Bruzzone L , Marconcini M . Domain adaptation problems: A dasvm classification technique and a circular validation strategy[J]. IEEE Trans Pattern Anal Mach Intell, 2010, 32 (5): 770- 787.
doi: 10.1109/TPAMI.2009.57
38 Gheisari M , Baghshah MS . Unsupervised domain adaptation via representation learning and adaptive classifier learning[J]. Neurocomputing, 2015, 165 (OCT.1): 300- 311.
39 Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J/OL]. arXiv, 2015: 1503.02531[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2015arXiv150302531H.
40 Borgwardt KM , Gretton A , Rasch MJ , et al. Integrating structured biological data by kernel maximum mean discrepancy[J]. Bioinformatics, 2006, 22 (14): E49- E57.
doi: 10.1093/bioinformatics/btl242
41 Ghifary M, Bastiaan Kleijn W, Zhang M. Domain adaptive neural networks for object recognition[J/OL]. arXiv, 2014: 1409.6041[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2014ar, Xiv1409.6041G.
42 Li Y , Wang N , Shi J , et al. Adaptive batch normalization for practical domain adaptation[J]. Pattern Recognition, 2018, 80 (2018): 109- 117.
43 Gopalan R, Ruonan L, Chellappa R. Domain adaptation for object recognition: An unsupervised approach[C]. International Conference on Computer, 2011: 999-1006.
44 Liu M-Y, Tuzel O. Coupled generative adversarial networks[J/OL]. arXiv, 2016: 1606.07536[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2016arXiv1606075-36L.
45 Ganin Y, Lempitsky V. Unsupervised domain adaptation by backpropagation[J/OL]. arXiv, 2014: 1409.7495[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2014arXiv1409.7495G.
46 Bousmalis K, Trigeorgis G, Silberman N, et al. Domain separation networks[J/OL]. arXiv, 2016: 1608.06019[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2016arXiv160806019B.
47 Yi Z, Zhang H, Tan P, et al. Dualgan: Unsupervised dual learning for image-to-image translation[J/OL]. arXiv, 2017: 1704.02510[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2017arXiv170402510Y.
48 Zhu J, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C].IEEE International Conference on Computer Vision (ICCV), 2017: 2242-2251.
49 Yang Y, Soatto S. Fda: Fourier domain adaptation for semantic segmentation[J/OL]. arXiv, 2020: 2004.05498[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2020arXiv200405498Y.
50 Johnson JM , Khoshgoftaar TM . Survey on deep learning with class imbalance[J]. Journal of Big Data, 2019, 6 (1): 1- 54.
51 Xiong J, He AW, Fu M, et al. Improve unseen domain generalization via enhanced local color transformation[C]. Proceedings of the Medical Image Computing and Computer Assisted Intervention-MICCAI 2020, Cham, F, 2020: 433-443.
52 Davide C . Can we open the black box of ai?[J]. Nature, 2016, 538 (7623): 20.
doi: 10.1038/538020a
53 Ge Z, Demyanov S, Chen Z, et al. Generative openmax for multi-class open set classification[J/OL]. arXiv, 2017: 1707.07418[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2017arXiv170707418G.
54 Heaven WD. Google's medical ai was super accurate in a lab. Real life was a different story[EB/OL]. (2020-08-16)[2020-08-18]. https://www.technologyreview.com/2020/04/27/1000658/google-medical-ai-accurate-lab-real-life-clinic-covid-diabetes-retina-disease/.
55 Airdoc取得nmpa人工智能医疗器械三类注册证[EB/OL]. (2020-08-16)[2020-08-18]. https://www.sohu.com/a/412357707_102972.
[1] LIANG Chen, YAO Lijuan, LYU Longfei, TANG Ze, QIN Da, CUI Youbin,YU Xiaoqi. Application of 3D reconstruction combined with CT-guided puncture localization in minimally invasive treatment of pulmonary ground-glass nodules [J]. Journal of Shandong University (Health Sciences), 2026, 64(5): 74-82.
[2] Intelligent Orthopedics Subgroup of Chinese Association of Orthopedic, Subgroup for Prevention and Control of Spinal and Spinal Cord Injury Diseases of Professional Committee for Prevention and Control of Spinal Diseases of Chinese Preventive Medicine Association. Expert consensus on measurement sites and annotation of artificial intelligence-based spinal degenerative imaging(2025) [J]. Journal of Shandong University (Health Sciences), 2026, 64(2): 1-10.
[3] JI Xinyu, YU Siyi, SUN Yuanyuan, JI Bing. Orthopedic disease diagnosis and treatment assistance methods based on artificial intelligence and gait analysis [J]. Journal of Shandong University (Health Sciences), 2026, 64(2): 34-43.
[4] WANG Jianmin, LI Xiaofeng, YOU Zhitao, DONG Shengjie, ZHAO Yuchi, LI Zhanju, ZOU Dexin, ZHANG Jianfeng, SUN Tao, DU Wei. Construction of a chronic post-surgical pain prediction model for posterior lumbar interbody fusion surgery based on interpretable machine learning [J]. Journal of Shandong University (Health Sciences), 2026, 64(2): 78-88.
[5] ZHANG Xinru, LI Yang, SUN Meng, NIE Wei, MA Zhe. Application and evaluation of Vision-LSTM model in diagnostic ultrasound imaging of Thyroid Imaging Reporting and Data System Category 4b thyroid nodules [J]. Journal of Shandong University (Health Sciences), 2025, 63(11): 68-74.
[6] WU Qiqi, CHENG Miaomiao, XIAO Xiaoyan. Multimodal models in the field of kidney disease [J]. Journal of Shandong University (Health Sciences), 2025, 63(10): 117-124.
[7] LIANG Bowen, LU Qingsheng. Advances in robotic-assisted endovascular aortic repair [J]. Journal of Shandong University (Health Sciences), 2024, 62(9): 61-65.
[8] TANG Yuning, PAN Tianyue, DONG Zhihui, FU Weiguo. Research progress of deep learning in automatic segmentation of aortic images [J]. Journal of Shandong University (Health Sciences), 2024, 62(9): 66-73.
[9] ZHANG Jinghui, WANG Juan, ZHAO Yujie, DUAN Miao, LIU Yiran, LIN Minjuan, QIAO Xu, LI Zhen, ZUO Xiuli. Construction of a machine learning-based tongue diagnosis model for gastrointestinal diseases [J]. Journal of Shandong University (Health Sciences), 2024, 62(1): 38-47.
[10] Shiqing FENG. Computer vision and lumbar degenerative disease [J]. Journal of Shandong University (Health Sciences), 2023, 61(3): 1-6.
[11] Nan WU,Jianguo ZHANG,Yuanpeng ZHU,Guilin CHEN,Zefu CHEN. Application of artificial intelligence in the diagnosis and treatment of spinal deformity [J]. Journal of Shandong University (Health Sciences), 2023, 61(3): 14-20.
[12] Lin HUANG,Zhen CHE,Ming LI,Yuxi LI,Qing NING. Research advances of artificial intelligence in the diagnosis and treatment of orthopaedic diseases [J]. Journal of Shandong University (Health Sciences), 2023, 61(3): 37-45.
[13] WANG Hui, WANG Lianlei, WU Tianchi, TIAN Yonghao, YUAN Suomao, WANG Xia, LYU Weijia, LIU Xinyu. Artificial intelligence-assisted 3D printing of surgical guides for pedicle screw Insertion in scoliosis surgeries [J]. Journal of Shandong University (Health Sciences), 2023, 61(3): 127-133.
[14] Pei NIE,Ximing WANG. Research progress in the application of artificial intelligence in myocardial imaging [J]. Journal of Shandong University (Health Sciences), 2023, 61(12): 1-6.
[15] Ziliang XU,Minwen ZHENG. Innovation and challenge of imaging artificial intelligence in medical field [J]. Journal of Shandong University (Health Sciences), 2023, 61(12): 7-12, 20.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of Journal of Shandong University (Health Sciences)
Supported by Beijing Magtech Co.Ltd