Journal of Shandong University (Health Sciences) ›› 2020, Vol. 58 ›› Issue (11): 17-23.doi: 10.6040/j.issn.1671-7554.0.2020.1227

• Special topic on new progress in ophthalmic artificial intelligence • Previous Articles     Next Articles

New developments in ophthalmic AI algorithms

Zongyuan GE*(),Wanji HE,Lie JU,Xuan YAO,Lin WANG,Yelin HUANG,Zhiwen YANG,Jianhao XIONG,Yining BAO,Ming LI,Bing ZHANG,Xin ZHAO   

  1. Monash e-Research Center and Faculty of Engineering, Monash University, Melbourne 3800, Victoria, Australia
  • Received:2020-08-29 Online:2020-11-10 Published:2020-11-04
  • Contact: Zongyuan GE E-mail:zongyuan.ge@monash.edu

RichHTML

52

PDF (PC)

427

Abstract:

Deep neural networks, combined with high quality annotated medical data and low-cost GPU devices, have been successfully implemented in the field of ophthalmology. Impressive grounding outcomes have occurred in both in-hospital and out-hospital scenarios. Some of the published results demonstrated AI could achieve better diagnosis performance than general practice in the diagnosis of certain retinal diseases. In this article, we will discuss how classification, detection, segmentation and domain adaptation play their roles in AI ophthalmology. We will also discuss the limitations of the current state-of-the-art (SOTA) algorithms, hoping to provide prevision for future research in this field.

Key words: Deep neural networks, Retinal disease diagnosis, Artificial intelligence, Computer vision

CLC Number: 

  • R770.4
1 路寻. 卓越的人工智能科学家——马文·明斯基[J]. 自然辩证法通讯, 2010, 32 (2): 104- 111, 128.
2 Govindaiah A, Smith RT, Bhuiyan A. A new and improved method for automated screening of age-related macular degeneration using ensemble deep neural networks[C]. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2018: 702-705.
3 Russakoff DB , Lamin A , Oakley JD , et al. Deep learning for prediction of amd progression: A pilot study[J]. Invest Ophthalmol Vis Sci, 2019, 60 (2): 712- 722.
doi: 10.1167/iovs.18-25325
4 Ting DSW , Cheung CY , Lim G , et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes[J]. JAMA, 2017, 318 (22): 2211- 2223.
doi: 10.1001/jama.2017.18152
5 Brown JM , Campbell JP , Beers A , et al. Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks[J]. JAMA Ophthalmol, 2018, 136 (7): 803- 810.
doi: 10.1001/jamaophthalmol.2018.1934
6 Kesim C , Tas AY , Karslıoglu MZ , et al. Validation results of a deep learning algorithm for detection of diabetic retinopathy with lesion localization from retinal fundus photographs[J]. Invest Ophthalmol Vis Sci, 2020, 62 (7): 1626.
7 Lakra A, Tripathi P, Keshari R, et al. Segdensenet: Iris segmentation for pre and post cataract surgery[J/OL]. arXiv, 2018: 1801.10100[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2018arXiv180110100L.
8 Dong Y, Zhang Q, Qiao Z, et al. Classification of cataract fundus image based on deep learning[C]. Proceedings of IEEE International Conference on Imaging Systems and Techniques, 2017: 1-5.
9 Varadarajan AV , Poplin R , Blumer K , et al. Deep learning for predicting refractive error from retinal fundus images[J]. Invest Ophthalmol Vis Sci, 2018, 59 (7): 2861- 2868.
doi: 10.1167/iovs.18-23887
10 Yau JW , Rogers SL , Kawasaki R , et al. Global prevalence and major risk factors of diabetic retinopathy[J]. Diabetes Care, 2012, 35 (3): 556- 564.
11 Ortiz A , Munilla J , Gorriz JM , et al. Ensembles of deep learning architectures for the early diagnosis of the alzheimer's disease[J]. Int J Neural Syst, 2016, 26 (7): 1650025.
doi: 10.1142/S0129065716500258
12 Gulshan V , Peng L , Coram M , et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs[J]. JAMA, 2016, 316 (22): 2402- 2410.
doi: 10.1001/jama.2016.17216
13 Diaz-Pinto A , Morales S , Naranjo V , et al. Cnns for automatic glaucoma assessment using fundus images: An extensive validation[J]. Biomed Eng Online, 2019, 18 (1): 29.
doi: 10.1186/s12938-019-0649-y
14 Wang X, Ju L, Zhao X, et al. Retinal abnormalities recognition using regional multitask learning[C]. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2019: 30-38.
15 Ruder S. An overview of multi-task learning in deep neural networks[J/OL]. arXiv, 2017: 1706.05098[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2017arXiv170605098R.
16 Ravindran S . How artificial intelligence is helping to prevent blindness[J]. Nature, 2019, 568 (7751): 6.
17 Nair A , Merkel S , Straub J , et al. Real-time pupil detection for fundus imager using single-shot detector and hard- negative training[J]. Invest Ophthalmol Vis Sci, 2020, 61 (9): PB00110- PB00110.
18 Mitra A , Banerjee PS , Roy S , et al. The region of interest localization for glaucoma analysis from retinal fundus image using deep learning[J]. Comput Methods Programs Biomed, 2018, 165 (2018): 25- 35.
19 Selvaraju RR, Das A, Vedantam R, et al. Grad-cam: Why did you say that?[J/OL]. arXiv, 2016: 1611.07450.[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2016arXiv161107450S.
20 Haloi M. Improved microaneurysm detection using deep neural networks[J/OL]. arXiv, 2015: 1505.04424[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2015arXiv-150504424H.
21 Khojasteh P , Passos Junior LA , Carvalho T , et al. Exudate detection in fundus images using deeply-learnable features[J]. Comput Biol Med, 2019, 104 (2019): 62- 69.
22 Mitani A , Huang A , Venugopalan S , et al. Detection of anaemia from retinal fundus images via deep learning[J]. Nat Biomed Eng, 2020, 4 (1): 18- 27.
23 Quellec G , Lamard M , Conze PH , et al. Automatic detection of multiple pathologies in fundus photographs[J]. Invest Ophthalmol Vis Sci, 2020, 61 (7): 1641- 6141.
24 Fang L , Cunefare D , Wang C , et al. Automatic segmentation of nine retinal layer boundaries in oct images of non-exudative amd patients using deep learning and graph search[J]. Biomed Opt Express, 2017, 8 (5): 2732- 2744.
doi: 10.1364/BOE.8.002732
25 De Fauw J , Ledsam JR , Romera-Paredes B , et al. Clinically applicable deep learning for diagnosis and referral in retinal disease[J]. Nat Med, 2018, 24 (9): 1342- 1350.
doi: 10.1038/s41591-018-0107-6
26 Ma W, Yu S, Ma K, et al. Multi-task neural networks with spatial activation for retinal vessel segmentation and artery/vein classification[C]. Proceedings of the Medical Image Computing and Computer Assisted Intervention-MICCAI 2019, Cham, F, 2019: 769-778.
27 Sevastopolsky A . Optic disc and cup segmentation methods for glaucoma detection with modification of u-net convolutional neural network[J]. Pattern Recog& Ima Anal, 2017, 27 (3): 618- 624.
28 Sarhan MH, Albarqouni S, Yigitsoy M, et al. Multi-scale microaneurysms segmentation using embedding triplet loss[C]. Proceedings of the Medical Image Computing and Computer Assisted Intervention-MICCAI 2019, Cham, F, 2019: 174-182.
29 Fu H , Cheng J , Xu Y , et al. Joint optic disc and cup segmentation based on multi-label deep network and polar transformation[J]. IEEE Trans Med Imaging, 2018, 37 (7): 1597- 1605.
doi: 10.1109/TMI.2018.2791488
30 Shelhamer E , Long J , Darrell T . Fully convolutional networks for semantic segmentation[J]. IEEE Trans Pattern Anal Mach Intell, 2017, 39 (4): 640- 651.
doi: 10.1109/TPAMI.2016.2572683
31 Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[J/OL]. arXiv, 2015: 1505.04597[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2015arXiv150504597R.
32 Chen LC, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[J/OL]. arXiv, 2018: 1802.02611[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2018arXiv180202611C.
33 Yan X, Jiang W, Shi Y, et al. Ms-nas: Multi-scale neural architecture search for medical image segmentation[J/OL]. arXiv, 2020: 2007.06151[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2020arXiv200706-151Y.
34 Jiang Y, Tan N. Retinal vessel segmentation based on conditional deep convolutional generative adversarial networks[J/OL]. arXiv, 2018: 1805.04224[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2018arXiv180504-224J.
35 Redko I, Habrard A, Morvant E, et al. 2 - domain adaptation problem[M]// Redko I, Habrard A, Morvant E, et al. Advances in domain adaptation theory. Amsterdam: Elsevier, 2019: 21-36.
36 Ma Y, Lao S, Takikawa E, et al. Discriminant analysis in correlation similarity measure space[C]. Proceedings of the 24th international conference on Machine learning, 2007: 577-584.
37 Bruzzone L , Marconcini M . Domain adaptation problems: A dasvm classification technique and a circular validation strategy[J]. IEEE Trans Pattern Anal Mach Intell, 2010, 32 (5): 770- 787.
doi: 10.1109/TPAMI.2009.57
38 Gheisari M , Baghshah MS . Unsupervised domain adaptation via representation learning and adaptive classifier learning[J]. Neurocomputing, 2015, 165 (OCT.1): 300- 311.
39 Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J/OL]. arXiv, 2015: 1503.02531[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2015arXiv150302531H.
40 Borgwardt KM , Gretton A , Rasch MJ , et al. Integrating structured biological data by kernel maximum mean discrepancy[J]. Bioinformatics, 2006, 22 (14): E49- E57.
doi: 10.1093/bioinformatics/btl242
41 Ghifary M, Bastiaan Kleijn W, Zhang M. Domain adaptive neural networks for object recognition[J/OL]. arXiv, 2014: 1409.6041[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2014ar, Xiv1409.6041G.
42 Li Y , Wang N , Shi J , et al. Adaptive batch normalization for practical domain adaptation[J]. Pattern Recognition, 2018, 80 (2018): 109- 117.
43 Gopalan R, Ruonan L, Chellappa R. Domain adaptation for object recognition: An unsupervised approach[C]. International Conference on Computer, 2011: 999-1006.
44 Liu M-Y, Tuzel O. Coupled generative adversarial networks[J/OL]. arXiv, 2016: 1606.07536[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2016arXiv1606075-36L.
45 Ganin Y, Lempitsky V. Unsupervised domain adaptation by backpropagation[J/OL]. arXiv, 2014: 1409.7495[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2014arXiv1409.7495G.
46 Bousmalis K, Trigeorgis G, Silberman N, et al. Domain separation networks[J/OL]. arXiv, 2016: 1608.06019[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2016arXiv160806019B.
47 Yi Z, Zhang H, Tan P, et al. Dualgan: Unsupervised dual learning for image-to-image translation[J/OL]. arXiv, 2017: 1704.02510[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2017arXiv170402510Y.
48 Zhu J, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C].IEEE International Conference on Computer Vision (ICCV), 2017: 2242-2251.
49 Yang Y, Soatto S. Fda: Fourier domain adaptation for semantic segmentation[J/OL]. arXiv, 2020: 2004.05498[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2020arXiv200405498Y.
50 Johnson JM , Khoshgoftaar TM . Survey on deep learning with class imbalance[J]. Journal of Big Data, 2019, 6 (1): 1- 54.
51 Xiong J, He AW, Fu M, et al. Improve unseen domain generalization via enhanced local color transformation[C]. Proceedings of the Medical Image Computing and Computer Assisted Intervention-MICCAI 2020, Cham, F, 2020: 433-443.
52 Davide C . Can we open the black box of ai?[J]. Nature, 2016, 538 (7623): 20.
doi: 10.1038/538020a
53 Ge Z, Demyanov S, Chen Z, et al. Generative openmax for multi-class open set classification[J/OL]. arXiv, 2017: 1707.07418[2020-08-16]. https://ui.adsabs.harvard.edu/abs/2017arXiv170707418G.
54 Heaven WD. Google's medical ai was super accurate in a lab. Real life was a different story[EB/OL]. (2020-08-16)[2020-08-18]. https://www.technologyreview.com/2020/04/27/1000658/google-medical-ai-accurate-lab-real-life-clinic-covid-diabetes-retina-disease/.
55 Airdoc取得nmpa人工智能医疗器械三类注册证[EB/OL]. (2020-08-16)[2020-08-18]. https://www.sohu.com/a/412357707_102972.
[1] Ju LIU,Qiang WU,Luyue YU,Fengming LIN. Brain tumor image segmentation based on deep learning techniques [J]. Journal of Shandong University (Health Sciences), 2020, 1(8): 42-49, 73.
[2] Xingang LI,Xin ZHANG,Anjing CHEN. The latest advances in human brain projects [J]. Journal of Shandong University (Health Sciences), 2020, 1(8): 5-9, 21.
[3] Jizong ZHAO. Neurosurgery plays a key role in brain science research [J]. Journal of Shandong University (Health Sciences), 2020, 1(8): 1-4.
[4] QU Yi, ZHANG Huankai, SONG Xian, CHU Baorui. Research progress of artificial intelligence diagnosis system in retinal diseases [J]. Journal of Shandong University (Health Sciences), 2020, 58(11): 39-44.
[5] Carol Y. Cheung, RAN Anran. Artificial intelligence deep learning in glaucoma imaging: current progress and future prospect [J]. Journal of Shandong University (Health Sciences), 2020, 58(11): 24-32.
[6] Haotian LIN,Longhui LI,Jingjing CHEN. Research progress of artificial intelligence in childhood eye diseases [J]. Journal of Shandong University (Health Sciences), 2020, 58(11): 11-16.
[7] Mingguang HE,Chi LIU,Zhixi LI. Applying artificial intelligence in ophthalmic real-world practice: opportunities and challenges [J]. Journal of Shandong University (Health Sciences), 2020, 58(11): 1-10.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] SUO Dongyang, SHEN Fei, GUO Hao, LIU Lichang, YANG Huimin, YANG Xiangdong. Expression and mechanism of Tim-3 in animal model of drug-induced acute kidney injury[J]. Journal of Shandong University (Health Sciences), 2020, 1(7): 1 -6 .
[2] MA Qingyuan, PU Peidong, HAN Fei, WANG Chao, ZHU Zhoujun, WANG Weishan, SHI Chenhui. Effect of miR-27b-3p regulating SMAD1 on osteosarcoma cell proliferation, migration and invasion[J]. Journal of Shandong University (Health Sciences), 2020, 1(7): 32 -37 .
[3] LI Ning, LI Juan, XIE Yan, LI Peilong, WANG Yunshan, DU Lutao, WANG Chuanxin. Expression of LncRNA AL109955.1 in 80 cases of colorectal cancer and its effect on cell proliferation, migration and invasion[J]. Journal of Shandong University (Health Sciences), 2020, 1(7): 38 -46 .
[4] XU Yuxiang, LIU Yudong, ZHANG Peng, DUAN Ruisheng. A retrospective analysis of risk factors of cerebral microbleeds in 101 patients with cerebral small vessel disease[J]. Journal of Shandong University (Health Sciences), 2020, 1(7): 67 -71 .
[5] LUO Xin, HE Bing, NIE Qingsheng, HOU Zhenbo, DONG Jun, LI Yuhua, ZENG Xiangqin, LIU Wei, KONG Demin, CAO Jinfeng. Comparison of the value of mono-exponential mode and diffusion kurtosis imaging mode in grading clear cell renal cell carcinoma using magnetic resonance diffusion-weighted imaging[J]. Journal of Shandong University (Health Sciences), 2020, 1(7): 89 -95 .
[6] Di ZHANG,Meng YU,Xia LIU. A general overview of neuromodulation[J]. Journal of Shandong University (Health Sciences), 2020, 1(8): 50 -60 .
[7] Wei ZHANG,Wenhao TAN,Yibin LI. Locmotion control of quadruped robot based on deep reinforcement learning: review and prospect[J]. Journal of Shandong University (Health Sciences), 2020, 1(8): 61 -66 .
[8] Anjing CHEN,Xun ZHANG. The new strategies of targeting SUMOylation in the treatment of glioma[J]. Journal of Shandong University (Health Sciences), 2020, 1(8): 88 -94 .
[9] ZHANG Hongbin, ZHAO Hanhui, WANG Suxia, ZHOU Peng, HE Qingqing, WANG Yanqun, DING Weiping, LIU Gang. Perioperative observation and postoperative risk factors of severe hypocalcemia after parathyroidectomy: a report of 303 cases[J]. Journal of Shandong University (Health Sciences), 2020, 1(9): 14 -20 .
[10] SUN Yifeng, GAO Yu, LIANG Yongyuan, GAO Yang. Expression of CPLX2 and its in vitro effects on the proliferation migration and invasion of hepatocellular carcinoma cells[J]. Journal of Shandong University (Health Sciences), 2020, 1(9): 34 -39 .
Copyright 2018 © Editorial office of Journal of Shandong University (Health Sciences)
Supported by Beijing Magtech Co.Ltd