Journal of Shandong University (Health Sciences) ›› 2020, Vol. 58 ›› Issue (8): 42-49, 73.doi: 10.6040/j.issn.1671-7554.0.2020.0391

• Special Topic on Brain Science and Brain Like Intelligence • Previous Articles     Next Articles

Brain tumor image segmentation based on deep learning techniques

Ju LIU1,2,*(),Qiang WU1,2,Luyue YU1,Fengming LIN1   

  1. 1. School of Information Science and Engineering, Shandong University, Qingdao 266237, Shandong, China
    2. Institute of Brain and Brain-Inspired Science, Shandong University, Jinan 250012, Shandong, China
  • Received:2020-03-20 Online:2020-08-07 Published:2020-08-07
  • Contact: Ju LIU E-mail:juliu@sdu.edu.cn

RichHTML

109

PDF (PC)

950

Abstract:

Artificial intelligence technology is widely applied in the field of computer vision and deep learning. Image segmentation technology based on deep learning is essential in industries such as autonomous driving, drones, and clinical diagnosis and treatment. This paper reviews the methods of brain tumor image segmentation, including the traditional methods of image segmentation and methods based on deep learning and some typical methods. The paper also compares our research advances with the typical methods and discusses future research directions and challenges.

Key words: Artificial intelligence, Deep learning, Image segmentation, Brain tumor image, Neural network

CLC Number: 

  • R574

Fig.1

Structure of dilated convolution networks [13]"

Fig.2

Structure of FPN[15]"

Fig.3

Modular structure of DANet [17]"

Fig.4

Generative adversarial network structure used in segmentation [24]"

Fig.5

ShuffleSeg network architecture[28]"

Fig.6

U-Net for BraTS2017"

Fig.7

Architecture of Attention R2U-Net[33]"

Fig.8

Schematic visualization of the network architecture[35]"

Table 1

Comparison with state-of-the-arts on the testing set of BRATS2018 [40]"

BraTS2018Dice
WT TC ET
VGG 0.872 4 0.693 3 0.659 8
DUNet 0.863 1 0.688 4 0.620 5
FCNN 0.868 7 0.653 7 0.582 8
HPUNet 0.893 0 0.810 8 0.755 8
baseline(HPUNet+ResNet) 0.892 5 0.781 1 0.746 3
baseline+FCU 0.895 4 0.809 1 0.778 3
baseline+FCU+MIMU 0.895 6 0.831 3 0.806 7
baseline+FCU+MIMU+SIMU 0.901 9 0.837 4 0.817 6
1 李伟峰. 一种新的PSO优化FCM方法在图像分类中的应用[J]. 软件导刊, 2013, 12 (8): 72- 75.
LI Weifeng . Application of web data ming on accurate prediction of susceptible populations of acute mountain sickness[J]. Software Guide, 2013, 12 (8): 72- 75.
2 张长水. 机器学习面临的挑战[J]. 中国科学:信息科学, 2013, 43 (12): 1612- 1623.
3 林正春, 王知衍, 张艳青. 最优进化图像阈值分割算法[J]. 计算机辅助设计与图形学学报, 2010, 22 (7): 1201- 1206.
LIN Zhengchun , WANG Zhiyan , ZHANG Yanqing . Optimal evolution algorithm for image thresholding[J]. Journal of Computer-aided Design & Computer Graphics, 2010, 22 (7): 1201- 1206.
4 陆剑锋, 林海, 潘志庚. 自适应区域生长算法在医学图像分割中的应用[J]. 计算机辅助设计与图形学学报, 2005, 17 (10): 2168- 2173.
LU Jianfeng , LIN Hai , PAN Zhigeng . Adaptive region growing algorithm in medical images segmentation[J]. Journal of Computer-aided Design & Computer Graphics, 2005, 17 (10): 2168- 2173.
5 Boykov YY, Jolly MP. Interactive graph cuts for optimal boundary & region segmentation of objects in ND images [C]. Proceedings 8th IEEE International Conference on Computer Vision, 2001: 105-112.
6 Wang Y , Loe KF , Wu JK . A dynamic conditional random field model for foreground and shadow segmentation[J]. IEEE Trans Pattern Anal Mach Intell, 2006, 28 (2): 279- 289.
doi: 10.1109/TPAMI.2006.25
7 Masooleh MG , Ali S , Moosavi S . An improved fuzzy algorithm for image segmentation[J]. World Academy of Science, Engineering and Technology, 2008, 1 (40): 400- 404.
8 Mostajabi M, Yadollahpour P, Shakhnarovich G. Feedforward semantic segmentation with zoom-out features [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 3376-3385.
9 Shelhamer E , Long J , Darrell T . Fully convolutional networks for semantic segmentation[J]. IEEE Trans Pattern Anal Mach Intell, 2015, 39 (4): 640- 651.
10 Badrinarayanan V , Kendall A , Cipolla R . SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Trans Pattern Anal Mach Intell, 2017, 39 (12): 2481- 2495.
doi: 10.1109/TPAMI.2016.2644615
11 Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation [C]. Proceedings of the IEEE International Conference on Computer Vision, 2015: 1520-1528.
12 Chen LC , Papandreou G , Kokkinos I , et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Trans Pattern Anal Mach Intell, 2018, 40 (4): 834- 848.
doi: 10.1109/TPAMI.2017.2699184
13 Chen LC, Papandreou G, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation [EB/OL]. (2017-12-5) [2020-3-19]. https://arxiv.org/abs/1706.05587.
14 Liu W, Rabinovich A, Berg AC. ParseNet: Looking Wider to See Better [EB/OL]. (2015-11-19) [2020-3-19]. https://arxiv.org/abs/1506.04579.
15 Ghiasi G, Fowlkes CC. Laplacian pyramid reconstruction and refinement for semantic segmentation [C]. European Conference on Computer Vision, Springer, Cham, 2016: 519-534.
16 Lin TY, Dollár P, Girshick R, et al. Feature pyramid networks for object detection [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2117-2125.
17 Fu J, Liu J, Tian H, et al. Dual attention network for scene segmentation [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019: 3146-3154.
18 Hong S, Noh H, Han B. Decoupled deep neural network for semi-supervised semantic segmentation [C]. Advances in Neural Information Processing Systems, 2015: 1495-1503.
19 Kolesnikov A, Lampert CH. Seed, expand and constrain: Three principles for weakly-supervised image segmentation [C]. European Conference on Computer Vision, Springer, Cham, 2016: 695-711.
20 Zhou B, Khosla A, Lapedriza A, et al. Learning deep features for discriminative localization [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 2921-2929.
21 Lin D, Dai J, Jia J, et al. Scribblesup: Scribble-supervised convolutional networks for semantic segmentation [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 3159-3167.
22 Cheplygina V , de BM , Pluim JPW . Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis[J]. MedIA, 2019, 54 (1): 280- 296.
doi: 10.1016/j.media.2019.03.009
23 Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets [C]. Advances in Neural Information Processing Systems, 2014: 2672-2680.
24 Luc P, Couprie C, Chintala S, et al. Semantic Segmentation using Adversarial Networks [EB/OL]. (2016-11-25) [2020-3-19]. https://arxiv.org/abs/1611.08408.
25 Hoffman J, Wang D, Yu F, et al. FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation [EB/OL]. (2016-12-8) [2020-3-19]. https://arxiv.org/abs/1612.02649.
26 Souly N, Spampinato C, Shah M. Semi supervised semantic segmentation using generative adversarial network [C]. 2017 IEEE International Conference on Computer Vision, 2017: 5689-5697.
27 Tsai YH, Hung WC, Schulter S, et al. Learning to adapt structured output space for semantic segmentation [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7472-7481.
28 Gamal M, Siam M, Abdel-Razek M. ShuffleSeg: Real-time Semantic Segmentation Network [EB/OL]. (2018-3-15) [2020-3-19]. https://arxiv.org/abs/1803.03816.
29 田娟秀, 刘国才, 谷珊珊, 等. 医学图像分析深度学习方法研究与挑战[J]. 自动化学报, 2018, 44 (3): 401- 424.
TIAN Xiujuan , LIU Guocai , GU Shanshan , et al. Deep learning in medical image analysis and its challenges[J]. Acta Automatica Sinica, 2018, 44 (3): 401- 424.
30 Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation [C]. International Conference on Medical Image Computing and Computer-assisted Intervention, Springer, Cham, 2015: 234-241.
31 He K, Zhang X, Ren S, et al. Deep residual learning for image recognition [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
32 Huang G, Liu Z, Van DML, et al. Densely connected convolutional networks [C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4700-4708.
33 Alom MZ , Yakopcic C , Hasan M , et al. Recurrent residual U-Net for medical image segmentation[J]. Journal of Medical Imaging, 2019, 6 (1): 345- 350.
doi: 10.1117/1.JMI.6.1.014006
34 Oktay O, Schlemper J, Folgoc LL, et al. Attention u-net: Learning where to look for the pancreas[EB/OL]. (2018-5-20) [2020-3-19]. https://arxiv.org/abs/1804.03999.
35 Myronenko A. 3D MRI brain tumor segmentation using autoencoder regularization[C]. International MICCAI Brainlesion Workshop, Springer, Cham, 2018: 311-320.
36 Zhou C, Chen S, Ding C, et al. Learning contextual and attentive information for brain tumor segmentation[C]. International MICCAI Brainlesion Workshop, Springer, Cham, 2018: 497-507.
37 McKinley R, Meier R, Wiest R. Ensembles of densely-connected CNNs with label-uncertainty for brain tumor segmentation[C]. International MICCAI Brainlesion Workshop, Springer, Cham, 2018: 456-465.
38 Kong X, Sun G, Wu Q, et al. Hybrid pyramid U-Net model for brain tumor segmentation[C]. International Conference on Intelligent Information Processing, Springer, Cham, 2018: 346-355.
39 Lin F, Wu Q, Liu J, et al. Path aggregation U-Net model for brain tumor segmentation[EB/OL]. (2020-03-19) [2020-03-19]. https://doi.org/10.1007/s11042-020-08795-9.
40 Lin F, Liu J, Wu Q, et al. FMNet: Feature mining networks for brain tumor segmentation[C]. IEEE 31st International Conference on Tools with Artificial Intelligence, IEEE Computer Society, 2019: 555-560.
[1] Jizong ZHAO. Neurosurgery plays a key role in brain science research [J]. Journal of Shandong University (Health Sciences), 2020, 58(8): 1-4.
[2] Xingang LI,Xin ZHANG,Anjing CHEN. The latest advances in human brain projects [J]. Journal of Shandong University (Health Sciences), 2020, 58(8): 5-9, 21.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] SUO Dongyang, SHEN Fei, GUO Hao, LIU Lichang, YANG Huimin, YANG Xiangdong. Expression and mechanism of Tim-3 in animal model of drug-induced acute kidney injury[J]. Journal of Shandong University (Health Sciences), 2020, 58(7): 1 -6 .
[2] ZHANG Baowen, LEI Xiangli, LI Jinna, LUO Xiangjun, ZOU Rong. miR-21-5p targeted TIMP3 to inhibit proliferation and extracellular matrix accumulation of mesangial cells in Type II diabetic nephropathy mice[J]. Journal of Shandong University (Health Sciences), 2020, 58(7): 7 -14 .
[3] LONG Tingting, XIE Ming, ZHOU Lu, ZHU Junde. Effect of Noggin protein on learning and memory abilities and the dentate gyrus structure after cerebral ischemia reperfusion injury in mice[J]. Journal of Shandong University (Health Sciences), 2020, 58(7): 15 -23 .
[4] FU Jieqi, ZHANG Man, ZHANG Xiaolu, LI Hui, CHEN Hong. Molecular mechanism of Toll-like receptor 4 in the aggravation of blood lipid accumulation by inhibiting the peroxisome proliferator-activate receptor γ[J]. Journal of Shandong University (Health Sciences), 2020, 58(7): 24 -31 .
[5] MA Qingyuan, PU Peidong, HAN Fei, WANG Chao, ZHU Zhoujun, WANG Weishan, SHI Chenhui. Effect of miR-27b-3p regulating SMAD1 on osteosarcoma cell proliferation, migration and invasion[J]. Journal of Shandong University (Health Sciences), 2020, 58(7): 32 -37 .
[6] LI Ning, LI Juan, XIE Yan, LI Peilong, WANG Yunshan, DU Lutao, WANG Chuanxin. Expression of LncRNA AL109955.1 in 80 cases of colorectal cancer and its effect on cell proliferation, migration and invasion[J]. Journal of Shandong University (Health Sciences), 2020, 58(7): 38 -46 .
[7] SHI Shuang, LI Juan, MI Qi, WANG Yunshan, DU Lutao, WANG Chuanxin. Construction and application of a miRNAs prognostic risk assessment model of gastric cancer[J]. Journal of Shandong University (Health Sciences), 2020, 58(7): 47 -52 .
[8] XIAO Juan, XIAO Qiang, CONG Wei, LI Ting, DING Shouluan, ZHANG Yuan, SHAO Chunchun, WU Mei, LIU Jianing, JIA Hongying. Comparison of diagnostic efficacy of two kinds of thyroid imagine reporting and data systems[J]. Journal of Shandong University (Health Sciences), 2020, 58(7): 53 -59 .
[9] DING Xiangyun, YU Qingmei, ZHANG Wenfang, ZHUANG Yuan, HAO Jing. Correlation of the expression of insulin-like growth factor II in granulosa cells and ovulation induction outcomes of 84 patients with polycystic ovary syndrome[J]. Journal of Shandong University (Health Sciences), 2020, 58(7): 60 -66 .
[10] XU Yuxiang, LIU Yudong, ZHANG Peng, DUAN Ruisheng. A retrospective analysis of risk factors of cerebral microbleeds in 101 patients with cerebral small vessel disease[J]. Journal of Shandong University (Health Sciences), 2020, 58(7): 67 -71 .
Copyright 2018 © Editorial office of Journal of Shandong University (Health Sciences)
Supported by Beijing Magtech Co.Ltd