Deep Learning in Retinal Image Analysis: A Review
Keywords:Retinal Imaging, Retinal Image Analysis, Computer aided diagnosis, Systemic Review
Retinal image analysis holds an imperative position for the identification and classification of retinal diseases such as Diabetic Retinopathy, Hypertensive Retinopathy, Glacuma, Age Related Macular Degeneration, Retinal Detachment, and other retinal disease. Automated identification of retinal diseases is a big step towards early diagnosis and prevention of exacerbation of the disease. A number of state-of-the-art methods have been developed in the past that helped in the automatic segmentation and identification of retinal landmarks and pathology. However, the current unprecedented advancements in deep learning and modern imaging modalities in ophthalmology have opened a whole new arena for researchers. This paper is a review of deep learning techniques applied to 2-D fundus and 3D-OCT retinal images for automated identification of retinal landmarks, pathology, and disease classification. The methodologies are analyzed in terms of sensitivity, specificity, Area under ROC curve, accuracy, and F score on publically available datasets which include DRIVE, STARE, CHASE-DB1, DRiDB, NIH AREDS, ARIA, MESSIDOR-2, E-OPTHA, EyePACS-1 DIARETDB and OCT image datasets.
(2) Abràmoff, M.D., M.K. Garvin, and M. Sonka, Retinal imaging and image analysis. IEEE reviews in biomedical engineering, 2010. 3: p. 169-208.
(3) Abdullah, M. and M.M. Fraz. Application of grow cut algorithm for localization and extraction of optic disc in retinal images. in 2015 12th International Conference on High-capacity Optical Networks and Enabling/Emerging Technologies (HONET). 2015. IEEE.
(4) Fraz, M.M., et al. Automated Arteriole and Venule Recognition in Retinal Images using Ensemble Classification. in 9th International Conference on Computer Vision Theory and Applications (VISAAP). 2014. Lisbon, Portugal.
(5) Fraz, M., et al., Computational methods for exudates detection and macular edema estimation in retinal images: a survey. Archives of Computational Methods in Engineering, 2018: p. 1-28.
(6) Fraz, M.M., et al., Multiscale segmentation of exudates in retinal images using contextual cues and ensemble classification. Biomedical Signal Processing and Control, 2017. 35: p. 50-62.
(7) Basit, A. and M.M. Fraz, Optic disc detection and boundary extraction in retinal images. Applied Optics, 2015. 54(11): p. 3440-3447.
(8) Arunkumar, R. and P. Karthigaikumar, Multi-retinal disease classification by reduced deep learning features. Neural Computing and Applications, 2017. 28(2): p. 329-334.
(9) Sidibé, D., I. Sadek, and F. Mériaudeau, Discrimination of retinal images containing bright lesions using sparse coded features and SVM. Computers in biology and medicine, 2015. 62: p. 175-184.
(10) Sadek, I., D. Sidibé, and F. Meriaudeau. Automatic discrimination of color retinal images using the bag of words approach. in SPIE Medical Imaging. 2015. International Society for Optics and Photonics.
(11) Veras, R., et al. SURF descriptor and pattern recognition techniques in automatic identification of pathological retinas. in Intelligent Systems (BRACIS), 2015 Brazilian Conference on. 2015. IEEE.
(12) Lahiri, A., et al. Deep neural ensemble for retinal vessel segmentation in fundus images towards achieving label-free angiography. in Engineering in Medicine and Biology Society (EMBC), 2016 IEEE 38th Annual International Conference of the. 2016. IEEE.
(13) Badar, M., M. Shahzad, and M.M. Fraz. Simultaneous Segmentation of Multiple Retinal Pathologies Using Fully Convolutional Deep Neural Network. 2018. Cham: Springer International Publishing.
(14) Zahoor, M.N. and M.M. Fraz, Fast Optic Disc Segmentation in Retina Using Polar Transform. IEEE Access, 2017. 5: p. 7.
(15) Patton, N., et al., Retinal image analysis: concepts, applications and potential. Progress in retinal and eye research, 2006. 25(1): p. 99-127.
(16) Albert, D.M. and W.H. Miller, Jan Purkinje and the ophthalmoscope. American journal of ophthalmology, 1973. 76(4): p. 494-499.
(17) Fraz, M.M., et al., QUARTZ: Quantitative Analysis of Retinal Vessel Topology and size – An automated system for quantification of retinal vessels morphology. Expert Systems with Applications, 2015. 42(20): p. 7221-7234.
(18) Huang, D., et al., Optical coherence tomography. Science (New York, NY), 1991. 254(5035): p. 1178.
(19) 19. Bennett, T.J. and C.J. Barry, Ophthalmic imaging today: an ophthalmic photographer's viewpoint–a review. Clinical & experimental ophthalmology, 2009. 37(1): p. 2-13.
(20) Venkatesh, P., et al., Detection of retinal lesions in diabetic retinopathy: comparative evaluation of 7-field digital color photography versus red-free photography. International ophthalmology, 2015. 35(5): p. 635-640.
(21) Tyler, M.E., Stereo fundus photography: Principles and technique. J Ophthalmic Photogr, 1996. 18(2): p. 6-81.
(22) Hirohara, Y., et al., Validity of retinal oxygen saturation analysis: Hyperspectral imaging in visible wavelength with fundus camera and liquid crystal wavelength tunable filter. Optical review, 2007. 14(3): p. 151.
(23) Alabboud, I., et al. New spectral imaging techniques for blood oximetry in the retina. in European Conference on Biomedical Optics. 2007. Optical Society of America.
(24) Webb, R.H. and G.W. Hughes, Scanning laser ophthalmoscope. IEEE Transactions on Biomedical Engineering, 1981(7): p. 488-492.
(25) Roorda, A., et al., Adaptive optics scanning laser ophthalmoscopy. Optics express, 2002. 10(9): p. 405-412.
(26) Ng, E., et al., Ophthalmological Imaging and Applications. 2014: CRC Press.
(27) Slakter, J.S., et al., Indocyanine-green angiography. Current opinion in ophthalmology, 1995. 6(3): p. 25-32.
(28) Niemeijer, M., et al., DRIVE: digital retinal images for vessel extraction. Methods for Evaluating Segmentation and Indexing Techniques Dedicated to Retinal Ophthalmology, 2004.
(29) Hoover, A., V. Kouznetsova, and M. Goldbaum, Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Transactions on Medical imaging, 2000. 19(3): p. 203-210.
(30) ARIA Online, Retinal Image Archive.http://www.eyecharity.com/aria online/, 2006.
(31) Fraz, M.M., et al., An Ensemble Classification-Based Approach Applied to Retinal Blood Vessel Segmentation. IEEE Transactions on Biomedical Engineering, 2012. 59(9): p. 2538-2548.
(32) Quellec, G., et al., Optimal wavelet transform for the detection of microaneurysms in retina photographs. IEEE Transactions on Medical Imaging, 2008. 27(9): p. 1230-1241.
(33) Decencière, E., et al., Feedback on a publicly distributed image database: the Messidor database. Image Analysis & Stereology, 2014. 33(3): p. 231-234.
(34) Decencière, E., et al., TeleOphta: Machine learning and image processing methods for teleophthalmology. Irbm, 2013. 34(2): p. 196-203.
(35) Prentašić, P., et al. Diabetic retinopathy image database(DRiDB): A new database for diabetic retinopathy screening programs research. in 2013 8th International Symposium on Image and Signal Processing and Analysis (ISPA). 2013.
(36) Drexler, W. and J.G. Fujimoto, Optical coherence tomography: technology and applications. 2015: Springer.
(37) Jakobiec, F.A., Ocular anatomy, embryology, and teratology. 1982: Harpercollins.
(38) Welikala, R.A., et al., Automated retinal image quality assessment on the UK Biobank dataset for epidemiological studies. Computers in biology and medicine, 2016. 71: p. 67-76.
(39) Owen, C.G., et al., Retinal Vasculometry Associations with Cardiometabolic Risk Factors in the European Prospective Investigation of Cancer—Norfolk Study. Ophthalmology, 2018.
(40) Beagley, J., et al., Global estimates of undiagnosed diabetes in adults. Diabetes research and clinical practice, 2014. 103(2): p. 150-160.
(41) Raman, V., P. Then, and P. Sumari. Proposed retinal abnormality detection and classification approach: Computer aided detection for diabetic retinopathy by machine learning approaches. in Communication Software and Networks (ICCSN), 2016 8th IEEE International Conference on. 2016. IEEE.
(42) Welikala, R.A., et al., Genetic algorithm based feature selection combined with dual classification for the automated detection of proliferative diabetic retinopathy. Computerized Medical Imaging and Graphics, 2015. 43(0): p. 64-77.
(43) Engerman, R.L., Pathogenesis of diabetic retinopathy. Diabetes, 1989. 38(10): p. 1203-1206.
(44) Kowluru, R.A. and P.-S. Chan, Capillary dropout in diabetic retinopathy, in Diabetic Retinopathy. 2008, Springer. p. 265-282.
(45) Kanski, J.J. and B. Bowling, Clinical ophthalmology: a systematic approach. 2011: Elsevier Health Sciences.
(46) Klein, R., et al., The Wisconsin Epidemiologic Study of Diabetic Retinopathy XXIII: the twenty-five-year incidence of macular edema in persons with type 1
diabetes. Ophthalmology, 2009. 116(3): p. 497-503.
(47) Varma, R., et al., Prevalence of and risk factors for diabetic macular edema in the United States. JAMA ophthalmology, 2014. 132(11): p. 1334-1340.
(48) Jager, R.D., W.F. Mieler, and J.W. Miller, Age-related macular degeneration. New England Journal of Medicine, 2008. 358(24): p. 2606-2617.
(49) Alfaro, D.V., Age-related macular degeneration: a comprehensive textbook. 2006: Lippincott Williams & Wilkins.
(50) Fraz, M.M., et al. Retinal vessel segmentation using ensemble classifier of bagged decision trees. in IET Conference Publications. 2012. IEE.
(51) Fraz, M.M., et al., Delineation of blood vessels in pediatric retinal images using decision trees-based ensemble classification. International Journal of Computer
Assisted Radiology and Surgery, 2014. 9(5): p. 795-811.
(52) Liew, G. and J.J. Wang, Retinal vascular signs: a window to the heart? Revista Española de Cardiología (English Edition), 2011. 64(6): p. 515-521.
(53) Quigley, H.A. and A.T. Broman, The number of people with glaucoma worldwide in 2010 and 2020. British journal of ophthalmology, 2006. 90(3): p. 262-267.
(54) Zahoor, M.N. and M.M. Fraz, A Correction to the Article “Fast Optic Disc Segmentation in Retina Using Polar Transform”. IEEE Access, 2018. 6: p. 4845-4849.
(55) Fraz, M.M., et al., Delineation of blood vessels in pediatric retinal images using decision trees-based ensemble classification. International Journal of Computer Assisted Radiology and Surgery, 2014. 9(5): p. 795-811.
(56) Thomas, R. and R.S. Parikh, How to assess a patient for glaucoma. Community eye health, 2006. 19(59): p. 36.
(57) Goodfellow, I., Y. Bengio, and A. Courville, Deep learning. 2016: MIT press.
(58) Hsu, F.-H., Behind Deep Blue: Building the computer that defeated the world chess champion. 2002: Princeton University Press.
(59) LeCun, Y., Y. Bengio, and G. Hinton, Deep learning. Nature, 2015. 521(7553): p. 436-444.
(60) Angelov, P. and A. Sperduti. Challenges in deep learning. in Proceedings of the 24th European symposium on artificial neural networks (ESANN). 2016.
(61) Wang, S., et al., Hierarchical retinal blood vessel segmentation based on feature and ensemble learning. Neurocomputing, 2015. 149: p. 708-717.
(62) Fang, T., et al. Retinal vessel landmark detection using deep learning and hessian matrix. in Image and Signal Processing (CISP), 2015 8th International
Congress on. 2015. IEEE.
(63) Maji, D., et al. Deep neural network and random forest hybrid architecture for learning to detect retinal vessels in fundus images. in Engineering in Medicine and Biology Society (EMBC), 2015 37th Annual International Conference of the IEEE. 2015. IEEE.
(64) Melinščak, M., P. Prentašić, and S. Lončarić. Retinal vessel segmentation using deep neural networks. in VISAPP 2015 (10th International Conference on Computer Vision Theory and Applications). 2015.
(65) Fu, H., et al. Retinal vessel segmentation via deep learning network and fully-connected conditional random fields. in Biomedical Imaging (ISBI), 2016 IEEE 13th International Symposium on. 2016. IEEE.
(66) Fu, H., et al. DeepVessel: Retinal Vessel Segmentation via Deep Learning and Conditional Random Field. in International Conference on Medical Image Computing and Computer-Assisted Intervention. 2016. Springer.
(67) Liskowski, P. and K. Krawiec, Segmenting Retinal Blood Vessels With<? Pub _newline?> Deep Neural Networks. IEEE transactions on medical imaging, 2016. 35(11): p. 2369-2380.
(68) Li, Q., et al., A cross-modality learning approach for vessel segmentation in retinal images. IEEE transactions on medical imaging, 2016. 35(1): p. 109-118.
(69) Yao, Z., Z. Zhang, and L.-Q. Xu. Convolutional Neural Network for Retinal Blood Vessel Segmentation. in Computational Intelligence and Design (ISCID), 2016 9th International Symposium on. 2016. IEEE.
(70) Dasgupta, A. and S. Singh, A Fully Convolutional Neural Network based Structured Prediction Approach Towards the Retinal Vessel Segmentation. arXiv preprint arXiv:1611.02064, 2016.
(71) Maninis, K.-K., et al. Deep retinal image understanding. in International Conference on Medical Image Computing and Computer-Assisted Intervention. 2016. Springer.
(72) Tan, J.H., et al., Segmentation of optic disc, fovea and retinal vasculature using a single convolutional neural network. Journal of Computational Science, 2017.
(73) Prentašić, P. and S. Lončarić. Detection of exudates in fundus photographs using convolutional neural networks. in Image and Signal Processing and Analysis (ISPA), 2015 9th International Symposium on. 2015. IEEE.
(74) Prentašić, P. and S. Lončarić, Detection of exudates in fundus photographs using deep neural networks and anatomical landmark detection fusion. Computer Methods and Programs in Biomedicine, 2016. 137: p. 281-292.
(75) Shan, J. and L. Li. A Deep Learning Method for Microaneurysm Detection in Fundus Images. in Connected Health: Applications, Systems and Engineering Technologies (CHASE), 2016 IEEE First International Conference on. 2016. IEEE.
(76) Pratt, H., et al., Convolutional Neural Networks for Diabetic Retinopathy. Procedia Computer Science, 2016. 90: p. 200-205.
(77) Abràmoff, M.D., et al., Improved Automated Detection of Diabetic Retinopathy on a Publicly Available Dataset Through Integration of Deep LearningDeep Learning Detection of Diabetic Retinopathy. Investigative Ophthalmology & Visual Science, 2016. 57(13): p. 5200-5206.
(78) Gulshan, V., et al., Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA, 2016.
316(22): p. 2402-2410.
(79) Colas, E., et al., Deep learning approach for diabetic retinopathy screening. Acta Ophthalmologica, 2016. 94(S256).
(80) Gargeya, R. and T. Leng, Automated Identification of Diabetic Retinopathy Using Deep Learning. Ophthalmology, 2017.
(81) Burlina, P., et al. Detection of age-related macular degeneration via deep learning. in Biomedical Imaging (ISBI), 2016 IEEE 13th International Symposium on. 2016. IEEE.
(82) Burlina, P., et al., Comparing humans and deep learning performance for grading AMD: A study in using universal deep features and transfer learning for automated AMD analysis. Computers in Biology and Medicine, 2017. 82: p. 80-86.
(83) Lee, C.S., D.M. Baughman, and A.Y. Lee, Deep Learning Is Effective for Classifying Normal versus Age-Related Macular Degeneration Optical Coherence Tomography Images. Ophthalmology Retina, 2017.
(84) Xie, S. and Z. Tu. Holistically-nested edge detection. in Proceedings of the IEEE International Conference on Computer Vision. 2015.
(85) Jia, Y., et al. Caffe: Convolutional architecture for fast feature embedding. in Proceedings of the 22nd ACM international conference on Multimedia. 2014. ACM.
(86) Sopharak, A., et al., Automatic detection of diabetic retinopathy exudates from non-dilated retinal images using mathematical morphology methods. Computerized medical imaging and graphics, 2008. 32(8): p. 720-727.
(87) Walter, T. and J.-C. Klein. Segmentation of color fundus images of the human retina: Detection of the optic disc and the vascular tree using morphological techniques. in International Symposium on Medical Data Analysis. 2001. Springer.
(88) Frangi, A.F., et al. Multiscale vessel enhancement filtering. in International Conference on Medical Image Computing and Computer-Assisted Intervention. 1998. Springer.
(89) Szegedy, C., et al. Rethinking the inception architecture for computer vision. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.
(90) Badawi, S.A. and M.M. Fraz, Optimizing the trainable B-COSFIRE filter for retinal blood vessel segmentation. PeerJ, 2018. 6: p. e5855.