Fungal Microscopic Image Classification Based on Multi-scale Attention Mechanism

ZHANG Xueyuan; XU Hongyan; DONG Yueming; LIU Danfeng; SUN Pengrui; YAN Rui; CUI Hongliang; LEI Hong; REN Fei

doi:10.12290/xhyxzz.2022-0169

Volume 14 Issue 1

Jan. 2023

Turn off MathJax

Article Contents

Abstract

References

Medical Journal of Peking Union Medical College Hospital > 2023 > 14(1): 139-147. > DOI: 10.12290/xhyxzz.2022-0169

ZHANG Xueyuan, XU Hongyan, DONG Yueming, LIU Danfeng, SUN Pengrui, YAN Rui, CUI Hongliang, LEI Hong, REN Fei. Fungal Microscopic Image Classification Based on Multi-scale Attention Mechanism[J]. Medical Journal of Peking Union Medical College Hospital, 2023, 14(1): 139-147. DOI: 10.12290/xhyxzz.2022-0169

Citation:

PDF (5069 KB)

Fungal Microscopic Image Classification Based on Multi-scale Attention Mechanism

1.
Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China
2.
Beijing Zhijian Life Technology Co., Ltd., Beijing 100036, China
3.
Eighth Medical Center of PLA General Hospital, Beijing 100091, China

Funds:

National Key R & D Program 2021YFF1201005

Medical Big Data and Artificial Intelligence R & D Project of PLA General Hospital 2019MBD-050

More Information

Corresponding author:
LEI Hong, E-mail: jlh309@163.com

REN Fei, E-mail: renfei@ict.ac.cn
Received Date: March 30, 2022
Accepted Date: May 25, 2022
Available Online: September 19, 2022
Issue Publish Date: January 29, 2023

Graphical Abstract

Abstract

Abstract

Objective To establish a fungal image-assisted classification model using deep learning technology.
Methods The microscope images of people infected with Aspergillus, Saccharomyces and Cryptococcus neoformans were retrospectively collected from the Eighth Medical Center of PLA General Hospital from September 2020 to April 2021. The images were randomly divided into training set, validation set and test set according to the ratio of 7∶1.5∶1.5. The improved MobileNetV2 network structure was trained using the training set, a convolutional neural network (CNN) fungal image 11 classification model based on multi-scale attention mechanism was constructed and the parameters were debugged based on the validation set. Machine identification results were taken as the gold standard, the performance of the model on 11 fungal image classification tasks was evaluated, and the results were shown by precision, recall and F1 value. In addition, the performance of the proposed model with 5 classic CNN models were compared, and the results were measured in terms of model parameters, memory usage, frames per second (FPS), accuracy, and area under the curve (AUC) of receiver operating characteristic curve.
Results A total of 7666 fungal microscope images were collected, including 2781, 4115, and 770 images of Aspergillus, Saccharomyces, and Cryptococcus neoformans, respectively. Among them, there were 5366 training images, 1150 validation images, and 1150 test images. The improved MobileNetV2 model had high performance for the classification of 11 fungal images in the test set. The precision rate was distributed between 96.36% and 100%, the recall rate was distributed between 96.53% and 100%, and the F1 value was distributed between 97.01% and 100%. The parameters, memory usage, FPS, accuracy, and AUC of the improved MobileNetV2 model were 4.22 M, 356.89 M, 573, (99.09±0.18)%, and 0.9944±0.0018, respectively, and the comprehensive performance was better than 5 kinds of classical networks.
Conclusion The proposed fungal image classification model based on the improved MobileNetV2 can obtain higher fungal image recognition ability while maintaining low computational cost, with an overall performance better than classical CNN model.
- fungal,
- convolutional neural network,
- image classification,
- multi-scale attention mechanism

FullText(HTML)

References (29)

References

[1]	Chen M, Xu Y, Hong N, et al. Epidemiology of fungal infections in China[J]. Front Med, 2018, 12: 58-75. DOI: 10.1007/s11684-017-0601-0
[2]	Sanguinetti M, Posteraro B, Beigelman-Aubry C, et al. Diagnosis and treatment of invasive fungal infections: looking ahead[J]. J Antimicrob Chemother, 2019, 74: 27-37.
[3]	何文军, 李曼, 李涛, 等. 基于血细胞形态识别的自动检测系统的研发[J]. 现代检验医学杂志, 2019, 34: 110-114. https://www.cnki.com.cn/Article/CJFDTOTAL-SXYN201902027.htm He WJ, Li M, Li T, et al. Study on Automatic Detection System Base on Blood Cell Morphology Recognition[J]. Xiandai Jianyan Yixue Zazhi, 2019, 34: 110-114. https://www.cnki.com.cn/Article/CJFDTOTAL-SXYN201902027.htm
[4]	赵颖, 李志荣, 赵建宏, 等. 河北地区临床实验室丝状真菌分离鉴定情况分析[J]. 中国真菌学杂志, 2020, 15: 206-212. DOI: 10.3969/j.issn.1673-3827.2020.04.004 Zhao Y, Li ZR, Zhao JH, et al. Analysis of filamentous fungi isolations from clinical laboratories in Hebei province[J]. Zhongguo Zhenjunxue Zazhi, 2020, 15: 206-212. DOI: 10.3969/j.issn.1673-3827.2020.04.004
[5]	Tamiev D, Furman PE, Reuel NF. Automated classification of bacterial cell sub-populations with convolutional neural networks[J]. PLoS One, 2020, 15: e0241200. DOI: 10.1371/journal.pone.0241200
[6]	Kulwa F, Li C, Zhang J, et al. A new pairwise deep learning feature for environmental microorganism image analysis[J]. Environ Sci Pollut Res Int, 2022. doi: 10.1007/s11356-022-18849-0.
[7]	Zhang J, Li C, Kosov S, et al. LCU-Net: A novel low-cost U-Net for environmental microorganism image segmentation[J]. Pattern Recogn, 2021, 115: 107885. DOI: 10.1016/j.patcog.2021.107885
[8]	Liang CM, Lai CC, Wang SH, et al. Environmental microorganism classification using optimized deep learning model[J]. Environ Sci Pollut Res Int, 2021, 28: 31920-31932. DOI: 10.1007/s11356-021-13010-9
[9]	李卓识, 陈晓旭, 温长吉, 等. 基于机器学习的鹅膏属真菌形态特征分类模型研究[J]. 中国农机化学报, 2020, 41: 136-143. https://www.cnki.com.cn/Article/CJFDTOTAL-GLJH202001026.htm Li ZS, Chen XX, Wen CJ, et al. Study on classification model of morphological characteristics of amanita fungi based on machine learning[J]. Zhongguo Nongjihua Xuebao, 2020, 41: 136-143. https://www.cnki.com.cn/Article/CJFDTOTAL-GLJH202001026.htm
[10]	Tahir MW, Zaidi NA, Rao AA, et al. A fungus spores dataset and a convolutional neural network based approach for fungus detection[J]. IEEE Transact Nano Biosci, 2018, 17: 281-290.
[11]	Zhang J, Lu S, Wang X, et al. Automatic identification of fungi in microscopic leucorrhea images[J]. J Opt Soc Am A Opt Image Sci Vis, 2017, 34: 1484-1489. DOI: 10.1364/JOSAA.34.001484
[12]	周院. 基于深度学习的真菌图像分类算法研究[D]. 西安: 西安理工大学, 2019.
[13]	郝如茜. 白带显微图像中霉菌自动识别及清洁度判定的研究[D]. 成都: 电子科技大学, 2017.
[14]	Otsu N. A threshold selection method from gray-level histograms[J]. IEEE Transact Syst Man Cyb, 2007, 9: 62-66.
[15]	Hao R, Wang X, Zhang J, et al. Automatic detection of fungi in microscopic leucorrhea images based on convolu-tional neural network and morphological method[C]. 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), 2019: 2491-2494.
[16]	Sandler M, Howard A, Zhu M, et al. Mobilenetv2: Inverted residuals and linear bottlenecks[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4510-4520.
[17]	Deng J, Dong W, Socher R, et al. Imagenet: A large-scale hierarchical image database[C]. 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009: 248-255.
[18]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141.
[19]	Yu F, Koltun V. Multi-Scale Context Aggregation by Dilated Convolutions[C]. International Conference on Learning Representations, 2016. https://doi.org/10.48550/arXiv.1511.07122.
[20]	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
[21]	Huang G, Liu Z, Laurens V, et al. Densely connected convolutional networks[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4700-4708.
[22]	Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 2818-2826.
[23]	Loshchilov I, Hutter F. SGDR: Stochastic Gradient Descent with Restarts[J]. ICLR 2017 Conference Paper, 2016. https://doi.org/10.48550/arXiv.1608.03983.
[24]	Pan SJ, Qiang Y. A survey on transfer learning[J]. IEEE Transact Knowl Data En, 2010, 22: 1345-1359.
[25]	Selvaraju RR, Cogswell M, Das A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization[J]. Int J Comput Vision, 2020, 128: 336-359.
[26]	Mital ME, Tobias RR, Villaruel H, et al. Transfer learning approach for the classification of conidial fungi (genus aspergillus) thru pre-trained deep learning models[C]. 2020 IEEE Region 10 Conference (Tencon), 2020: 1069-1074.
[27]	Billones RK, Calilung EJ, Dadios EP, et al. Aspergillus species fungi identification using microscopic scale images[C]. 2020 IEEE 12th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management(HNICEM), 2020. doi: 10.1109/HNICEM51456.2020.9400039.
[28]	Zawadzki P. Deep learning approach to the classification of selected fungi and bacteria[C]. 2020 IEEE 21st International Conference on Computational Problems of Electrical Engineering (CPEE), 2020: 1-4.
[29]	Zieliński B, Sroka-Oleksiak A, Rymarczyk D, et al. Deep learning approach to describe and classify fungi microscopic images[J]. PLoS One, 2020, 15: e0234806.

Cited By

Cited by

Periodical cited type(0)

Other cited types(2)

Get Citation

PDF

XML

Article Metrics

Article views (841) PDF downloads (63) Cited by(2)

Address：	Room 302,Third Floor,Building 7, No. 1 Shuaifuyuan, Wangfujing, Dongcheng District, Beijing 100730, China
Tel：	010-69154261/4262
E - mail：	medj@pumch.cn mjpumch@126.com
Copyright of website：	Editorial Office of Medical Journal of Peking Union Medical College Hospital

Fungal Microscopic Image Classification Based on Multi-scale Attention Mechanism

Abstract

References

Cited by

Periodical cited type(0)

Other cited types(2)

Catalog

Article Metrics

Related

Fungal Microscopic Image Classification Based on Multi-scale Attention Mechanism

Abstract

References

Cited by

Periodical cited type(0)

Other cited types(2)

Catalog

Article Metrics

Related

Export File

Citation

Format

Content