留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于多尺度注意力机制的真菌显微图像分类方法

张雪媛 许鸿雁 董跃明 刘丹凤 孙鹏蕊 颜锐 崔洪亮 雷红 任菲

张雪媛, 许鸿雁, 董跃明, 刘丹凤, 孙鹏蕊, 颜锐, 崔洪亮, 雷红, 任菲. 基于多尺度注意力机制的真菌显微图像分类方法[J]. 协和医学杂志, 2023, 14(1): 139-147. doi: 10.12290/xhyxzz.2022-0169
引用本文: 张雪媛, 许鸿雁, 董跃明, 刘丹凤, 孙鹏蕊, 颜锐, 崔洪亮, 雷红, 任菲. 基于多尺度注意力机制的真菌显微图像分类方法[J]. 协和医学杂志, 2023, 14(1): 139-147. doi: 10.12290/xhyxzz.2022-0169
ZHANG Xueyuan, XU Hongyan, DONG Yueming, LIU Danfeng, SUN Pengrui, YAN Rui, CUI Hongliang, LEI Hong, REN Fei. Fungal Microscopic Image Classification Based on Multi-scale Attention Mechanism[J]. Medical Journal of Peking Union Medical College Hospital, 2023, 14(1): 139-147. doi: 10.12290/xhyxzz.2022-0169
Citation: ZHANG Xueyuan, XU Hongyan, DONG Yueming, LIU Danfeng, SUN Pengrui, YAN Rui, CUI Hongliang, LEI Hong, REN Fei. Fungal Microscopic Image Classification Based on Multi-scale Attention Mechanism[J]. Medical Journal of Peking Union Medical College Hospital, 2023, 14(1): 139-147. doi: 10.12290/xhyxzz.2022-0169

基于多尺度注意力机制的真菌显微图像分类方法

doi: 10.12290/xhyxzz.2022-0169
基金项目: 

国家重点研发计划 2021YFF1201005

解放军总医院医疗大数据与人工智能研发项目 2019MBD-050

详细信息
    通讯作者:

    雷红, E-mail:jlh309@163.com

    任菲, E-mail:renfei@ict.ac.cn

  • 中图分类号: R756; TP29

Fungal Microscopic Image Classification Based on Multi-scale Attention Mechanism

Funds: 

National Key R & D Program 2021YFF1201005

Medical Big Data and Artificial Intelligence R & D Project of PLA General Hospital 2019MBD-050

More Information
  • 摘要:   目的  利用深度学习技术,建立临床常见的侵袭性真菌图像辅助分类模型。  方法  回顾性收集2020年9月—2021年4月解放军总医院第八医学中心曲霉菌属、酵母菌属和新型隐球菌属真菌感染者的显微镜图像,按7∶1.5∶1.5的比例随机分为训练集、验证集和测试集。使用训练集和验证集图像对改进的MobileNetV2网络结构进行训练和参数调试,构建基于多尺度注意力机制的卷积神经网络(convolutional neural network, CNN)真菌图像11分类模型。以机器鉴定结果为金标准,以查准率、召回率和F1值为指标评价该模型对测试集真菌图像的分类效果。将该模型与5种经典CNN模型进行比较,评价指标包括模型参数量、内存占用量、网络每秒处理的图像数量(frames per second, FPS)、准确率及受试者操作特征曲线下面积(area under the curve,AUC)。  结果  共纳入真菌显微镜图像7666张,分别包括曲霉菌属、酵母菌属和新型隐球菌属图像2781张、4115张、770张。其中训练集5366张、验证集1150张、测试集1150张。改进的MobileNetV2模型对测试集11种真菌图像具有较高的分类性能,查准率为96.36%~100%,召回率为96.53%~100%,F1值为97.01%~100%。该模型的参数量、内存占用量分别为4.22 M、356.89 M,FPS为573,准确率为(99.09±0.18)%,AUC为0.9944±0.0018,综合性能优于5种经典网络模型。  结论  本研究提出的真菌图像分类模型,在保持低运算成本的情况下,可获得较高的真菌图像识别能力,其整体性能优于常见的经典模型。
    作者贡献:张雪媛、许鸿雁负责研究方案设计、实验操作、数据分析与论文撰写;董跃明、刘丹凤、孙鹏蕊负责数据采集与标注;颜锐、崔洪亮、雷红、任菲负责数据审核及内容审校。
    利益冲突:所有作者均声明不存在利益冲突
  • 图  1  去噪前后的真菌图像对比

    图  2  基于改进的MobileNetV2模型结构示意图

    图  3  Dilated SE模块运行示意图

    图  4  11种真菌显微镜图像

    图  5  基于改进的MobileNetV2真菌分类模型混淆矩阵

    图  6  部分真菌原始显微镜图像和类激活热力图

    A.光滑念珠菌;B.葡萄芽菌;C.新型隐球菌;D.黑曲霉菌;E.烟曲霉菌;F.杂色曲霉菌
    (1)原始显微镜图像;(2)类激活热力图,颜色越趋于红色,表示该区域受网络结构的关注度越高

    表  1  改进的MobileNetV2真菌分类模型在测试集中的表现(x±s,%)

    菌种 查准率 召回率 F1值
    烟曲霉菌 99.77±0.16 98.95±0.29 99.36±0.08
    杂色曲霉菌 96.41±0.62 99.77±0.16 98.06±0.25
    黑曲霉菌 100±0 97.16±0.63 98.56±0.32
    解脂假丝酵母菌 99.63±0 100±0 99.81±0
    近平滑念珠菌 96.36±0.51 97.90±0.46 97.12±0.46
    葡萄芽菌 97.50±0.70 96.53±0.17 97.01±0.40
    季也蒙念珠菌 99.87±0.18 99.35±0.49 99.61±0.16
    克柔念珠菌 100±0 100±0 100±0
    光滑念珠菌 99.39±0.86 96.97±1.71 98.15±0.77
    热带念珠菌 96.53±1.87 99.28±1.02 97.86±0.85
    新型隐球菌 100±0 99.57±0 99.78±0
    下载: 导出CSV

    表  2  不同分类模型在测试集中的运行结果

    指标 参数量(M) 内存占用量(M) FPS 准确率(x±s,%) AUC(x±s)
    ResNet18 11.18 252.44 527 98.83±0.16 0.9935±0.0008
    MobileNetV2 2.24 603.82 501 98.68±0.12 0.9928±0.0006
    SENet 0.74 292.24 562 97.78±1.70 0.9878±0.0032
    DenseNet121 6.97 1341.44 177 98.81±0.27 0.9939±0.0011
    Inception V3 21.81 468.05 105 98.84±0.11 0.9948±0.0007
    改进的MobileNetV2 4.22 356.89 573 99.09±0.18 0.9944±0.0018
    FPS:网络每秒处理的图像数量;AUC:曲线下面积
    下载: 导出CSV
  • [1] Chen M, Xu Y, Hong N, et al. Epidemiology of fungal infections in China[J]. Front Med, 2018, 12: 58-75. doi:  10.1007/s11684-017-0601-0
    [2] Sanguinetti M, Posteraro B, Beigelman-Aubry C, et al. Diagnosis and treatment of invasive fungal infections: looking ahead[J]. J Antimicrob Chemother, 2019, 74: 27-37.
    [3] 何文军, 李曼, 李涛, 等. 基于血细胞形态识别的自动检测系统的研发[J]. 现代检验医学杂志, 2019, 34: 110-114. https://www.cnki.com.cn/Article/CJFDTOTAL-SXYN201902027.htm

    He WJ, Li M, Li T, et al. Study on Automatic Detection System Base on Blood Cell Morphology Recognition[J]. Xiandai Jianyan Yixue Zazhi, 2019, 34: 110-114. https://www.cnki.com.cn/Article/CJFDTOTAL-SXYN201902027.htm
    [4] 赵颖, 李志荣, 赵建宏, 等. 河北地区临床实验室丝状真菌分离鉴定情况分析[J]. 中国真菌学杂志, 2020, 15: 206-212. doi:  10.3969/j.issn.1673-3827.2020.04.004

    Zhao Y, Li ZR, Zhao JH, et al. Analysis of filamentous fungi isolations from clinical laboratories in Hebei province[J]. Zhongguo Zhenjunxue Zazhi, 2020, 15: 206-212. doi:  10.3969/j.issn.1673-3827.2020.04.004
    [5] Tamiev D, Furman PE, Reuel NF. Automated classification of bacterial cell sub-populations with convolutional neural networks[J]. PLoS One, 2020, 15: e0241200. doi:  10.1371/journal.pone.0241200
    [6] Kulwa F, Li C, Zhang J, et al. A new pairwise deep learning feature for environmental microorganism image analysis[J]. Environ Sci Pollut Res Int, 2022. doi:  10.1007/s11356-022-18849-0.
    [7] Zhang J, Li C, Kosov S, et al. LCU-Net: A novel low-cost U-Net for environmental microorganism image segmentation[J]. Pattern Recogn, 2021, 115: 107885. doi:  10.1016/j.patcog.2021.107885
    [8] Liang CM, Lai CC, Wang SH, et al. Environmental microorganism classification using optimized deep learning model[J]. Environ Sci Pollut Res Int, 2021, 28: 31920-31932. doi:  10.1007/s11356-021-13010-9
    [9] 李卓识, 陈晓旭, 温长吉, 等. 基于机器学习的鹅膏属真菌形态特征分类模型研究[J]. 中国农机化学报, 2020, 41: 136-143. https://www.cnki.com.cn/Article/CJFDTOTAL-GLJH202001026.htm

    Li ZS, Chen XX, Wen CJ, et al. Study on classification model of morphological characteristics of amanita fungi based on machine learning[J]. Zhongguo Nongjihua Xuebao, 2020, 41: 136-143. https://www.cnki.com.cn/Article/CJFDTOTAL-GLJH202001026.htm
    [10] Tahir MW, Zaidi NA, Rao AA, et al. A fungus spores dataset and a convolutional neural network based approach for fungus detection[J]. IEEE Transact Nano Biosci, 2018, 17: 281-290.
    [11] Zhang J, Lu S, Wang X, et al. Automatic identification of fungi in microscopic leucorrhea images[J]. J Opt Soc Am A Opt Image Sci Vis, 2017, 34: 1484-1489. doi:  10.1364/JOSAA.34.001484
    [12] 周院. 基于深度学习的真菌图像分类算法研究[D]. 西安: 西安理工大学, 2019.
    [13] 郝如茜. 白带显微图像中霉菌自动识别及清洁度判定的研究[D]. 成都: 电子科技大学, 2017.
    [14] Otsu N. A threshold selection method from gray-level histograms[J]. IEEE Transact Syst Man Cyb, 2007, 9: 62-66.
    [15] Hao R, Wang X, Zhang J, et al. Automatic detection of fungi in microscopic leucorrhea images based on convolu-tional neural network and morphological method[C]. 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), 2019: 2491-2494.
    [16] Sandler M, Howard A, Zhu M, et al. Mobilenetv2: Inverted residuals and linear bottlenecks[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 4510-4520.
    [17] Deng J, Dong W, Socher R, et al. Imagenet: A large-scale hierarchical image database[C]. 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009: 248-255.
    [18] Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141.
    [19] Yu F, Koltun V. Multi-Scale Context Aggregation by Dilated Convolutions[C]. International Conference on Learning Representations, 2016. https://doi.org/10.48550/arXiv.1511.07122.
    [20] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
    [21] Huang G, Liu Z, Laurens V, et al. Densely connected convolutional networks[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4700-4708.
    [22] Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 2818-2826.
    [23] Loshchilov I, Hutter F. SGDR: Stochastic Gradient Descent with Restarts[J]. ICLR 2017 Conference Paper, 2016. https://doi.org/10.48550/arXiv.1608.03983. doi:  10.48550/arXiv.1608.03983
    [24] Pan SJ, Qiang Y. A survey on transfer learning[J]. IEEE Transact Knowl Data En, 2010, 22: 1345-1359.
    [25] Selvaraju RR, Cogswell M, Das A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization[J]. Int J Comput Vision, 2020, 128: 336-359.
    [26] Mital ME, Tobias RR, Villaruel H, et al. Transfer learning approach for the classification of conidial fungi (genus aspergillus) thru pre-trained deep learning models[C]. 2020 IEEE Region 10 Conference (Tencon), 2020: 1069-1074.
    [27] Billones RK, Calilung EJ, Dadios EP, et al. Aspergillus species fungi identification using microscopic scale images[C]. 2020 IEEE 12th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management(HNICEM), 2020. doi: 10.1109/HNICEM51456.2020.9400039.
    [28] Zawadzki P. Deep learning approach to the classification of selected fungi and bacteria[C]. 2020 IEEE 21st International Conference on Computational Problems of Electrical Engineering (CPEE), 2020: 1-4.
    [29] Zieliński B, Sroka-Oleksiak A, Rymarczyk D, et al. Deep learning approach to describe and classify fungi microscopic images[J]. PLoS One, 2020, 15: e0234806.
  • 加载中
图(6) / 表(2)
计量
  • 文章访问数:  515
  • HTML全文浏览量:  130
  • PDF下载量:  29
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-03-31
  • 录用日期:  2022-05-26
  • 网络出版日期:  2022-09-20
  • 刊出日期:  2023-01-30

目录

    /

    返回文章
    返回

    【温馨提醒】近日,《协和医学杂志》编辑部接到作者反映,有多名不法人员冒充期刊编辑发送见刊通知,鼓动作者添加微信,从而骗取版面费的行为。特提醒您,本刊与作者联系的方式均为邮件通知或电话,稿件进度通知邮箱为:mjpumch@126.com,编辑部电话为:010-69154261,请提高警惕,谨防上当受骗!如有任何疑问,请致电编辑部核实。谢谢!