高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

一种结合三重注意力机制的双路径网络胸片疾病分类方法

李锵 王旭 关欣

李锵, 王旭, 关欣. 一种结合三重注意力机制的双路径网络胸片疾病分类方法[J]. 电子与信息学报, 2023, 45(4): 1412-1425. doi: 10.11999/JEIT220172
引用本文: 李锵, 王旭, 关欣. 一种结合三重注意力机制的双路径网络胸片疾病分类方法[J]. 电子与信息学报, 2023, 45(4): 1412-1425. doi: 10.11999/JEIT220172
LI Qiang, WANG Xu, GUAN Xin. A Dual-path Network Chest Film Disease Classification Method Combined with a Triple Attention Mechanism[J]. Journal of Electronics & Information Technology, 2023, 45(4): 1412-1425. doi: 10.11999/JEIT220172
Citation: LI Qiang, WANG Xu, GUAN Xin. A Dual-path Network Chest Film Disease Classification Method Combined with a Triple Attention Mechanism[J]. Journal of Electronics & Information Technology, 2023, 45(4): 1412-1425. doi: 10.11999/JEIT220172

一种结合三重注意力机制的双路径网络胸片疾病分类方法

doi: 10.11999/JEIT220172
基金项目: 国家自然科学基金(61471263, 61872267, 62071323),天津市自然科学基金(16JCZDJC31100),天津市科技计划项目(20YDTPJC01110),天津大学自主创新基金(2021XZC-0024)
详细信息
    作者简介:

    李锵:男,博士,主要研究方向为医学图像处理、立体视觉与人工智能

    王旭:男,硕士生,研究方向为医学图像处理

    关欣:女,博士,主要研究方向为音乐信号处理

    通讯作者:

    李锵 liqiang@tju.edu.cn

  • 中图分类号: TN911.73; TP391.41

A Dual-path Network Chest Film Disease Classification Method Combined with a Triple Attention Mechanism

Funds: The National Natural Science Foundation of China (61471263, 61872267, 62071323), The Natural Science Foundation of Tianjin(16JCZDJC31100), The Scientific and Technological Project of Tianjin (20YDTPJC01110), The Seed Foundation of Tianjin University (2021XZC-0024)
  • 摘要: 近年来,利用CNN进行医学图像处理,在胸片疾病分类任务中取得显著研究进展。然而,与单一结构CNN相比,双路径网络可结合不同CNN特点,从而提高疾病分类能力。其次,对于不同疾病,其位置、大小、形态、密度、纹理等特征均有不同,而注意力机制有助于模型提取不同病理特征,提升分类精度。因此针对胸片疾病分类问题,该文提出一种结合三重注意力机制的双路径卷积神经网络(TADPN),TADPN将ResNet和DenseNet结合的双路径网络DPN作为骨干网络,并利用3种不同形式的注意力机制改进DPN,在维持参数量稳定的同时提高网络复杂度,进而提升对胸片疾病的分类精度。在ChestXray14数据集上实验,并与目前较为先进的6种算法对比,14种疾病的平均AUC值达到0.8185,较前人提升1.1%,表明双路径CNN及三重注意力机制对胸片疾病分类的有效性及TADPN的先进性。
  • 图  1  模型整体框架

    图  2  DPN基本单元与TADPN基本单元

    图  3  一个患有结节的胸片成像

    图  4  SK结构

    图  5  CAM和SAM实现方式

    图  6  CAM结构

    图  7  SAM结构

    图  8  原数据和经CLAHE增强后数据对比

    图  9  TADPN在ChestX-ray14数据集上的ROC曲线和AUC值

    图  10  TADPN对不同测试集的泛化实验ROC曲线和AUC值

    图  11  8种胸部疾病的病灶区标注图与病灶区定位热图

    表  1  TADPN网络结构

    操作输出尺寸
    卷积层7×7卷积,步长为2112×112
    池化层3×3最大池化,步长为256×56
    TADPN块(1)$ \left\{\begin{array}{l}1\times \text{1}卷积、\text{3}\times \text{3 SK}卷积、1\times \text{1}卷积\\ \text{ CAM}、\text{SAM}\end{array}\right\} $×356×56
    TADPN块(2)$ \left\{\begin{array}{l}1\times \text{1}卷积、\text{3}\times \text{3 SK}卷积、1\times \text{1}卷积\\ \text{ CAM}、\text{SAM}\end{array}\right\} $×428×28
    TADPN块(3)$ \left\{\begin{array}{l}1\times \text{1}卷积、\text{3}\times \text{3 SK}卷积、1\times \text{1}卷积\\ \text{ CAM}、\text{SAM}\end{array}\right\} $×2014×14
    TADPN块(4)$ \left\{\begin{array}{l}1\times \text{1}卷积、\text{3}\times \text{3 SK}卷积、1\times \text{1}卷积\\ \text{ CAM}、\text{SAM}\end{array}\right\} $×37×7
    分类层14维全连接1×14
    下载: 导出CSV

    表  2  14种胸片疾病的影像学诊断依据

    疾病影像学依据
    肺不张肺野呈均匀致密影,气管、纵隔向患侧移位,肋间隙变窄
    心脏肥大心影增大
    积液肋膈角变钝或消失,纵隔向健侧移位,体液上缘呈外高内低凹面向上弧形影
    浸润患侧浸润性阴影
    肿块肺实质内呈高密度阴影(直径大于3 cm)
    结节肺实质内呈高密度阴影(直径小于3 cm)
    肺炎肺纹理增多、密度增高,呈毛玻璃影,片状模糊,边界不清
    气胸肺被压缩向肺门部收缩,呈均匀无肺纹理走形透亮影
    肺实变呈空气支气管征、肺泡充气征、阴影不透明、血管模糊
    水肿肺纹理以及肺门区血管增粗,肺透亮度降低、肋膈角改变或消失
    肺气肿肺体积增加、纹理增粗、透过度增大
    纤维化肺中下野呈毛玻璃状、典型性改变弥漫性线条状、结节状、云絮样、网状阴影
    胸膜增厚肋隔角变浅、变钝,呈不规则条状钙化
    疝气呈高透光度膨出,与肺组织相连
    下载: 导出CSV

    表  3  ChestXray14数据集疾病种类及数量

    疾病名称数量(张)疾病名称数量(张)疾病名称数量(张)
    肺不张(Atelectasis)11 559气胸(Pneumothorax)5 302无病60 361
    心脏肥大(Cardiomegaly)2 776肺实变(Consolidation)4 667
    积液(Effusion)13 317水肿(Edema)2 303
    浸润(Infiltration)19 894肺气肿(Emphysema)2 516
    肿块(Mass)5 782纤维化(Fibrosis)1 686
    结节(Nodule)6 331胸膜增厚(Pleural Thickening)3 385
    肺炎(Pneumonia)1 431疝气(Hernia)227
    患病合计51 759
    总计112 120
    下载: 导出CSV

    表  4  混淆矩阵

    预测类别真实类别
    1(患病)0(无病)
    1(患病)真阳 (TP)伪阳(FP)
    0(无病)伪阴 (FN)真阴(TN)
    下载: 导出CSV

    表  5  不同模型在ChestX-ray14数据集上的结果比较

    基于ResNet模型基于DenseNet模型TADPN
    文献[14]文献[35]文献[21]文献[36]文献[37]文献[28]本文
    肺不张0.755 70.800.800 40.762 70.7950.785 00.794 5
    心脏肥大0.886 50.870.879 80.883 50.8870.876 60.901 2
    积液0.819 10.870.8720.815 90.8750.862 80.882 3
    浸润0.689 20.700.712 20.678 60.7030.673 00.693 5
    肿块0.813 60.830.795 30.801 20.83 50.804 00.822 2
    结节0.754 50.750.720 50.729 30.7160.729 90.728 0
    肺炎0.729 20.670.734 70.709 70.7420.742 30.742 8
    气胸0.849 90.870.842 20.837 70.8630.842 60.874 5
    肺实变0.728 30.800.8010.744 30.7860.784 60.802 9
    水肿0.847 50.880.878 70.841 40.8920.872 70.895 9
    肺气肿0.907 50.910.853 60.883 60.8750.858 00.880 9
    纤维化0.817 90.780.7980.807 70.7560.775 40.794 3
    胸膜增厚0.764 70.760.743 10.753 60.7740.775 60.774 2
    疝气0.874 70.770.871 10.876 30.8360.864 50.872 4
    平均0.802 70.8040.807 30.794 10.809 70.802 00.818 5
    下载: 导出CSV

    表  6  3种骨干网络分类结果对比

    肺不张心脏肥大积液浸润肿块结节肺炎
    ResNet-1010.790 00.907 70.870 50.688 20.80740.720 80.706 9
    DenseNet-1210.787 20.903 00.872 40.685 30.802 50.709 60.722 2
    DPN-920.791 30.910 60.873 40.684 20.807 60.713 80.730 1
    气胸肺实变水肿肺气肿纤维化胸膜增厚疝气平均
    ResNet-1010.860 40.794 80.890 00.868 20.777 40.761 10.864 50.807 7
    DenseNet-1210.861 60.799 40.886 30.866 00.773 90.763 20.895 10.809 1
    DPN-920.861 50.797 90.888 80.867 70.782 70.774 60.897 40.813 0
    下载: 导出CSV

    表  7  消融结果对比

    肺不张心脏肥大积液浸润肿块结节肺炎
    骨干模型0.791 30.910 60.873 40.684 20.807 60.713 80.730 1
    移除SK0.787 40.910 20.878 90.690 80.808 70.720 40.743 5
    移除CAM0.791 90.907 40.875 00.684 60.815 30.722 30.736 9
    移除SAM0.786 40.907 40.875 80.688 70.816 40.721 70.738 6
    TADPN0.794 50.901 20.882 30.693 50.822 20.728 00.742 8
    气胸肺实变水肿肺气肿纤维化胸膜增厚疝气平均
    骨干模型0.861 50.797 90.888 80.867 70.782 70.774 60.897 40.813 0
    移除SK0.865 90.804 60.897 10.879 30.783 40.770 20.875 00.815 4
    移除CAM0.867 20.802 70.894 20.881 20.775 80.774 40.890 00.815 6
    移除SAM0.873 20.803 40.887 50.876 10.783 60.780 50.872 00.815 1
    TADPN0.874 50.802 90.895 90.880 90.794 30.774 20.872 40.818 5
    下载: 导出CSV

    表  8  SK卷积参数设置对比

    分支1分支2平均AUCParameters(M)FLOPs(G)
    卷积核1膨胀系数D1卷积核2膨胀系数D2
    3×313×320.818 54.565 120.579 4
    3×313×330.817 94.565 120.579 4
    3×323×330.817 74.565 120.579 4
    3×315×510.817 14.973 022.534 7
    3×317×710.815 55.584 825.467 7
    5×515×520.814 05.380 824.490 1
    5×517×710.812 25.992 727.423 1
    下载: 导出CSV

    表  9  注意力模块组合结果对比

    模型平均AUCParameters(M)FLOPs(G)
    DPN-920.813 017.109 54.315 1
    DPN-92+SK0.814 419.257 14.560 8
    DPN-92+CAM||SAM0.815 018.431 84.319 3
    DPN-92+CAM+SAM0.815 418.431 84.319 3
    DPN-92+SAM+CAM0.814 818.431 84.319 3
    TADPN0.818 520.579 44.565 1
    下载: 导出CSV
  • [1] BALA D. Childhood pneumonia recognition using convolutional neural network from chest X-ray images[J]. Journal of Electrical Engineering, Electronics, Control and Computer Science, 2021, 7(26): 33–40.
    [2] World Health Organization. Causes of death among children[EB/OL]. https://www.who.int/maternal_child_adolescent/data/causes-death-children/en/.2018.1.
    [3] KHOBRAGADE S, TIWARI A, PATIL C Y, et al. Automatic detection of major lung diseases using Chest Radiographs and classification by feed-forward artificial neural network[C]. 2016 IEEE 1st International Conference on Power Electronics, Intelligent Control and Energy Systems, Delhi, India, 2016: 1–5.
    [4] QIN Chunli, YAO Demin, SHI Yonghong, et al. Computer-aided detection in chest radiography based on artificial intelligence: A survey[J]. Biomedical Engineering Online, 2018, 17(1): 113. doi: 10.1186/s12938-018-0544-y
    [5] AYESHA H, IQBAl S, TARIQ M, et al. Automatic medical image interpretation: State of the art and future directions[J]. Pattern Recognition, 2021, 114: 107856. doi: 10.1016/j.patcog.2021.107856
    [6] PEZZANO G, RIPOLL V R, and RADEVA P. CoLe-CNN: Context-learning convolutional neural network with adaptive loss function for lung nodule segmentation[J]. Computer Methods and Programs in Biomedicine, 2021, 198: 105792. doi: 10.1016/j.cmpb.2020.105792
    [7] 赵奕名, 李锵, 关欣. 组卷积轻量级脑肿瘤分割网络[J]. 中国图象图形学报, 2020, 25(10): 2159–2170. doi: 10.11834/jig.200247

    ZHAO Yiming, LI Qiang, and GUAN Xin. Lightweight brain tumor segmentation algorithm based on a group convolutional neural network[J]. Journal of Image and Graphics, 2020, 25(10): 2159–2170. doi: 10.11834/jig.200247
    [8] 明涛, 王丹, 郭继昌, 等. 基于多尺度通道重校准的乳腺癌病理图像分类[J]. 浙江大学学报:工学版, 2020, 54(7): 1289–1297. doi: 10.3785/j.issn.1008-973X.2020.07.006

    MING Tao, WANG Dan, GUO Jichang, et al. Breast cancer histopathological image classification using multi-scale channel squeeze-and-excitation model[J]. Journal of Zhejiang University:Engineering Science, 2020, 54(7): 1289–1297. doi: 10.3785/j.issn.1008-973X.2020.07.006
    [9] TEIXEIRA V, BRAZ L, PEDRINI H, et al. DuaLAnet: Dual lesion attention network for thoracic disease classification in chest X-rays[C]. 2020 International Conference on Systems, Signals and Image Processing, Niteroi, Brazil, 2020: 69–74.
    [10] LUO Luyang, YU Lequan, CHEN Hao, et al. Deep mining external imperfect data for chest X-ray disease screening[J]. IEEE Transactions on Medical Imaging, 2020, 39(11): 3583–3594. doi: 10.1109/TMI.2020.3000949
    [11] PANT H, LOHANI M C, BHATT A K, et al. Thoracic disease detection using deep learning[C]. 2021 5th International Conference on Computing Methodologies and Communication, Erode, India, 2021: 1197–1203.
    [12] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. The 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778.
    [13] OUYANG Xi, KARANAM S, WU Ziyan, et al. Learning hierarchical attention for weakly-supervised chest X-ray abnormality localization and diagnosis[J]. IEEE Transactions on Medical Imaging, 2021, 40(10): 2698–2710. doi: 10.1109/TMI.2020.3042773
    [14] TANG Yuxing, WANG Xiaosong, HARRISON A P, et al. Attention-guided curriculum learning for weakly supervised classification and localization of thoracic diseases on chest radiographs[C]. The 9th International Workshop on Machine Learning in Medical Imaging, Granada, Spain, 2018: 249–258.
    [15] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 2261–2269.
    [16] WANG Hao, YANG Yuanyuan, PAN Yang, et al. Detecting thoracic diseases via representation learning with adaptive sampling[J]. Neurocomputing, 2020, 406: 354–360. doi: 10.1016/j.neucom.2019.06.113
    [17] GÜNDEL S, SETIO A A A, GHESU F C, et al. Robust classification from noisy labels: Integrating additional knowledge for chest radiography abnormality assessment[J]. Medical Image Analysis, 2021, 72: 102087. doi: 10.1016/j.media.2021.102087
    [18] GUAN Qingji, HUANG Yaping, LUO Yawei, et al. Discriminative feature learning for thorax disease classification in chest X-ray images[J]. IEEE Transactions on Image Processing, 2021, 30: 2476–2487. doi: 10.1109/TIP.2021.3052711
    [19] 黄欣, 方钰, 顾梦丹. 基于卷积神经网络的X线胸片疾病分类研究[J]. 系统仿真学报, 2020, 32(6): 1188–1194. doi: 10.16182/j.issn1004731x.joss.18-0712

    HUANG Xin, FANG Yu, and GU Mengdan. Classification of chest X-ray disease based on convolutional neural network[J]. Journal of System Simulation, 2020, 32(6): 1188–1194. doi: 10.16182/j.issn1004731x.joss.18-0712
    [20] GÜNDEL S, GRBIC S, GEORGESCU B, et al. Learning to recognize abnormalities in chest x-rays with location-aware dense networks[C]. The 23rd Iberoamerican Congress on Pattern Recognition, Madrid, Spain, 2018: 757–765.
    [21] YANG Xiaoyilei, XU Shuaijing, WANG Jian, et al. Attention mechanism in radiologist-level thorax diseases detection[J]. Procedia Computer Science, 2020, 174: 524–529. doi: 10.1016/j.procs.2020.06.120
    [22] HU Jie, SHEN Li, and SUN Gang. Squeeze-and-excitation networks[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 7132–7141.
    [23] GUAN Qingji and HUANG Yaping. Multi-label chest X-ray image classification via category-wise residual attention learning[J]. Pattern Recognition Letters, 2020, 130: 259–266. doi: 10.1016/j.patrec.2018.10.027
    [24] CHEN Yunpeng, LI Jianan, XIAO Huaxin, et al. Dual path networks[C]. The 31st International Conference on Neural Information Processing Systems, Long Beach, USA, 2017: 4470–4478.
    [25] LI Xiang, WANG Wenhai, HU Xiaolin, et al. Selective kernel networks[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 510–519.
    [26] WOO S, PARK J, LEE J Y, et al. CBAM: Convolutional block attention module[C]. The 15th European Conference on Computer Vision, Munich, Germany, 2018: 3–19.
    [27] WANG Xiaosong, PENG Yifan, LU Le, et al. ChestX-Ray8: Hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 3462–3471.
    [28] 张智睿, 李锵, 关欣. 密集挤压激励网络的多标签胸部X光片疾病分类[J]. 中国图象图形学报, 2020, 25(10): 2238–2248. doi: 10.11834/jig.200232

    ZHANG Zhirui, LI Qiang, and GUAN Xin. Multilabel chest X-ray disease classification based on a dense squeeze-and-excitation network[J]. Journal of Image and Graphics, 2020, 25(10): 2238–2248. doi: 10.11834/jig.200232
    [29] 王粉花, 赵波, 黄超, 等. 基于多尺度和注意力融合学习的行人重识别[J]. 电子与信息学报, 2020, 42(12): 3045–3052. doi: 10.11999/JEIT190998

    WANG Fenhua, ZHAO Bo, HUANG Chao, et al. Person re-identification based on multi-scale network attention fusion[J]. Journal of Electronics &Information Technology, 2020, 42(12): 3045–3052. doi: 10.11999/JEIT190998
    [30] 尹梦晓, 林振峰, 杨锋. 基于动态感受野的自适应多尺度信息融合的图像转换[J]. 电子与信息学报, 2021, 43(8): 2386–2394. doi: 10.11999/JEIT200675

    YIN Mengxiao, LIN Zhenfeng, and YANG Feng. Adaptive multi-scale information fusion based on dynamic receptive field for image-to-image translation[J]. Journal of Electronics &Information Technology, 2021, 43(8): 2386–2394. doi: 10.11999/JEIT200675
    [31] 王睿川, 王岩飞. 基于半监督空间-通道选择性卷积核网络的极化SAR图像地物分类[J]. 雷达学报, 2021, 10(4): 516–530. doi: 10.12000/JR21080

    WANG Ruichuan and WANG Yanfei. Terrain classification of polarimetric SAR images using semi-supervised spatial-channel selective kernel network[J]. Journal of Radars, 2021, 10(4): 516–530. doi: 10.12000/JR21080
    [32] XIE Saining, GIRSHICK R, DOLLÁR P, et al. Aggregated residual transformations for deep neural networks[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 5987–5995.
    [33] 黎英, 宋佩华. 迁移学习在医学图像分类中的研究进展[J]. 中国图象图形学报, 2022, 27(3): 672–686. doi: 10.11834/jig.210814

    LI Ying and SONG Peihua. Review of transfer learning in medical image classification[J]. Journal of Image and Graphics, 2022, 27(3): 672–686. doi: 10.11834/jig.210814
    [34] 赵晓晴, 李慧盈, 苏安炀, 等. 基于加权损失函数的粘连白细胞分割算法[J]. 吉林大学学报:理学版, 2021, 59(1): 85–91. doi: 10.13413/j.cnki.jdxblxb.2020003

    ZHAO Xiaoqing, LI Huiying, SU Anyang, et al. Adhesive leukocyte segmentation algorithm based on weighted loss function[J]. Journal of Jilin University:Science Edition, 2021, 59(1): 85–91. doi: 10.13413/j.cnki.jdxblxb.2020003
    [35] LI Zhe, WANG Chong, HAN Mei, et al. Thoracic disease identification and localization with limited supervision[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 8290–8299.
    [36] MA Yanbo, ZHOU Qiuhao, CHEN Xuesong, et al. Multi-attention network for thoracic disease classification and localization[C]. ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing, Brighton, UK, 2019: 1378–1382.
    [37] HO T K K and GWAK J. Multiple feature integration for classification of thoracic disease in chest radiography[J]. Applied Sciences, 2019, 9(19): 4130. doi: 10.3390/app9194130
    [38] PRASHANT P. Chest X-ray (Covid-19 & Pneumonia)[EB/OL]. https://www.kaggle.com/datasets/prashant268/chest-xray-covid19-pneumonia.
    [39] CHOWDHURY M E H, RAHMAN T, KHANDAKAR A, et al. Can AI help in screening viral and COVID-19 pneumonia?[J]. IEEE Access, 2020, 8: 132665–132676. doi: 10.1109/ACCESS.2020.3010287
    [40] SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization[C]. 2017 IEEE International Conference on Computer Vision, Venice, Italy, 2017: 618–626.
  • 加载中
图(11) / 表(9)
计量
  • 文章访问数:  574
  • HTML全文浏览量:  411
  • PDF下载量:  119
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-02-22
  • 修回日期:  2022-07-27
  • 录用日期:  2022-08-02
  • 网络出版日期:  2022-08-05
  • 刊出日期:  2023-04-10

目录

    /

    返回文章
    返回