基于sECANet通道注意力机制的肾透明细胞癌病理图像ISUP分级预测

杨昆; 常世龙; 王尉丞; 高聪; 刘筱; 刘爽; 薛林雁

doi:10.11999/JEIT210900

基于sECANet通道注意力机制的肾透明细胞癌病理图像ISUP分级预测

doi: 10.11999/JEIT210900

杨昆^{1, 2, 3},
常世龙¹,
王尉丞¹,
高聪¹,
刘筱¹,
刘爽^{1, 2, 3},
薛林雁^{1, 2, 3, ,}

1.
河北大学质量技术监督学院保定 071002
2.
计量仪器与系统国家地方联合工程研究中心保定 071002
3.
河北省新能源汽车动力系统轻量化技术创新中心保定 071002

基金项目: 河北省自然科学基金(H2019201378)

详细信息

作者简介:
杨昆：男，1976年生，教授，研究方向为分子医学影像技术、基于人工智能的医学影像处理

常世龙：男，1995年生，硕士生，研究方向为智能仪器与机器视觉

王尉丞：男，1996年生，硕士生，研究方向为智能仪器与机器视觉

高聪：男，1995年生，硕士生，研究方向为智能仪器与机器视觉

刘筱：女，1996年生，硕士生，研究方向为智能仪器与机器视觉

刘爽：女，1981年生，副教授，研究方向为生物医学图像处理

薛林雁：女，1981年生，副教授，研究方向为生物医学图像处理、视觉神经机制

通讯作者:
薛林雁　lyxue@hbu.edu.cn

中图分类号: TN911.73; TP391.72
计量
- 文章访问数: 1195
- HTML全文浏览量: 577
- PDF下载量: 83
- 被引次数: 0
出版历程
- 收稿日期: 2021-08-30
- 修回日期: 2021-12-26
- 录用日期: 2021-12-27
- 网络出版日期: 2022-01-04
- 刊出日期: 2022-01-10

Predict the ISUP Grade of Clear Cell Renal Cell Carcinoma Using Pathological Images Based on sECANet Chanel Attention

YANG Kun^{1, 2, 3},
CHANG Shilong¹,
WANG Yucheng¹,
GAO Cong¹,
LIU Xiao¹,
LIU Shuang^{1, 2, 3},
XUE Linyan^{1, 2, 3
, ,}

1.
College of Quality and Technical Supervision, Hebei University, Baoding 071002, China
2.
National & Local Joint Engineering Research Center of Metrology Instrument and System, Hebei University, Baoding 071002, China
3.
Hebei Technology Innovation Center for Lightweight of New Energy Vehicle Power System, Baoding 071002, China

Funds: The Natural Science Foundation of Hebei Province (H2019201378)

摘要

摘要: 为了对肾透明细胞癌(ccRCC)进行准确核分级以改善肾癌的治疗和预后，该文提出一种新的通道注意力模块sECANet，通过计算特征图中当前通道与临近通道以及当前通道与远距离通道之间的信息交互来获取更多有用的特征。实验中收集了90例患者的肾组织病理图像，进行裁切和增强后采用五折交叉验证法对改进后的网络在Patch级别进行验证。实验结果表明，该文所提出的模型在Patch级别上鉴别ISUP分级的准确率为78.48±3.17%，精确率为79.95±4.37%，召回率为78.43±2.44%，F1分数为78.51±3.04%。进一步地，对每个病例所有Patch的预测结果采用多数投票法得到Image级别的分类结果，所有病例的准确率为88.89%，精确率为89.88%，召回率为87.65%，F1分数为88.51%。因此，sECANet在Patch级别和Image级别上均优于其他注意力机制和基本网络模型ResNet50。据此，该文所构建的病理图像ccRCC ISUP分级模型有良好的诊断效能，可以为患者的治疗和预后提供一定的参考。
- 肾透明细胞癌 /
- ISUP分级 /
- 病理图像 /
- 深度学习 /
- 注意力机制
Abstract: In order to determine accurately International Society for Urology and Pathology (ISUP) grade of clear cell Renal Cell Carcinoma (ccRCC) and achieve subsequently better treatment and prognosis, a novel channel attention mechanism named sECANet is proposed. To obtain more useful features from the feature map, sECANet calculates the information interaction of the current channel and local channels, and calculates additionally the interaction of the current channel and remote channels. A total of 90 pathological images are collected and subsequently cut into patches. After data augmentation, 5-fold cross-validation is employed to verify the improved network at the patch level. The experiment results show that the proposed model achieves 78.48±3.17% accuracy, 79.95±4.37% precision, 78.43±2.44% recall and 78.51±3.04% F1-score for ccRCC grading at the patch level. Furthermore, for the prediction of all patches in each patient case, the majority voting method is used to obtain the overall classification of the image level. The accuracy, precision, recall and F1-score of the proposed model at the image level are 88.89%, 89.88%, 87.65%, and 88.51%, respectively. In conclusion, the improved network with sECANet outperforms other attention mechanisms and the baseline model of ResNet50 at both patch level and image level. Therefore, the model of ccRCC ISUP grade established in this paper has relatively high diagnostic efficiency, and can even provide a certain reference for the treatment and prognosis for ccRCC patients.
- Clear cell renal cell carcinoma /
- ISUP grade /
- Pathological images /
- Deep learning /
- Attention mechanism

HTML全文

图 1 sECANet通道注意力模块

下载: 全尺寸图片幻灯片

图 2 ResNet50网络的整体结构图以及sECANet模块插入位置示意图

下载: 全尺寸图片幻灯片

图 3 40倍放大倍率下病理图像切分以及图像增强过程

下载: 全尺寸图片幻灯片

图 4 融合不同注意力模块的ResNet50模型在Patch级别分类结果混淆矩阵

下载: 全尺寸图片幻灯片

图 5 数据集中4种不同病理级别的典型示意图

下载: 全尺寸图片幻灯片

图 6 融合不同注意力模块的ResNet50模型在Image级别分类结果混淆矩阵

下载: 全尺寸图片幻灯片

图 7 融合不同注意力模块的ResNet50模型相同输入图像下的Grad-CAM图

下载: 全尺寸图片幻灯片

表 1 No.U090KI01组织芯片的临床信息

项目	特征	数量
年龄(岁)	≤50	33
	51～70	49
	＞70	8
性别	男性	28
性别	女性	62
ISUP等级	ISUP 1级	38
	ISUP 2级	25
	ISUP 3级	17
	正常	10
Stage分期	Ⅰ期	59
	Ⅱ期	19
	Ⅲ期	2
TNM分期	T1N0M0	59
	T2N0M0	19
	T3N0M0	2
发病位置	左肾癌	10
	右肾癌	8
	未标明	62

下载: 导出CSV

表 2 融合不同注意力模块的ResNet50模型在Patch级别的分类性能(%)对比

方法	Acc	Pre	Rec	F1
ResNet50	76.57±4.38	78.26±5.49	76.18±4.49	76.67±4.46
ResNet50+SeNet	77.64±3.42	78.92±4.74	77.69±3.03	77.39±3.51
ResNet50+ECANet	77.27±3.90	79.78±5.03	77.82±3.76	78.01±3.87
本文ResNet50+sECANet	78.48±3.17	79.95±4.37	78.43±2.44	78.51±3.04

下载: 导出CSV

表 3 融合不同注意力模块的ResNet50模型在Image级别的分类性能(%)对比

方法	Acc	Pre	Rec	F1
ResNet50	85.56	85.7	84.52	84.46
ResNet50+SeNet	85.56	84.72	84.52	84.36
ResNet50+ECANet	86.67	88.59	85.99	86.99
ResNet50+sECANet(Ours)	88.89	89.88	87.65	88.51

下载: 导出CSV

表 4 不同网络在Patch级别的分类性能(%)对比

方法	Acc	Pre	Rec	F1
ShuffleNet V2^[20]	75.87±5.03	77.78±5.64	75.87±5.34	75.79±5.13
DenseNet121^[24]	76.64±3.81	76.96±5.06	76.42±3.49	75.93±3.72
VGG16^[15]	75.89±3.60	77.20±5.38	74.97±4.88	74.66±4.70
本文ResNet50+sECANet	78.48±3.17	79.95±4.37	78.43±2.44	78.51±3.04

下载: 导出CSV

表 5 不同网络在Image级别的分类性能(%)对比

方法	Acc	Pre	Rec	F1
ShuffleNet V2^[20]	85.56	87.6	83.71	84.92
DenseNet121^[24]	86.67	86.46	86.33	86.15
VGG16^[15]	84.44	84.95	82.49	83.38
本文ResNet50+sECANet	88.89	89.88	87.65	88.51

下载: 导出CSV

参考文献(24)

[1]	SUNG H, FERLAY J, SIEGEL R L, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries[J]. CA:A Cancer Journal for Clinicians, 2021, 71(3): 209–249. doi: 10.3322/caac.21660
[2]	韩苏军, 王栋, 李长岭, 等. 1998-2008年中国肾癌发病趋势分析[J]. 癌症进展, 2018, 16(10): 1234–1237. HAN Sujun, WANG Dong, LI Changling, et al. Analysis for the incidence trends of renal cell carcinoma in China, 1998–2008[J]. Oncology Progress, 2018, 16(10): 1234–1237.
[3]	ZIMPFER A, GLASS Ä, ZETTL H, et al. Histopathologische diagnose und prognose des nierenzellkarzinoms im kontext der WHO-klassifikation 2016[J]. Der Urologe, 2019, 58(9): 1057–1065. doi: 10.1007/s00120-019-0952-z
[4]	MOCH H, CUBILLA A L, HUMPHREY P A, et al. The 2016 who classification of tumours of the urinary system and male genital organs—Part A: Renal, penile, and testicular tumours[J]. European Urology, 2016, 70(1): 93–105. doi: 10.1016/j.eururo.2016.02.029
[5]	陈皓, 段红柏, 郭紫园, 等. 基于CT征象量化分析的肺结节恶性度分级[J]. 电子与信息学报, 2021, 43(5): 1405–1413. doi: 10.11999/JEIT200167 CHEN Hao, DUAN Hongbai, GUO Ziyuan, et al. Malignancy grading of lung nodules based on CT signs quantization analysis[J]. Journal of Electronics &Information Technology, 2021, 43(5): 1405–1413. doi: 10.11999/JEIT200167
[6]	韩冬, 张喜荣, 贾永军, 等. 基于增强CT构建鉴别肾透明细胞癌ISUP分级的神经网络模型[J]. 肿瘤防治研究, 2021, 48(1): 55–59. doi: 10.3971/j.issn.1000-8578.2021.20.0440 HAN Dong, ZHANG Xirong, JIA Yongjun, et al. A neural network model based on enhanced CT for distinguishing ISUP grade of clear cell renal cell carcinoma[J]. Cancer Research on Prevention and Treatment, 2021, 48(1): 55–59. doi: 10.3971/j.issn.1000-8578.2021.20.0440
[7]	陈心怡, 明樱, 韩雨晴, 等. 基于CT动脉期图像构建预测肾透明细胞癌分级人工智能模型及其效能分析[J]. 医学影像学杂志, 2020, 30(6): 1033–1036. CHEN Xinyi, MING Ying, HAN Yuqing, et al. Construction of graded artificial intelligence model for predicting renal clear cell carcinoma based on CT arterial phase imaging and its efficacy analysis[J]. Journal of Medical Imaging, 2020, 30(6): 1033–1036.
[8]	康钦钦, 田冰, 边云, 等. 基于影像特征对透明细胞肾细胞癌病理分级的预测研究[J]. 影像诊断与介入放射学, 2021, 30(2): 103–110. doi: 10.3969/j.issn.1005-8001.2021.02.004 KANG Qinqin, TIAN Bing, BIAN Yun, et al. Prediction models based on CT features and RENAL score for clear cell renal cell carcinoma Fuhrman grade[J]. Diagnostic Imaging &Interventional Radiology, 2021, 30(2): 103–110. doi: 10.3969/j.issn.1005-8001.2021.02.004
[9]	LIN Fan, MA Changyi, XU Jinpeng, et al. A CT-based deep learning model for predicting the nuclear grade of clear cell renal cell carcinoma[J]. European Journal of Radiology, 2020, 129: 109079. doi: 10.1016/j.ejrad.2020.109079
[10]	HADJIYSKI N. Kidney cancer staging: Deep learning neural network based approach[C]. 2020 International Conference on E-Health and Bioengineering, Iasi, Romania, 2020: 1–4.
[11]	石博文, 叶靖, 段绍峰, 等. T2WI序列纹理分析联合机器学习的影像组学术前预测肾透明细胞癌的病理分级[J]. 中国医学影像学杂志, 2020, 28(8): 629–632. doi: 10.3969/j.issn.1005-5185.2020.08.018 SHI Bowen, YE Jing, DUAN Shaofeng, et al. Radiomics based on T2WI sequence texture analysis combined with machine learning imaging in predicting the pathological grade of clear cell renal cell carcinoma before surgery[J]. Chinese Journal of Medical Imaging, 2020, 28(8): 629–632. doi: 10.3969/j.issn.1005-5185.2020.08.018
[12]	张钰, 陈新元, 许宁, 等. 基于MRI纹理分析预测肾透明细胞癌核分级[J]. 中华放射学杂志, 2021, 55(1): 53–58. doi: 10.3760/cma.j.cn112149-20200712-00912 ZHANG Yu, CHEN Xinyuan, XU Ning, et al. Prediction of nuclear grade of renal clear cell carcinoma based on MRI texture analysis in combination with imaging features[J]. Chinese Journal of Radiology, 2021, 55(1): 53–58. doi: 10.3760/cma.j.cn112149-20200712-00912
[13]	LI Zewen, YANG Wenjie, PENG Shouheng, et al. A survey of convolutional neural networks: Analysis, applications, and prospects[EB/OL]. https://arxiv.org/abs/2004.02806, 2020.
[14]	KRIZHEVSKY A, SUTSKEVER I, and HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84–90. doi: 10.1145/3065386
[15]	SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]. The 3rd International Conference on Learning Representations, San Diego, USA, 2015: 7–12.
[16]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778.
[17]	徐从安, 吕亚飞, 张筱晗, 等. 基于双重注意力机制的遥感图像场景分类特征表示方法[J]. 电子与信息学报, 2021, 43(3): 683–691. doi: 10.11999/JEIT200568 XU Cong’an, LÜ Yafei, ZHANG Xiaohan, et al. A discriminative feature representation method based on dual attention mechanism for remote sensing image scene classification[J]. Journal of Electronics &Information Technology, 2021, 43(3): 683–691. doi: 10.11999/JEIT200568
[18]	HU Jie, SHEN Li, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011–2023. doi: 10.1109/TPAMI.2019.2913372
[19]	WANG Qilong, WU Banggu, ZHU Pengfei, et al. ECA-Net: Efficient channel attention for deep convolutional neural networks[C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 11531–11539.
[20]	ZHANG Xiangyu, ZHOU Xinyu, LIN Mengxiao, et al. ShuffleNet: An extremely efficient convolutional neural network for mobile devices[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 6848–6856.
[21]	LI Xiang, WANG Wenhai, HU Xiaolin, et al. Selective kernel networks[C]. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019: 510–519.
[22]	SHORTEN C and KHOSHGOFTAAR T M. A survey on image data augmentation for deep learning[J]. Journal of Big Data, 2019, 6(1): 60. doi: 10.1186/s40537-019-0197-0
[23]	SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization[J]. International Journal of Computer Vision, 2020, 128(2): 336–359. doi: 10.1007/s11263-019-01228-7
[24]	HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]. 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 2261–2269.