高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于病理图像集成深度学习的胃癌预后预测方法

金怀平 薛飞跃 李振辉 陶海波 王彬

金怀平, 薛飞跃, 李振辉, 陶海波, 王彬. 基于病理图像集成深度学习的胃癌预后预测方法[J]. 电子与信息学报, 2023, 45(7): 2623-2633. doi: 10.11999/JEIT220655
引用本文: 金怀平, 薛飞跃, 李振辉, 陶海波, 王彬. 基于病理图像集成深度学习的胃癌预后预测方法[J]. 电子与信息学报, 2023, 45(7): 2623-2633. doi: 10.11999/JEIT220655
JIN Huaiping, XUE Feiyue, LI Zhenhui, TAO Haibo, WANG Bin. Prognostic Prediction of Gastric Cancer Based on Ensemble Deep Learning of Pathological Images[J]. Journal of Electronics & Information Technology, 2023, 45(7): 2623-2633. doi: 10.11999/JEIT220655
Citation: JIN Huaiping, XUE Feiyue, LI Zhenhui, TAO Haibo, WANG Bin. Prognostic Prediction of Gastric Cancer Based on Ensemble Deep Learning of Pathological Images[J]. Journal of Electronics & Information Technology, 2023, 45(7): 2623-2633. doi: 10.11999/JEIT220655

基于病理图像集成深度学习的胃癌预后预测方法

doi: 10.11999/JEIT220655
基金项目: 国家自然科学基金(82001986),云南省科技厅-昆明医科大学应用基础研究联合专项项目(202101AY070001-181),云南省应用基础研究计划项目(202101AW070001)
详细信息
    作者简介:

    金怀平:男,博士,副教授,研究方向为医学图像分析处理、智能数据解析与预测建模

    薛飞跃:男,硕士生,研究方向为医学图像分析处理、机器视觉

    李振辉:男,博士,主任医师,研究方向为肿瘤影像诊断、医学影像大数据与人工智能

    陶海波:女,主任医师,研究方向为肿瘤影像诊断、医学影像大数据与人工智能

    王彬:女,博士,副教授,研究方向为图像分析与处理、类脑计算

    通讯作者:

    金怀平 jinhuaiping@126.com

  • 中图分类号: TN911.73; TP391.41

Prognostic Prediction of Gastric Cancer Based on Ensemble Deep Learning of Pathological Images

Funds: The National Natural Science Foundation of China (82001986), The Joint Special Funds for the Department of Science and Technology of Yunnan Province-Kunming Medical University(202101AY070001-181), The Applied Basic Research Projects of Yunnan Province, China, Outstanding Youth Foundation(202101AW070001)
  • 摘要: 病理图像分析对胃癌的诊断和预后具有重要意义,但在临床应用上仍然面临着目视阅片一致性低、多分辨率图像差异大等挑战。为此,该文提出一种基于病理图像集成深度学习的胃癌预后预测方法。首先,对患者不同分辨率下的病理图像进行切分、筛选等预处理;然后,采用ResNet, MobileNetV3, EfficientNetV2深度学习方法分别对不同分辨率下的切片(Tile)进行深度特征提取和融合,以此获得患者层面(Patient-level)的单分辨率子分类器预测结果;最终,采用双重集成策略对不同分辨率下异质子分类器预测结果进行融合以获得患者层面的预后预测结果。实验中收集了250例胃癌患者的组织病理图像,并以远处转移预测为例进行验证,实验结果表明,所提方法在测试集上的预测准确率为89.10%,敏感度为89.57%,特异度为88.61%,马修斯相关系数为78.19%,相比于单模型预测结果获得了显著提升,可为胃癌患者的治疗和预后提供重要参考。
  • 图  1  基于病理图像集成深度学习的胃癌预后预测方法整体结构图

    图  2  SE模块示意图

    图  3  MBConv和Fused-MBConv示意图

    图  4  切片信息融合方法示意图

    图  5  滑动窗口切分过程

    图  6  不同类型切片典型示意图

    图  7  不同网络在相同输入图像下的Grad-CAM图

    图  8  不同模型在患者层面远处预测混淆矩阵

    图  9  不同方法在患者层面远处预测混淆矩阵

    表  1  250例胃癌患者临床随访信息

    指标特征数量
    年龄≤5061
    51~70153
    >7036
    性别男性163
    女性87
    远处转移发生转移125
    未发生转移125
    病理类型腺癌177
    黏液腺癌11
    印戎细胞癌56
    其他6
    Lauren分型肠型27
    弥漫型71
    混合型32
    未标明120
    下载: 导出CSV

    表  2  不同量级子分类器网络在切片(10×分辨率)层面的预测性能(%)对比

    方法ACCSENSPEMCC
    ResNet-1876.4276.7276.1052.81
    ResNet-3478.3775.7981.0856.90
    ResNet-5079.6779.4379.8659.29
    MobileNetV3-small77.0371.3083.0454.62
    MobileNetV3-large79.2175.3882.8758.46
    EfficientNetV2-s81.1885.7876.3662.49
    EfficientNetV2-m82.9485.2780.5065.88
    EfficientNetV2-l83.6986.5280.8567.48
    下载: 导出CSV

    表  3  不同量级子分类器网络在切片(40×分辨率)层面的预测性能(%)对比

    方法ACCSENSPEMCC
    ResNet-1874.8473.5576.1949.73
    ResNet-3476.3375.9576.7352.66
    ResNet-5078.0576.7879.3856.15
    MobileNetV3-small73.4674.3072.5846.89
    MobileNetV3-large76.2478.0174.4752.52
    EfficientNetV2-s79.5882.8176.2059.19
    EfficientNetV2-m80.2382.9377.3960.45
    EfficientNetV2-l81.1683.9478.2462.32
    下载: 导出CSV

    表  4  不同分辨率下子分类器网络与集成模型在切片层面的预测性能(%)对比

    方法ACCSENSPEMCC
    ResNet-50(10×)79.6779.4379.8659.29
    MobileNetV3-large(10×)79.2175.3882.8758.46
    EfficientNetV2-l(10×)83.6986.5280.8567.48
    集成模型(10×)86.1687.6484.6672.34
    ResNet-50(40×)78.0576.7879.3856.15
    MobileNetV3-large(40×)76.2478.0174.4752.52
    EfficientNetV2-l(40×)81.1683.9478.2462.32
    集成模型(40×)84.6486.4282.8069.29
    下载: 导出CSV

    表  5  不同模型在患者层面的预测性能(%)对比

    方法ACCSENSPEMCC
    ResNet-5083.7883.0684.5367.57
    MobileNetV3-large
    EfficientNetV2-l
    80.21
    85.64
    76.27
    87.86
    84.35
    83.31
    60.74
    71.29
    10×分辨率集成模型87.5488.9686.0875.09
    40×分辨率集成模型86.1987.2385.4472.70
    本文89.1089.5788.6178.19
    下载: 导出CSV

    表  6  不同方法在患者层面的预测性能(%)对比

    方法ACCSENSPEMCC
    文献[14]
    文献[15]
    81.33
    85.17
    80.70
    86.79
    81.99
    83.54
    62.66
    70.38
    文献[20]83.1382.6383.6566.26
    本文89.1089.5788.6178.19
    下载: 导出CSV
  • [1] SUNG H, FERLAY J, SIEGEL R L, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries[J]. CA:A Cancer Journal for Clinicians, 2021, 71(3): 209–249. doi: 10.3322/caac.21660
    [2] DAGOGO-JACK I and SHAW A T. Tumour heterogeneity and resistance to cancer therapies[J]. Nature Reviews Clinical Oncology, 2018, 15(2): 81–94. doi: 10.1038/nrclinonc.2017.166
    [3] TAKTAK A F G and FISHER A C. Outcome Prediction in Cancer[M]. Amsterdam: Elsevier, 2006: 16-17.
    [4] GORLIA T, VAN DEN BENT M J, HEGI M E, et al. Nomograms for predicting survival of patients with newly diagnosed glioblastoma: Prognostic factor analysis of EORTC and NCIC trial 26981-22981/CE. 3[J]. The Lancet Oncology, 2008, 9(1): 29–38. doi: 10.1016/S1470-2045(07)70384-4
    [5] YU Chaoran and ZHANG Yujie. Development and validation of prognostic nomogram for young patients with gastric cancer[J]. Annals of Translational Medicine, 2019, 7(22): 641. doi: 10.21037/atm.2019.10.77
    [6] ZHANG Yanbo and YU Hengyong. Convolutional neural network based metal artifact reduction in X-ray computed tomography[J]. IEEE Transactions on Medical Imaging, 2018, 37(6): 1370–1381. doi: 10.1109/TMI.2018.2823083
    [7] HAMEED A A, KARLIK B, and SALMAN M S. Back-propagation algorithm with variable adaptive momentum[J]. Knowledge-Based Systems, 2016, 114: 79–87. doi: 10.1016/j.knosys.2016.10.001
    [8] 徐莹莹, 沈红斌. 基于模式识别的生物医学图像处理研究现状[J]. 电子与信息学报, 2020, 42(1): 201–213. doi: 10.11999/JEIT190657

    XU Yingying and SHEN Hongbin. Review of research on biomedical image processing based on pattern recognition[J]. Journal of Electronics &Information Technology, 2020, 42(1): 201–213. doi: 10.11999/JEIT190657
    [9] GREENSPAN H, VAN GINNEKEN B, and SUMMERS R M. Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique[J]. IEEE Transactions on Medical Imaging, 2016, 35(5): 1153–1159. doi: 10.1109/TMI.2016.2553401
    [10] LUO J, WU Min, GOPUKUMAR D, et al. Big data application in biomedical research and health care: A literature review[J]. Biomedical Informatics Insights, 2016, 8: 1–10. doi: 10.4137/BII.S31559
    [11] ESTEVA A, KUPREL B, NOVOA R A, et al. Dermatologist-level classification of skin cancer with deep neural networks[J]. Nature, 2017, 542(7639): 115–118. doi: 10.1038/nature21056
    [12] JIANG Yuming, JIN Cheng, YU Heng, et al. Development and validation of a deep learning CT signature to predict survival and chemotherapy benefit in gastric cancer: A multicenter, retrospective study[J]. Annals of Surgery, 2021, 274(6): e1153–e1161. doi: 10.1097/SLA.0000000000003778
    [13] 陈雯, 王旭, 段辉宏, 等. 深度学习在癌症预后预测模型中的应用研究[J]. 生物医学工程学杂志, 2020, 37(5): 918–929. doi: 10.7507/1001-5515.201909066

    CHEN Wen, WANG Xu, DUAN Huihong, et al. Application of deep learning in cancer prognosis prediction model[J]. Journal of Biomedical Engineering, 2020, 37(5): 918–929. doi: 10.7507/1001-5515.201909066
    [14] KATHER J N, PEARSON A T, HALAMA N, et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer[J]. Nature Medicine, 2019, 25(7): 1054–1056. doi: 10.1038/s41591-019-0462-y
    [15] SKREDE O J, DE RAEDT S, KLEPPE A, et al. Deep learning for prediction of colorectal cancer outcome: A discovery and validation study[J]. The Lancet, 2020, 395(10221): 350–360. doi: 10.1038/s41591-019-0462-y
    [16] 杨昆, 常世龙, 王尉丞, 等. 基于sECANet通道注意力机制的肾透明细胞癌病理图像ISUP分级预测[J]. 电子与信息学报, 2022, 44(1): 138–148. doi: 10.11999/JEIT210900

    YANG Kun, CHANG Shilong, WANG Yucheng, et al. Predict the ISUP grade of clear cell renal cell carcinoma using pathological images based on sECANet Chanel attention[J]. Journal of Electronics &Information Technology, 2022, 44(1): 138–148. doi: 10.11999/JEIT210900
    [17] HE Kaiming, ZHANG Xianyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778.
    [18] HOWARD A, SANDLER M, CHEN Bo, et al. Searching for MobileNetV3[C]. 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Korea (South), 2019: 1314–1324.
    [19] TAN Mingxing and LE Q. EfficientNetV2: Smaller models and faster training[C/OL]. The 38th International Conference on Machine Learning, 2021: 10096–10106.
    [20] 于凌涛, 夏永强, 闫昱晟, 等. 利用卷积神经网络分类乳腺癌病理图像[J]. 哈尔滨工程大学学报, 2021, 42(4): 567–573. doi: 10.11990/jheu.201909052

    YU Lingtao, XIA Yongqiang, YAN Yusheng, et al. Breast cancer pathological image classification based on a convolutional neural network[J]. Journal of Harbin Engineering University, 2021, 42(4): 567–573. doi: 10.11990/jheu.201909052
    [21] SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization[J]. International Journal of Computer Vision, 2020, 128(2): 336–359. doi: 10.1007/s11263-019-01228-7
  • 加载中
图(9) / 表(6)
计量
  • 文章访问数:  652
  • HTML全文浏览量:  518
  • PDF下载量:  166
  • 被引次数: 0
出版历程
  • 收稿日期:  2022-05-20
  • 修回日期:  2022-08-22
  • 录用日期:  2022-08-25
  • 网络出版日期:  2022-08-30
  • 刊出日期:  2023-07-10

目录

    /

    返回文章
    返回