Global Perception and Sparse Feature Associate Image-level Weakly Supervised Pathological Image Segmentation

ZHANG Yinhui; ZHANG Jinkai; HE Zifen; LIU Jiacen; WU Lin; LI Zhenhui; CHEN Guangchen

doi:10.11999/JEIT240364

Volume 46 Issue 9

Sep. 2024

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2024 > 46(9): 3672-3682

ZHANG Yinhui, ZHANG Jinkai, HE Zifen, LIU Jiacen, WU Lin, LI Zhenhui, CHEN Guangchen. Global Perception and Sparse Feature Associate Image-level Weakly Supervised Pathological Image Segmentation[J]. Journal of Electronics & Information Technology, 2024, 46(9): 3672-3682. doi: 10.11999/JEIT240364

Citation:

ZHANG Yinhui, ZHANG Jinkai, HE Zifen, LIU Jiacen, WU Lin, LI Zhenhui, CHEN Guangchen. Global Perception and Sparse Feature Associate Image-level Weakly Supervised Pathological Image Segmentation[J]. Journal of Electronics & Information Technology, 2024, 46(9): 3672-3682. doi: 10.11999/JEIT240364

Citation:

ZHANG Yinhui, ZHANG Jinkai, HE Zifen, LIU Jiacen, WU Lin, LI Zhenhui, CHEN Guangchen. Global Perception and Sparse Feature Associate Image-level Weakly Supervised Pathological Image Segmentation[J]. Journal of Electronics & Information Technology, 2024, 46(9): 3672-3682. doi: 10.11999/JEIT240364

PDF( 10058 KB)

Global Perception and Sparse Feature Associate Image-level Weakly Supervised Pathological Image Segmentation

doi: 10.11999/JEIT240364

1.
Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology, Kunming 650500, China
2.
Department of Pathology, Yunnan Cancer Hospital, Kunming 650106, China
3.
Department of Radiology, Yunnan Cancer Hospital, Kunming 650106, China

Funds: The National Natural Science Foundation of China (62061022, 62171206)

Received Date: 2024-05-09
Rev Recd Date: 2024-07-17

Available Online: 2024-08-02

Publish Date: 2024-09-26

Abstract

Abstract

The weakly supervised semantic segmentation methods have been widely applied in the analysis of Whole Slide Images (WSI), saving a considerable amount of manual annotation costs. Addressing the issues of pixel instance independence, local inconsistency in segmentation results, and insufficient supervision from image-level labels in Multiple-Instance Learning (MIL) methods for pathological image analysis, a novel end-to-end MIL approach named DASMob-MIL is proposed in this paper. Firstly, to overcome the independence among pixel instances, features are extracted using a local perception network to establish local pixel dependencies, while a Global Information Perception Branch (GIPB) is constructed by cascading cross-attention modules to establish global pixel dependencies. Secondly, a Pixel-Adaptive Refinement (PAR) module is introduced to address the problem of local inconsistency in weakly supervised semantic segmentation results by constructing affinity kernels based on the similarity between multi-scale neighborhood local sparse features. Finally, a Deep Association Supervision (DAS) module is designed to optimize the training process by performing weighted fusion on the segmentation maps generated from multi-stage feature maps. Then, employing a weighted factor-associated loss function to mitigate the impact of insufficient supervision from weakly supervised image-level labels. Compared with other models, the DASMob-MIL model demonstrates advanced segmentation performance on the self-built colorectal cancer dataset YN-CRC and the public weakly supervised histopathology image dataset LUAD-HistoSeg-BC, with a model weight of only 14MB and an F1 score of 89.5% on the YN-CRC dataset, which was 3% higher than that of the advanced Multi-Layer Pseudo-Supervision (MLPS) model. Experimental results indicate that DASMob-MIL achieves pixel-level segmentation utilizing only image-level labels, effectively improving the segmentation performance of weakly supervised histopathological images.
- Weakly supervised semantic segmentation,
- Histopathological images,
- Multi-Instance Learning (MIL),
- Global perception,
- Sparse features

FullText(HTML)

References(24)

References

[1]	BRAY F, FERLAY J, SOERJOMATARAM I, et al. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries[J]. CA: A Cancer Journal for Clinicians, 2018, 68(6): 394–424. doi: 10.3322/caac.21492.
[2]	ZIDAN U, GABER M M, and ABDELSAMEA M M. SwinCup: Cascaded swin transformer for histopathological structures segmentation in colorectal cancer[J]. Expert Systems with Applications, 2023, 216: 119452. doi: 10.1016/j.eswa.2022.119452.
[3]	JIA Zhipeng, HUANG Xingyi, CHANG E I C, et al. Constrained deep weak supervision for histopathology image segmentation[J]. IEEE Transactions on Medical Imaging, 2017, 36(11): 2376–2388. doi: 10.1109/TMI.2017.2724070.
[4]	CAI Hongmin, YI Weiting, LI Yucheng, et al. A regional multiple instance learning network for whole slide image segmentation[C]. 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Las Vegas, USA, 2022: 922–928. doi: 10.1109/BIBM55620.2022.9995017.
[5]	LI Kailu, QIAN Ziniu, HAN Yingnan, et al. Weakly supervised histopathology image segmentation with self-attention[J]. Medical Image Analysis, 2023, 86: 102791. doi: 10.1016/j.media.2023.102791.
[6]	ZHOU Yanzhao, ZHU Yi, YE Qixiang, et al. Weakly supervised instance segmentation using class peak response[C]. The 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 3791–3800. doi: 10.1109/CVPR.2018.00399.
[7]	ZHONG Lanfeng, WANG Guotai, LIAO Xin, et al. HAMIL: High-resolution activation maps and interleaved learning for weakly supervised segmentation of histopathological images[J]. IEEE Transactions on Medical Imaging, 2023, 42(10): 2912–2923. doi: 10.1109/TMI.2023.3269798.
[8]	HAN Chu, LIN Jiatai, MAI Jinhai, et al. Multi-layer pseudo-supervision for histopathology tissue semantic segmentation using patch-level classification labels[J]. Medical Image Analysis, 2022, 80: 102487. doi: 10.1016/j.media.2022.102487.
[9]	DIETTERICH T G, LATHROP R H, and LOZANO-PÉREZ T. Solving the multiple instance problem with axis-parallel rectangles[J]. Artificial Intelligence, 1997, 89(1/2): 31–71. doi: 10.1016/S0004-3702(96)00034-3.
[10]	XU Gang, SONG Zhigang, SUN Zhuo, et al. CAMEL: A weakly supervised learning framework for histopathology image segmentation[C]. The 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Korea (South), 2019: 10681–10690. doi: 10.1109/ICCV.2019.01078.
[11]	徐金东, 赵甜雨, 冯国政, 等. 基于上下文模糊C均值聚类的图像分割算法[J]. 电子与信息学报, 2021, 43(7): 2079–2086. doi: 10.11999/JEIT200263. XU Jindong, ZHAO Tianyu, FENG Guozheng, et al. Image segmentation algorithm based on context fuzzy C-means clustering[J]. Journal of Electronics & Information Technology, 2021, 43(7): 2079–2086. doi: 10.11999/JEIT200263.
[12]	杭昊, 黄影平, 张栩瑞, 等. 面向道路场景语义分割的移动窗口变换神经网络设计[J]. 光电工程, 2024, 51(1): 230304. doi: 10.12086/oee.2024.230304. HANG Hao, HUANG Yingping, ZHANG Xurui, et al. Design of Swin Transformer for semantic segmentation of road scenes[J]. Opto-Electronic Engineering, 2024, 51(1): 230304. doi: 10.12086/oee.2024.230304.
[13]	QIAN Ziniu, LI Kailu, LAI Maode, et al. Transformer based multiple instance learning for weakly supervised histopathology image segmentation[C]. The 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, Singapore, 2022: 160–170. doi: 10.1007/978-3-031-16434-7_16.
[14]	HUANG Zilong, WANG Xinggang, HUANG Lichao, et al. CCNet: Criss-cross attention for semantic segmentation [C]. The IEEE/CVF International Conference on Computer Vision, Seoul, Korea (South), 2019: 603–612. doi: 10.1109/ICCV.2019.00069.
[15]	RU Lixiang, ZHAN Yibing, YU Baosheng, et al. Learning affinity from attention: End-to-end weakly-supervised semantic segmentation with transformers[C]. The 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 16825–16834. doi: 10.1109/CVPR52688.2022.01634.
[16]	XIE Yuhan, ZHANG Zhiyong, CHEN Shaolong, et al. Detect, Grow, Seg: A weakly supervision method for medical image segmentation based on bounding box[J]. Biomedical Signal Processing and Control, 2023, 86: 105158. doi: 10.1016/j.bspc.2023.105158.
[17]	KWEON H, YOON S H, KIM H, et al. Unlocking the potential of ordinary classifier: Class-specific adversarial erasing framework for weakly supervised semantic segmentation[C]. The 2021 IEEE/CVF International Conference on Computer Vision, Montreal, Canada, 2021: 6974–6983. doi: 10.1109/ICCV48922.2021.00691.
[18]	HOWARD A, SANDLER M, CHEN Bo, et al. Searching for MobileNetV3[C]. The 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Korea (South), 2019: 1314–1324. doi: 10.1109/ICCV.2019.00140.
[19]	VIOLA P, PLATT J C, and ZHANG Cha. Multiple instance boosting for object detection[J]. Proceedings of the 18th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2005: 1417–1424.
[20]	RONNEBERGER O, FISCHER P, and BROX T. U-net: Convolutional networks for biomedical image segmentation[C]. The 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 2015: 234–241. doi: 10.1007/978-3-319-24574-4_28.
[21]	SIMONYAN K and ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]. The 3rd International Conference on Learning Representations, San Diego, USA, 2015.
[22]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]. The 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770–778. doi: 10.1109/CVPR.2016.90.
[23]	MA Ningning, ZHANG Xiangyu, ZHENG Haitao, et al. ShuffleNet v2: Practical guidelines for efficient CNN architecture design[C]. The 15th European Conference on Computer Vision, Munich, Germany, 2018: 122–138. doi: 10.1007/978-3-030-01264-9_8.
[24]	TAN Mingxing and LE Q V. EfficientNetV2: Smaller models and faster training[C]. The 38th International Conference on Machine Learning, 2021: 10096–10106.