Hierarchical Fusion Multi-Instance Learning for Weakly Supervised Pathological Image Classification

CHEN Xiaohe; ZHANG Jiaang; LI Lingzhi; LI Guixiu; OU Zirong; BAO Yuehua; LIU Xinxin; YU Qiuchen; MA Yuhan; ZHAO Keyu; BAI Hua

doi:10.11999/JEIT250726

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2026 >

CHEN Xiaohe, ZHANG Jiaang, LI Lingzhi, LI Guixiu, OU Zirong, BAO Yuehua, LIU Xinxin, YU Qiuchen, MA Yuhan, ZHAO Keyu, BAI Hua. Hierarchical Fusion Multi-Instance Learning for Weakly Supervised Pathological Image Classification[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250726

Citation:

CHEN Xiaohe, ZHANG Jiaang, LI Lingzhi, LI Guixiu, OU Zirong, BAO Yuehua, LIU Xinxin, YU Qiuchen, MA Yuhan, ZHAO Keyu, BAI Hua. Hierarchical Fusion Multi-Instance Learning for Weakly Supervised Pathological Image Classification[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250726

Citation:

CHEN Xiaohe, ZHANG Jiaang, LI Lingzhi, LI Guixiu, OU Zirong, BAO Yuehua, LIU Xinxin, YU Qiuchen, MA Yuhan, ZHAO Keyu, BAI Hua. Hierarchical Fusion Multi-Instance Learning for Weakly Supervised Pathological Image Classification[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250726

PDF( 3740 KB)

Hierarchical Fusion Multi-Instance Learning for Weakly Supervised Pathological Image Classification

doi: 10.11999/JEIT250726 cstr: 32379.14.JEIT250726

1.
The First Affiliated Hospital of Ningbo University, Ningbo 315010, China
2.
College of Artificial Intelligence, China University of Petroleum (Beijing), Beijing 102249, China
3.
Beijing Research Institute of Automation for Machinery Industry Co., Ltd., Beijing 100120, China

Funds: The Ningbo Major Research and Development Plan Project (2024Z228), The Health Major Science and Technology Planning Project of Zhejiang Province, China (WKJ-ZJ-2411), The Public Welfare Projects of Ningbo, China (2022S065), The Project of Ningbo Leading Medical & Health Discipline (2022-F23)

Accepted Date: 2026-01-22
Rev Recd Date: 2026-01-22

Available Online: 2026-02-12

Abstract

Abstract

Objective As the mortality rates of cancer in China continue to rise, the significance of pathological image classification in cancer diagnosis is increasingly recognized. Pathological images are characterized by a multi-level structure. However, most existing methods primarily focus on the highest resolution of pathological images or employ simple feature concatenation strategies to fuse multi-scale information, failing to effectively utilize the multi-level information inherent in these images. Furthermore, existing methods typically employ random pseudo-bag division strategies to address the challenge of high-resolution pathological images. However, due to the sparsity of cancerous regions in positive slides, such random sampling often results in incorrect pseudo-labels and low signal-to-noise ratios, thereby posing additional challenges to classification accuracy. To address these issues, this study proposes a Hierarchical Fusion Multi-Instance Learning (HFMIL) method, integrating multi-level feature fusion with a pseudo-bag division strategy based on an attention evaluation function. It is designed to enhance the accuracy and interpretability of pathological image classification, thereby providing a more effective tool for clinical diagnosis. Methods A weakly supervised classification method based on a multi-level model is proposed in this study to leverage the multi-level characteristics of pathological images, thereby enhancing the performance of cancer pathological image classification. The proposed method is composed of three core steps. Initially, multi-level feature extraction is performed. Blank areas are removed from pathological images, and low-resolution images are segmented into image patches and then mapped to their corresponding high-resolution patches through index mapping. Semantic features are extracted to capture multi-level information, including low-resolution tissue structures and high-resolution cellular details. Subsequently, a pseudo-bag division method based on an attention evaluation function is employed. Classification scores for each image patch are computed through class activation mapping (CAM) to assess the importance of low-resolution features. Patches are ranked by scores, and potential positive features are selected to form pseudo-bags, while low-scoring features are discarded to ensure that pseudo-bags contain information relevant to pseudo-labels. Corresponding high-resolution pseudo-bags are then generated via feature index mapping, effectively addressing the issues of incorrect pseudo-labels and low signal-to-noise ratios. Finally, a two-stage classification model is developed. In the first stage, low-resolution pseudo-bags are aggregated using a gated attention mechanism for preliminary classification. In the second stage, a cross-attention mechanism is employed to fuse the most contributory low-resolution features with their corresponding high-resolution counterparts. The fused features are then concatenated with the aggregated high-resolution pseudo-bags to form a comprehensive image-level feature representation, which is input into a classifier for final prediction. Model training is conducted using a two-stage loss function, combining cross-entropy losses from low-resolution classification and overall classification to ensure effective integration of multi-level information. The method is experimentally validated on three pathological image datasets, demonstrating its effectiveness in weakly supervised pathological image classification tasks. Results and Discussions The proposed method is compared with several state-of-the-art weakly supervised classification methods, including ABMIL, CLAM, TransMIL, and DTFD. Evaluations are conducted on three pathological image datasets: the publicly available Camelyon16 and TCGA-LUNG datasets, and a private skin cancer dataset, NBU-Skin. Experimental results indicate that the proposed method achieves significant performance improvements on the test sets. On the Camelyon16 dataset, a classification accuracy of 88.3% and an AUC value of 0.979 are obtained (Table 2). On the TCGA-LUNG dataset, a classification accuracy of 86.0% and an AUC value of 0.931 are obtained (Table 2), surpassing comparative methods. On the NBU-Skin dataset, a classification accuracy of 90.5% and an AUC value of 0.976 are achieved for multi-classification tasks (Table 2).To further validate the effectiveness of the proposed approach, ablation studies are conducted to assess the necessity of the multi-level feature fusion and pseudo-bag division modules. The results demonstrate that the combination of these modules enhances classification performance. For instance, on the skin cancer dataset, the removal of the pseudo-bag division module was observed to reduce classification accuracy from 93.8% to 90.7%, and the subsequent removal of the multi-level feature fusion module further reduced it to 80.0% (Table 3). These findings collectively confirm the effectiveness of each component in the method. Conclusions A weakly supervised pathological image classification algorithm is proposed in this study, integrating multi-level feature fusion and an attention-based pseudo-bag division method. This approach effectively leverages multi-level information within pathological images and mitigates challenges related to incorrect pseudo-labels and low signal-to-noise ratios. Experimental results demonstrate that the proposed method outperforms existing approaches in terms of classification accuracy and AUC across three pathological image datasets. The primary contributions include: (1) A multi-level feature extraction and fusion strategy. Unlike existing strategies that primarily focus on the highest resolution or employ simple feature concatenation, this method deeply fuses feature information across different levels via a Cross-Attention Mechanism, effectively utilizing multi-scale information. (2) A pseudo-bag division method based on an attention evaluation function. By scoring features to identify potential positive regions and restructuring training samples via pseudo-bag division, this method not only maximizes the correctness of pseudo-labels through a top-k mechanism but also improves the signal-to-noise ratio by discarding low-scoring background noise. (3) Superior performance over all comparative models. The accuracy of weakly supervised pathological image classification is significantly improved, providing new insights for computer-aided cancer diagnosis. The following future research directions are proposed: (1) optimization of cross-level attention mechanisms; (2) extension of the framework to other medical imaging tasks, such as prognosis prediction or lesion segmentation; (3) designing more efficient feature extraction and fusion methods, and exploring their applications in other disease types and tasks, to better meet clinical needs.
- Pathological Image,
- Multiple Instance Learning,
- Deep Learning,
- Artificial Intelligence

FullText(HTML)

References(24)

References

[1]	HAN Bingfeng, ZHENG Rongshou, ZENG Hongmei, et al. Cancer incidence and mortality in China, 2022[J]. Journal of the National Cancer Center, 2024, 4(1): 47–53. doi: 10.1016/j.jncc.2024.01.006.
[2]	姜梦琦, 韩昱晨, 傅小龙. 基于人工智能的H-E染色全切片病理学图像分析在肺癌研究中的进展[J]. 中国癌症杂志, 2024, 34(3): 306–315. doi: 10.19401/j.cnki.1007-3639.2024.03.009. JIANG Mengqi, HAN Yuchen, and FU Xiaolong. Research progress on H-E stained whole slide image analysis by artificial intelligence in lung cancer[J]. China Oncology, 2024, 34(3): 306–315. doi: 10.19401/j.cnki.1007-3639.2024.03.009.
[3]	王钰萌, 刘振丙, 刘再毅. 隐私保护的联邦弱监督组织病理学亚型分类方法[J/OL]. https://jeit.ac.cn/cn/article/doi/10.11999/JEIT250842, 2025. WANG Yumeng, LIU Zhenbing, and LIU Zaiyi. Privacy-preserving federated weakly-supervised learning for cancer subtyping on histopathology images[J/OL]. https://jeit.ac.cn/cn/article/doi/10.11999/JEIT250842, 2025.
[4]	金怀平, 薛飞跃, 李振辉, 等. 基于病理图像集成深度学习的胃癌预后预测方法[J]. 电子与信息学报, 2023, 45(7): 2623–2633. doi: 10.11999/JEIT220655. JIN H P, XUE F Y, LI Z H, et al. Prognostic prediction of gastric cancer based on ensemble deep learning of pathological images[J]. Journal of Electronics & Information Technology, 2023, 45(7): 2623–2633. doi: 10.11999/JEIT220655.
[5]	FEI Manman, ZHANG Xin, CHEN Dongdong, et al. Whole slide cervical cancer classification via graph attention networks and contrastive learning[J]. Neurocomputing, 2025, 613: 128787. doi: 10.1016/j.neucom.2024.128787.
[6]	ZHANG Jiawei, SUN Zhanquan, WANG Kang, et al. Prognosis prediction based on liver histopathological image via graph deep learning and transformer[J]. Applied Soft Computing, 2024, 161: 111653. doi: 10.1016/j.asoc.2024.111653.
[7]	LI Mingze, ZHANG Bingbing, SUN Jian, et al. Weakly supervised breast cancer classification on WSI using transformer and graph attention network[J]. International Journal of Imaging Systems and Technology, 2024, 34(4): e23125. doi: 10.1002/ima.23125.
[8]	WANG Fuying, XIN Jiayi, ZHAO Weiqin, et al. TAD-graph: Enhancing whole slide image analysis via task-aware subgraph disentanglement[J]. IEEE Transactions on Medical Imaging, 2025, 44(6): 2683–2695. doi: 10.1109/TMI.2025.3545680.
[9]	WU Kun, JIANG Zhiguo, TANG Kunming, et al. Pan-cancer histopathology WSI pre-training with position-aware masked autoencoder[J]. IEEE Transactions on Medical Imaging, 2025, 44(4): 1610–1623. doi: 10.1109/TMI.2024.3513358.
[10]	张印辉, 张金凯, 何自芬, 等. 全局感知与稀疏特征关联图像级弱监督病理图像分割[J]. 电子与信息学报, 2024, 46(9): 3672–3682. doi: 10.11999/JEIT240364. ZHANG Yinhui, ZHANG Jinkai, HE Zifen, et al. Global perception and sparse feature associate image-level weakly supervised pathological image segmentation[J]. Journal of Electronics & Information Technology, 2024, 46(9): 3672–3682. doi: 10.11999/JEIT240364.
[11]	YAN Rui, LV Zhilong, YANG Zhidong, et al. Sparse and hierarchical transformer for survival analysis on whole slide images[J]. IEEE Journal of Biomedical and Health Informatics, 2024, 28(1): 7–18. doi: 10.1109/JBHI.2023.3307584.
[12]	MA Yingfan, LUO Xiaoyuan, FU Kexue, et al. Transformer-based video-structure multi-instance learning for whole slide image classification[C]. Proceedings of the 38th AAAI Conference on Artificial Intelligence, Vancouver, Canada, 2024: 14263–14271. doi: 10.1609/aaai.v38i13.29338.
[13]	ILSE M, TOMCZAK J, and WELLING M. Attention-based deep multiple instance learning[C]. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, 2018: 2127–2136.
[14]	LU M Y, WILLIAMSON D F K, CHEN T Y, et al. Data-efficient and weakly supervised computational pathology on whole-slide images[J]. Nature Biomedical Engineering, 2021, 5(6): 555–570. doi: 10.1038/s41551-020-00682-w.
[15]	SHAO Zhuchen, BIAN Hao, CHEN Yang, et al. TransMIL: Transformer based correlated multiple instance learning for whole slide image classification[C]. Proceedings of the 35th International Conference on Neural Information Processing Systems, 2021: 164. (查阅网上资料, 未找到本条文献出版地信息, 请确认).
[16]	ZHANG Hongrun, MENG Yanda, ZHAO Yitian, et al. DTFD-MIL: Double-tier feature distillation multiple instance learning for histopathology whole slide image classification[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 18780–18790. doi: 10.1109/CVPR52688.2022.01824.
[17]	LI Bin, LI Yin, and ELICEIRI K W. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 14313–14323. doi: 10.1109/CVPR46437.2021.01409.
[18]	CHEN Y C and LU C S. RankMix: Data augmentation for weakly supervised learning of classifying whole slide images with diverse sizes and imbalanced categories[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, 2023: 23936–23945. doi: 10.1109/CVPR52729.2023.02292.
[19]	LIU Pei, JI Luping, ZHANG Xinyu, et al. Pseudo-bag mixup augmentation for multiple instance learning-based whole slide image classification[J]. IEEE Transactions on Medical Imaging, 2024, 43(5): 1841–1852. doi: 10.1109/TMI.2024.3351213.
[20]	YANG Jiawei, CHEN Hanbo, ZHAO Yu, et al. ReMix: A general and efficient framework for multiple instance learning based whole slide image classification[C]. Proceedings of the 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, Singapore, Singapore, 2022: 35–45. doi: 10.1007/978-3-031-16434-7_4.
[21]	BEJNORDI B E, VETA M, VAN DIEST P J, et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer[J]. JAMA, 2017, 318(22): 2199–2210. doi: 10.1001/jama.2017.14585.
[22]	TOMCZAK K, CZERWIŃSKA P, and WIZNEROWICZ M. The Cancer Genome Atlas (TCGA): An immeasurable source of knowledge[J]. Contemporary Oncology, 2015, 19(1A): A68–A77. doi: 10.5114/wo.2014.47136.
[23]	ZHOU S K, RUECKERT D, and FICHTINGER G. Handbook of Medical Image Computing and Computer Assisted Intervention[M]. London: Academic Press, 2020: 521–546.
[24]	LOU Wei, LI Guanbin, WAN Xiang, et al. Multi-modal denoising diffusion pre-training for whole-slide image classification[C]. Proceedings of the 32nd ACM International Conference on Multimedia, Melbourne, Australia, 2024: 10804–10813. doi: 10.1145/3664647.3680882.