Dense Heavy Parameter Lightweight Transformer Model for CT Image Recognition of COVID-19

ZHOU Tao; YE Xinyu; LIU Fengzhen; LU Huiling; ZHOU Jingce; DU Yuhu

doi:10.11999/JEIT221180

Volume 45 Issue 10

Oct. 2023

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2023 > 45(10): 3520-3528

ZHOU Tao, YE Xinyu, LIU Fengzhen, LU Huiling, ZHOU Jingce, DU Yuhu. Dense Heavy Parameter Lightweight Transformer Model for CT Image Recognition of COVID-19[J]. Journal of Electronics & Information Technology, 2023, 45(10): 3520-3528. doi: 10.11999/JEIT221180

Citation:

ZHOU Tao, YE Xinyu, LIU Fengzhen, LU Huiling, ZHOU Jingce, DU Yuhu. Dense Heavy Parameter Lightweight Transformer Model for CT Image Recognition of COVID-19[J]. Journal of Electronics & Information Technology, 2023, 45(10): 3520-3528. doi: 10.11999/JEIT221180

Citation:

PDF( 5603 KB)

Dense Heavy Parameter Lightweight Transformer Model for CT Image Recognition of COVID-19

doi: 10.11999/JEIT221180

ZHOU Tao^{1, 4},
YE Xinyu^{1, 4
,
,},
LIU Fengzhen^{1, 4},
LU Huiling²,
ZHOU Jingce³,
DU Yuhu^{1, 4}

1.
Department of Computer Science and Engineering, North Minzu University, Yinchuan 750021, China
2.
School of Science, Ningxia Medical University, Yinchuan 750004, China
3.
School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122, China
4.
Key Laboratory of Image and Graphics Intelligent Processing of State Ethnic Affairs Commission, North Minzu University, Yinchuan 750021, China

Funds: The National Natural Science Foundation of China (62062003), The National Natural Science Foundation of Ningxia Autonomous Region (2022AAC03149), The Key Research and Development Projects of Ningxia Autonomous Region (2020BEB04022), 2022 graduate innovation Project of North Minzu University for Nationalities (YCX22198)

Received Date: 2022-09-08
Rev Recd Date: 2022-10-31

Available Online: 2022-11-07

Publish Date: 2023-10-31

Abstract

Abstract

COrona VIrus Disease 2019(COVID-19) is a serious threat to human health, deep learning computer aided diagnosis method can effectively improve the diagnosis efficiency. But deep learning models have usually complex structure which have large number of parameters and computations, therefore, a Dense Reparameter Lightweight Transformer(DRLTransformer) for COVID-19 CT recognition is proposed. Firstly, reparameter dense block and hierarchical Transformer are proposed to improve lightweight degree of model, which can improve computation speed and reduce parameters without losing model performance. Secondly, in order to fully extract global and local information of lesions, using hierarchical Transformer enhance global attention on local feature relevance, which use grouping to extract global features and fused between different groups to obtain multi-level information, and then information fusion is used to further improve interaction of intra group and inter group features. In addition, all global features are aggregated to achieve deep fusion of deep and shallow features. Finally, comparative experiments in COVID-19 CT dataset, the results show that the parameters and computations of DRLTransformer are 1.47 M and 81.232 M. Compared to Dense Convolutional Network(DenseNet), parameters are reduced by 29 times and computations are reduced by 23 times. The model proposed in this paper has positive implications for computer aided diagnosis of COVID-19 and provides a new idea for lightweight deep learning model.
- COrona VIrus Disease 2019(COVID-19),
- DenseNet,
- Reparameter dense block,
- Hierarchical Transformer,
- Lightweight

FullText(HTML)

References(21)

References

[1]	KAYA A T and AKMAN B. Mediastinal lymph node enlargement in COVID-19: Relationships with mortality and CT findings[J]. Heart & Lung, 2022, 54: 19–26. doi: 10.1016/j.hrtlng.2022.03.006
[2]	周涛, 刘赟璨, 陆惠玲, 等. ResNet及其在医学图像处理领域的应用: 研究进展与挑战[J]. 电子与信息学报, 2022, 44(1): 149–167. doi: 10.11999/JEIT210914 ZHOU Tao, LIU Yuncan, LU Huiling, et al. ResNet and its application to medical image processing: Research progress and challenges[J]. Journal of Electronics &Information Technology, 2022, 44(1): 149–167. doi: 10.11999/JEIT210914
[3]	SONG Ying, ZHENG Shuangjia, LI Liang, et al. Deep learning enables accurate diagnosis of novel coronavirus (COVID-19) with CT images[J]. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2021, 18(6): 2775–2780. doi: 10.1109/TCBB.2021.3065361
[4]	YE Qinghao, GAO Yuan, DING Weiping, et al. Robust weakly supervised learning for COVID-19 recognition using multi-center CT images[J]. Applied Soft Computing, 2022, 116: 108291. doi: 10.1016/j.asoc.2021.108291
[5]	RAHHAL M, BAZI Y, JOMAA R M, et al. Deep learning approach for COVID-19 detection in computed tomography images[J]. Computers, Materials & Continua, 2021, 67(2): 2093–2110. doi: 10.32604/CMC.2021.014956
[6]	KONG Lingzhi and CHENG Jinyong. Classification and detection of COVID-19 X-Ray images based on DenseNet and VGG16 feature fusion[J]. Biomedical Signal Processing and Control, 2022, 77: 103772. doi: 10.1016/j.bspc.2022.103772
[7]	GARG A, SALEHI S, LA ROCCA M, et al. Efficient and visualizable convolutional neural networks for COVID-19 classification using Chest CT[J]. Expert Systems with Applications, 2022, 195: 116540. doi: 10.1016/j.eswa.2022.116540
[8]	MONTALBO F J P. Diagnosing Covid-19 chest x-rays with a lightweight truncated DenseNet with partial layer freezing and feature fusion[J]. Biomedical Signal Processing and Control, 2021, 68: 102583. doi: 10.1016/j.bspc.2021.102583
[9]	ASIF S, ZHAO Ming, TANG Fengxiao, et al. A deep learning-based framework for detecting COVID-19 patients using chest X-rays[J]. Multimedia Systems, 2022, 28(4): 1495–1513. doi: 10.1007/s00530-022-00917-7
[10]	CHAKRABORTY M, DHAVALE S V, and INGOLE J. Corona-Nidaan: Lightweight deep convolutional neural network for chest X-Ray based COVID-19 infection detection[J]. Applied Intelligence, 2021, 51(5): 3026–3043. doi: 10.1007/s10489-020-01978-9
[11]	DEHKORDI H A, KASHIANI H, IMANI A A H, et al. Lightweight local transformer for COVID-19 detection using chest CT scans[C]. The 2021 11th International Conference on Computer Engineering and Knowledge (ICCKE), Tehran, Iran, 2021: 328–333.
[12]	PARK S, KIM G, OH Y, et al. Multi-task vision Transformer using low-level chest X-ray feature corpus for COVID-19 diagnosis and severity quantification[J]. Medical Image Analysis, 2022, 75: 102299. doi: 10.1016/j.media.2021.102299
[13]	SOARES E, ANGELOV P, BIASO S, et al. SARS-CoV-2 CT-scan dataset: A large dataset of real patients CT scans for SARS-CoV-2 identification[EB/OL]. https://www.medrxiv.org/content/10.1101/2022.04.24.20078584, 2020.
[14]	ZHAO Jinyu, ZHANG Yichen, HE Xuehai, et al. COVID-CT-dataset: A CT scan dataset about COVID-19[EB/OL].https://arxiv.org/abs/2003.13865v1, 2020.
[15]	ZHOU Tao, YE Xinyu, LU Huiling, et al. Dense convolutional network and its application in medical image analysis[J]. BioMed Research International, 2022, 2022: 2384830. doi: 10.1155/2022/2384830
[16]	RADOSAVOVIC I, KOSARAJU R P, GIRSHICK R, et al. Designing network design spaces[C]. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 10425–10433.
[17]	PENG Zhiliang, HUANG Wei, GU Shanzhi, et al. Conformer: Local features coupling global representations for visual recognition[C]. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, Canada, 2021: 357–366,
[18]	DING Xiaohan, ZHANG Xiangyu, ZHOU Yizhuang, et al. Scaling up your kernels to 31x31: Revisiting large kernel design in CNNs[J]. arXiv preprint arXiv: 2203.06717, 2022.
[19]	RAVI V, NARASIMHAN H, CHAKRABORTY C, et al. Deep learning-based meta-classifier approach for COVID-19 classification using CT scan and chest X-ray images[J]. Multimedia Systems, 2022, 28(4): 1401–1415. doi: 10.1007/s00530-021-00826-1
[20]	JAVADIMOGHADDAM S and GHOLAMALINEJAD H. A novel deep learning based method for COVID-19 detection from CT image[J]. Biomedical Signal Processing and Control, 2021, 70: 102987. doi: 10.1016/j.bspc.2021.102987
[21]	ARDAKANI A A, KANAFI A R, ACHARYA U R, et al. Application of deep learning technique to manage COVID-19 in routine clinical practice using CT images: Results of 10 convolutional neural networks[J]. Computers in Biology and Medicine, 2020, 121: 103795. doi: 10.1016/j.compbiomed.2020.103795