A Semantic-Enhanced Cybersecurity Named Entity Recognition Approach Oriented to Lightweight Adaptation of Large Language Models

HU Ze; XU Tongwu; YANG Hongyu

doi:10.11999/JEIT251260

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2026 >

HU Ze, XU Tongwu, YANG Hongyu. A Semantic-Enhanced Cybersecurity Named Entity Recognition Approach Oriented to Lightweight Adaptation of Large Language Models[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT251260

Citation:

HU Ze, XU Tongwu, YANG Hongyu. A Semantic-Enhanced Cybersecurity Named Entity Recognition Approach Oriented to Lightweight Adaptation of Large Language Models[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT251260

Citation:

PDF( 1223 KB)

A Semantic-Enhanced Cybersecurity Named Entity Recognition Approach Oriented to Lightweight Adaptation of Large Language Models

doi: 10.11999/JEIT251260 cstr: 32379.14.JEIT251260

HU Ze^1
,,
XU Tongwu¹,
YANG Hongyu^{1, 2
,
,}

1.
School of Safety Science and Engineering, Civil Aviation University of China, Tianjin 300300, China
2.
School of Computer Science and Artificial Intelligence, Civil Aviation University of China, Tianjin 300300, China

Funds: The National Natural Science Foundation of China (62201576, U2433205), The Supporting Fund of the National Natural Science Foundation of China (3122023PT10)

Received Date: 2025-11-26
Accepted Date: 2026-03-05
Rev Recd Date: 2026-03-05

Available Online: 2026-03-18

Abstract

Abstract

Objective Named Entity Recognition (NER) in cybersecurity is a core technology for threat intelligence analysis, vulnerability management, and security incident response. However, this field faces several challenges, including dense technical terminology, limited labeled data, dynamic entity categories, and highly complex semantic features. These factors reduce the domain adaptability and semantic fusion capacity of traditional deep learning models and existing Large Language Models (LLMs). To address these issues while meeting the need for lightweight deployment, a cybersecurity NER approach is proposed to strengthen domain semantic representation, improve rare-entity recognition, and support low-resource environments. This approach provides a reliable technical path for intelligent threat analysis in cybersecurity scenarios. Methods To address the complex semantic features of cybersecurity texts, a semantically enhanced, lightweight, and LLM-adaptable cybersecurity NER approach is proposed. LLM2Vec is used to achieve bidirectional semantic reconstruction of large-model decoders, and Low-Rank Adaptation (LoRA) is combined for low-rank fine-tuning. This design preserves deep semantic encoding capacity while substantially reducing the number of updated parameters. To address sparse keywords and severe noise interference in cybersecurity texts, a sparse gated attention mechanism is proposed to strengthen keyword-focused feature extraction. High-contribution cybersecurity terms are selected dynamically through global gating and sparse inference. A SecRoBERTa-based semantic enhancement module is also proposed. This module uses a domain-pretrained model to generate similar-word embeddings, improves feature robustness in small-sample settings, and reduces the difficulty of identifying out-of-vocabulary words and low-frequency terms. Finally, a Masked Conditional Random Field (MCRF) is used to constrain label transitions and ensure BIO-compliant output sequences, thereby achieving robust and consistent entity boundary prediction. Results and Discussions Extensive experiments are conducted on two public cybersecurity datasets, DNRTI and APTNER. The proposed approach achieves an F1 score of 91.91% on DNRTI, exceeding the previous state-of-the-art model by 2.14%. On APTNER, it achieves an F1 score of 80.37%, exceeding the best baseline by 2.97%. Ablation studies confirm the contribution of each key component. The sparse gated attention mechanism improves F1 by 3.57% over standard multi-head attention on DNRTI. The semantic enhancement module contributes a 2.32% increase in F1. The model also shows efficient training and inference, consistent with the goals of lightweight design. Conclusions A lightweight LLM-based adaptation approach is proposed for NER in the cybersecurity domain. The approach effectively addresses the limitations of existing LLM-based NER methods in domain adaptation and rare-entity recognition. By integrating LLM2Vec and LoRA for lightweight fine-tuning, a sparse gated attention mechanism for domain feature fusion, and a SecRoBERTa-based semantic enhancement module for similar-word precomputation, the proposed approach achieves strong performance on the DNRTI and APTNER datasets. This research provides an efficient technical path for NER tasks in low-resource cybersecurity scenarios and supports downstream tasks such as automated threat intelligence analysis.
- Named Entity Recognition (NER),
- Cybersecurity,
- Large Language Models (LLMs),
- Deep learning,
- Natural language processing

FullText(HTML)

References(29)

References

[1]	陈曙东, 欧阳小叶. 命名实体识别技术综述[J]. 无线电通信技术, 2020, 46(3): 251–260. doi: 10.3969/j.issn.1003-3114.2020.03.001. CHEN Shudong and OUYANG Xiaoye. Overview of named entity recognition technology[J]. Radio Communications Technology, 2020, 46(3): 251–260. doi: 10.3969/j.issn.1003-3114.2020.03.001.
[2]	SATVAT K, GJOMEMO R, and VENKATAKRISHNAN V N. Extractor: Extracting attack behavior from threat reports[C]. 2021 IEEE European Symposium on Security and Privacy (EuroS&P), Vienna, Austria, 2021: 598–615. doi: 10.1109/EuroSP51992.2021.00046.
[3]	GAO Chen, ZHANG Xuan, HAN Mengting, et al. A review on cyber security named entity recognition[J]. Frontiers of Information Technology & Electronic Engineering, 2021, 22(9): 1153–1168. doi: 10.1631/FITEE.2000286.
[4]	李永斌, 刘楝, 郑杰. 一种面向特定信息领域的大模型命名实体识别方法[J]. 电子与信息学报, 2026, 48(2): 662–672. doi: 10.11999/JEIT250764. LI Yongbin, LIU Lian, and ZHENG Jie. A method for named entity recognition in military intelligence domain using large language models[J]. Journal of Electronics & Information Technology, 2026, 48(2): 662–672. doi: 10.11999/JEIT250764.
[5]	HU Chenxi, WU Tao, LIU Chunsheng, et al. Joint contrastive learning and belief rule base for named entity recognition in cybersecurity[J]. Cybersecurity, 2024, 7(1): 19. doi: 10.1186/s42400-024-00206-y.
[6]	BEHNAMGHADER P, ADLAKHA V, MOSBACH M, et al. LLM2Vec: Large language models are secretly powerful text encoders[J]. arXiv preprint arXiv: 2404.05961, 2024. doi: 10.48550/arXiv.2404.05961.
[7]	HU E J, SHEN Yelong, WALLIS P, et al. LoRA: Low-rank adaptation of large language models[C]. The 10th International Conference on Learning Representations, 2022.
[8]	YI Feng, JIANG Bo, WANG Lu, et al. Cybersecurity named entity recognition using multi-modal ensemble learning[J]. IEEE Access, 2020, 8: 63214–63224. doi: 10.1109/ACCESS.2020.2984582.
[9]	MA Pingchuan, JIANG Bo, LU Zhigang, et al. Cybersecurity named entity recognition using bidirectional long short-term memory with conditional random fields[J]. Tsinghua Science and Technology, 2021, 26(3): 259–265. doi: 10.26599/TST.2019.9010033.
[10]	YI Junkai, LIU Yuan, JIANG Zhongbai, et al. Text command intelligent understanding for cybersecurity testing[J]. Electronics, 2024, 13(21): 4330. doi: 10.3390/electronics13214330.
[11]	胡泽, 李文君, 杨宏宇. 基于字符表示学习与时序边界扩散的网络安全实体识别方法[J]. 电子与信息学报, 2025, 47(5): 1554–1568. doi: 10.11999/JEIT240953. HU Ze, LI Wenjun, and YANG Hongyu. A cybersecurity entity recognition approach based on character representation learning and temporal boundary diffusion[J]. Journal of Electronics & Information Technology, 2025, 47(5): 1554–1568. doi: 10.11999/JEIT240953.
[12]	ZHANG Yunlong, LIU Jingju, ZHONG Xiaofeng, et al. SecLMNER: A framework for enhanced named entity recognition in multi-source cybersecurity data using large language models[J]. Expert Systems with Applications, 2025, 271: 126651. doi: 10.1016/j.eswa.2025.126651.
[13]	ZHANG Hao, WU Tingmin, ZHU Tianqing, et al. CyberLLaMA: A fine-tuned large language model for cybersecurity named entity recognition[J]. Knowledge-Based Systems, 2025, 328: 114183. doi: 10.1016/j.knosys.2025.114183.
[14]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]. The 31st International Conference on Neural Information Processing Systems, Long Beach, USA, 2017: 6000–6010.
[15]	Hugging Face. jackaduma/SecRoBERTa[EB/OL]. https://huggingface.co/jackaduma/SecRoBERTa, 2024.
[16]	才华, 冉越, 付强, 等. 多粒度文本感知分层特征交互的视觉定位方法[J]. 电子与信息学报, 2025, 47(11): 4594–4605. doi: 10.11999/JEIT250387. CAI Hua, RAN Yue, FU Qiang, et al. Multi-granularity text perception and hierarchical feature interaction method for visual grounding[J]. Journal of Electronics & Information Technology, 2025, 47(11): 4594–4605. doi: 10.11999/JEIT250387.
[17]	姜小波, 邓晗珂, 莫志杰, 等. 规则压缩模型和灵活架构的Transformer加速器设计[J]. 电子与信息学报, 2024, 46(3): 1079–1088. doi: 10.11999/JEIT230188. JIANG Xiaobo, DENG Hanke, MO Zhijie, et al. Design of transformer accelerator with regular compression model and flexible architecture[J]. Journal of Electronics & Information Technology, 2024, 46(3): 1079–1088. doi: 10.11999/JEIT230188.
[18]	YE Tianzhu, DONG Li, XIA Yuqing, et al. Differential transformer[J]. arXiv preprint arXiv: 2410.05258, 2024. doi: 10.48550/arXiv.2410.05258.
[19]	SU Jianlin, AHMED M, LU Yu, et al. RoFormer: Enhanced transformer with rotary position embedding[J]. Neurocomputing, 2024, 568: 127063. doi: 10.1016/j.neucom.2023.127063.
[20]	LIU Peipei, LI Hong, WANG Zuoguang, et al. Multi-features based semantic augmentation networks for named entity recognition in threat intelligence[C]. The 26th International Conference on Pattern Recognition (ICPR), Montreal, Canada, 2022: 1557–1563. doi: 10.1109/ICPR56361.2022.9956373.
[21]	WEI Tianwen, QI Jianwei, HE Shenghuan, et al. Masked conditional random fields for sequence labeling[C]. 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021: 2024–2035. doi: 10.18653/v1/2021.naacl-main.163.
[22]	WANG Xuren, LIU Xinpei, AO Shengqin, et al. DNRTI: A large-scale dataset for named entity recognition in threat intelligence[C].2020 IEEE 19th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), Guangzhou, China, 2020: 1842–1848. doi: 10.1109/TrustCom50675.2020.00252.
[23]	WANG Xuren, HE Songheng, XIONG Zihan, et al. APTNER: A specific dataset for NER missions in cyber threat intelligence field[C]. 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD), Hangzhou, China, 2022: 1233–1238. doi: 10.1109/CSCWD54268.2022.9776031.
[24]	CAI Yongxin, KANG Lei, LENG Tao, et al. BNER: A broad learning system-based named entity recognition method for cyber threat intelligence[C].2025 11th IEEE International Conference on Privacy Computing and Data Security (PCDS), Hakodate, Japan, 2025: 397–404. doi: 10.1109/PCDS65695.2025.00060.
[25]	DU Chao, LIU Xuhong, MIAO Lin, et al. Threat intelligence named entity recognition based on global gated feature fusion[C]. 2024 6th International Conference on Internet of Things, Automation and Artificial Intelligence (IoTAAI), Guangzhou, China, 2024: 618–622. doi: 10.1109/IoTAAI62601.2024.10692655.
[26]	WANG Peng and LIU Jingju. A cyber threat entity recognition method based on robust feature representation and adversarial training[C]. 2023 12th International Conference on Computing and Pattern Recognition, Qingdao, China, 2024: 255–259. doi: 10.1145/3633637.3633677.
[27]	CHANG Yu, WANG Gang, ZHU Peng, et al. Research on unified cyber threat intelligence entity recognition method based on multiple features[C].2023 4th International Conference on Computers and Artificial Intelligence Technology (CAIT), Macau, China, 2023: 233–240. doi: 10.1109/CAIT59945.2023.10469250.
[28]	孙语晨. 基于大语言模型的威胁情报信息抽取研究与实现[D]. [硕士论文], 北京邮电大学, 2025. doi: 10.26969/d.cnki.gbydu.2025.002794. SUN Yuchen. Research and implemention of threat intelligence information extraction based on large language model[D]. [Master dissertation], Beijing University of Posts and Telecommunications, 2025. doi: 10.26969/d.cnki.gbydu.2025.002794.
[29]	汪溢镭, 孙歆, 韩嘉佳, 等. 暗网高质量威胁情报获取技术与实现[J/OL]. https://doi.org/10.19678/j.issn.1000-3428.0068805, 2024. WANG Yilei, SUN Xin, HAN Jiajia, et al. Techniques and implementation of high-quality threat intelligence acquisition from the dark web[J/OL]. https://doi.org/10.19678/j.issn.1000-3428.0068805, 2024.