A Method for Named Entity Recognition in Military Intelligence Domain Using Large Language Models

LI Yongbin; LIU Lian; ZHENG Jie

doi:10.11999/JEIT250764

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 >

LI Yongbin, LIU Lian, ZHENG Jie. A Method for Named Entity Recognition in Military Intelligence Domain Using Large Language Models[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250764

Citation:

LI Yongbin, LIU Lian, ZHENG Jie. A Method for Named Entity Recognition in Military Intelligence Domain Using Large Language Models[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250764

Citation:

PDF( 2593 KB)

A Method for Named Entity Recognition in Military Intelligence Domain Using Large Language Models

doi: 10.11999/JEIT250764 cstr: 32379.14.JEIT250764

LI Yongbin^{1, 2},
LIU Lian^{1, 2},
ZHENG Jie^{1, 2
,
,}

1.
Political and Legal Committee of CMC, Beijing 100010, China
2.
Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100080, China

Received Date: 2025-08-14
Rev Recd Date: 2025-10-21

Available Online: 2025-10-24

Abstract

Abstract

Objective Named Entity Recognition (NER) is a fundamental task in information extraction within specialized domains, particularly military intelligence. It plays a critical role in situation assessment, threat analysis, and decision support. However, conventional NER models face major challenges. First, the scarcity of high-quality annotated data in the military intelligence domain is a persistent limitation. Due to the sensitivity and confidentiality of military information, acquiring large-scale, accurately labeled datasets is extremely difficult, which severely restricts the training performance and generalization ability of supervised learning–based NER models. Second, military intelligence requires handling complex and diverse information extraction tasks. The entities to be recognized often possess domain-specific meanings, ambiguous boundaries, and complex relationships, making it difficult for traditional models with fixed architectures to adapt flexibly to such complexity or achieve accurate extraction. This study aims to address these limitations by developing a more effective NER method tailored to the military intelligence domain, leveraging Large Language Models (LLMs) to enhance recognition accuracy and efficiency in this specialized field. Methods To achieve the above objective, this study focuses on the military intelligence domain and proposes a NER method based on LLMs. The central concept is to harness the strong semantic reasoning capabilities of LLMs, which enable deep contextual understanding of military texts, accurate interpretation of complex domain-specific extraction requirements, and autonomous execution of extraction tasks without heavy reliance on large annotated datasets. To ensure that general-purpose LLMs can rapidly adapt to the specialized needs of military intelligence, two key strategies are employed. First, instruction fine-tuning is applied. Domain-specific instruction datasets are constructed to include diverse entity types, extraction rules, and representative examples relevant to military intelligence. Through fine-tuning with these datasets, the LLMs acquire a more precise understanding of the characteristics and requirements of NER in this field, thereby improving their ability to follow targeted extraction instructions. Second, Retrieval-Augmented Generation (RAG) is incorporated. A domain knowledge base is developed containing expert knowledge such as entity dictionaries, military terminology, and historical extraction cases. During the NER process, the LLM retrieves relevant knowledge from this base in real time to support entity recognition. This strategy compensates for the limited domain-specific knowledge of general LLMs and enhances recognition accuracy, particularly for rare or complex entities. Results and Discussions Experimental results indicate that the proposed LLM–based NER method, which integrates instruction fine-tuning and RAG, achieves strong performance in military intelligence NER tasks. Compared with conventional NER models, it demonstrates higher precision, recall, and F1-score, particularly in recognizing complex entities and managing scenarios with limited annotated data. The effectiveness of this method arises from several key factors. The powerful semantic reasoning capability of LLMs enables a deeper understanding of contextual nuances and ambiguous expressions in military texts, thereby reducing missed and false recognitions commonly caused by rigid pattern-matching approaches. Instruction fine-tuning allows the model to better align with domain-specific extraction requirements, ensuring that the recognition results correspond more closely to the practical needs of military intelligence analysis. Furthermore, the incorporation of RAG provides real-time access to domain expert knowledge, markedly enhancing the recognition of entities that are highly specialized or morphologically variable within military contexts. This integration effectively mitigates the limitations of traditional models that lack sufficient domain knowledge. Conclusions This study proposes a LLM–based NER method for the military intelligence domain, effectively addressing the challenges of limited annotated data and complex extraction requirements encountered by traditional models. By combining instruction fine-tuning and RAG, general-purpose LLMs can be rapidly adapted to the specialized demands of military intelligence, enabling the construction of an efficient domain-specific expert system at relatively low cost. The proposed method provides an effective and scalable solution for NER tasks in military intelligence scenarios, enhancing both the efficiency and accuracy of information extraction in this field. It offers not only practical value for military intelligence analysis and decision support but also methodological insight for NER research in other specialized domains facing similar data and complexity constraints, such as aerospace and national security. Future research will focus on optimizing instruction fine-tuning strategies, expanding the domain knowledge base, and reducing computational cost to further improve model performance and applicability.
- Large Language Models (LLMs),
- Named Entity Recognition (NER),
- Instruction fine-tuning,
- Retrieval-augmented generation,
- Knowledge base

FullText(HTML)

References(18)

References

[1]	LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural architectures for named entity recognition[C]. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, USA, 2016: 260–270. doi: 10.18653/v1/N16-1030.
[2]	DAI Zhenjin, WANG Xutao, NI Pin, et al. Named entity recognition using BERT BiLSTM CRF for Chinese electronic health records[C]. Proceedings of the 2019 12th International Congress on Image and Signal Processing, Biomedical Engineering and Informatics (CISP-BMEI), Suzhou, China, 2019: 1–5. doi: 10.1109/CISP-BMEI48845.2019.8965823.
[3]	WANG Chenguang, LIU Xiao, CHEN Zui, et al. Zero-shot information extraction as a unified text-to-triple translation[C]. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Punta Cana, Dominican Republic, 2021: 1225–1238. doi: 10.18653/v1/2021.emnlp-main.94.
[4]	YANG Qingping, HU Yingpeng, CAO Rongyu, et al. Zero-shot key information extraction from mixed-style tables: Pre-training on Wikipedia[C]. Proceedings of the 2021 IEEE International Conference on Data Mining (ICDM), Auckland, New Zealand, 2021: 1451–1456. doi: 10.1109/icdm51629.2021.00187.
[5]	LU Yaojie, LIU Qing, DAI Dai, et al. Unified structure generation for universal information extraction[C]. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Dublin, Ireland, 2022: 5755–5772. doi: 10.18653/v1/2022.acl-long.395.
[6]	GUO Daya, YANG Dejian, ZHANG Haowei, et al. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning[EB/OL]. https://arxiv.org/abs/2501.12948, 2025.
[7]	LIU Aixin, FENG Bei, XUE Bing, et al. Deepseek-v3 technical report[EB/OL]. https://arxiv.org/abs/2412.19437, 2024.
[8]	BI Xiao, CHEN Deli, CHEN Guanting, et al. DeepSeek LLM: scaling open-source language models with longtermism[EB/OL]. https://arxiv.org/abs/2401.02954, 2024.
[9]	YUAN Jingyang, GAO Huazuo, DAI Damai, et al. Native sparse attention: hardware-aligned and natively trainable sparse attention[C]. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, Vienna, Austria, 2025: 23078–23097. doi: 10.18653/v1/2025.acl-long.1126.
[10]	WANG Xiao, ZHOU Weikang, ZU Can, et al. InstructUIE: multi-task instruction tuning for unified information extraction[EB/OL]. https://arxiv.org/abs/2304.08085, 2023.
[11]	HU Danqing, LIU Bing, ZHU Xiaofeng, et al. Zero-shot information extraction from radiological reports using ChatGPT[J]. International Journal of Medical Informatics, 2024, 183: 105321. doi: 10.1016/j.ijmedinf.2023.105321.
[12]	KARTCHNER D, RAMALINGAM S, AL-HUSSAINI I, et al. Zero-shot information extraction for clinical meta-analysis using large language models[C]. Proceedings of the 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, Toronto, Canada, 2023: 396–405. doi: 10.18653/v1/2023.bionlp-1.37.
[13]	张国宾, 姬红兵, 王佳萌, 等. 基于通用信息抽取大模型的特定领域文本实体关系抽取研究[J]. 中国信息界, 2024(8): 159–161. ZHANG Guobin, JI Hongbing, WANG Jiameng, et al. Research on entity-relationship extraction from domain-specific texts leveraging generalized information extraction large models[J]. Information China, 2024(8): 159–161. (查阅网上资料, 未找到对应英文翻译信息, 请确认).
[14]	皮乾坤, 卢记仓, 祝涛杰, 等. 一种基于大语言模型增强的零样本知识抽取方法[J/OL]. 计算机科学. https://link.cnki.net/urlid/50.1075.TP.20250123.1638.012, 2025. PI Qiankun, LU Jicang, ZHU Taojie, et al. A zero-shot knowledge extraction method based on large language model enhanced[J/OL]. Computer Science. https://link.cnki.net/urlid/50.1075.TP.20250123.1638.012, 2025.
[15]	户才顺. 基于大语言模型的审计领域命名实体识别算法研究[J]. 计算机科学, 2025, 52(S1): 72–75. LU Caishun. Study on named entity recognition algorithms in audit domain based on large language models[J]. Computer Science, 2025, 52(S1): 72–75.
[16]	胡慧云, 葛杨, 崔凌潇, 等. 融合多模态信息与大语言模型的生成式命名实体识别方法[J/OL]. 计算机工程与应用. https://doi.org/10.3778/j.issn.1002-8331.2503-0243, 2025. HU Huiyun, GE Yang, CUI Lingxiao, et al. Generative named entity recognition method integrating multimodal information and large language models[J/OL]. Computer Engineering and Applications. https://doi.org/10.3778/j.issn.1002-8331.2503-0243, 2025.
[17]	HU E J, SHEN Y, WALLIS P, et al. LoRA: low-rank adaptation of large language models[EB/OL]. https://arxiv.org/abs/2106.09685, 2021.
[18]	LEWIS P, PEREZ E, PIKTUS A, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks[C]. Proceedings of the 34th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2020: 793. doi: 10.5555/3495724.3496517.