基于语义理解注意力神经网络的多元特征融合中文文本分类

谢金宝; 侯永进; 康守强; 李佰蔚; 张霄

doi:10.11999/JEIT170815

基于语义理解注意力神经网络的多元特征融合中文文本分类

doi: 10.11999/JEIT170815

基金项目:

黑龙江省海外学人基金(1253HQ019)

计量
- 文章访问数: 1810
- HTML全文浏览量: 323
- PDF下载量: 335
- 被引次数: 0
出版历程
- 收稿日期: 2017-08-17
- 修回日期: 2018-01-15
- 刊出日期: 2018-05-19

Multi-feature Fusion Based on Semantic Understanding Attention Neural Network for Chinese Text Categorization

Funds:

The Overseas Scholars Fund Project of Heilongjiang Province (1253HQ019)

摘要

摘要: 在中文文本分类任务中，针对重要特征在中文文本中位置分布分散、稀疏的问题，以及不同文本特征对文本类别识别贡献不同的问题，该文提出一种基于语义理解的注意力神经网络、长短期记忆网络(LSTM)与卷积神经网络(CNN)的多元特征融合中文文本分类模型(3CLA)。模型首先通过文本预处理将中文文本分词、向量化。然后，通过嵌入层分别经过CNN通路、LSTM通路和注意力算法模型通路以提取不同层次、具有不同特点的文本特征。最终，文本特征经融合层融合后，由softmax分类器进行分类。基于中文语料进行了文本分类实验。实验结果表明，相较于CNN结构模型与LSTM结构模型，提出的算法模型对中文文本类别的识别能力最多提升约8%。
- 中文文本分类 /
- 多元特征融合 /
- 注意力算法 /
- 长短期记忆网络 /
- 卷积神经网络
Abstract: In Chinese text categorization tasks, the locations of the important features in the Chinese texts are disperse and sparse, and the different characteristics of Chinese texts contributes differently for the recognition of their categories. In order to solve the above problems, this paper proposes a multi-feature fusion model Three Convolutional neural network paths and Long short term memory path fused with Attention neural network path (3CLA) for Chinese text categorization, which is based on Convolutional Neural Network (CNN), Long Short Term Memory (LSTM) and semantic understanding attention neural networks. The model first uses text preprocessing to finish the segmentation and vectorization of the Chinese text. Then, through the embedding layer, the input data are sent to the CNN path, the LSTM path and the attention path respectively to extract text features of different levels and different characteristics. Finally, the text features are fused by the fusion layer and classified by the classifier. Based on the Chinese corpus, the text classification experiment is carried out. The results of the experiments show that compared with the CNN structure model and the LSTM structure model, the proposed algorithm model improves the recognition ability of Chinese text categories by up to about 8%.
- Chinese text categorization /
- Multi-feature fusion /
- Attention algorithm /
- Long Short Term Memory (LSTM) network /
- Convolutional Neural Network (CNN)

HTML全文

参考文献(20)

孙晓, 彭晓琪, 胡敏, 等. 基于多维扩展特征与深度学习的微博短文本情感分析[J]. 电子与信息学报, 2017, 39(9): 2048-2055. doi: 10.11999/JEIT160975.

SUN Xiao, PENG Xiaoqi, HU Min, et al. Extended multi- modality features and deep learning based Microblog short text sentiment analysis[J]. Journal of Electronics Information Technology, 2017, 39(9): 2048-2055. doi: 10.11999/JEIT160975.

KIM Yoon. Convolutional neural networks for sentence classification[C]. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, 2014: 1746-1751.

PHAM Ngocquan, KRUSZEWSKI German, and BOLEDA Gemma. Convolutional neural network language models[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 1153-1162.

陈钊, 徐睿峰, 桂林, 等. 结合卷积神经网络和词语情感序列特征的中文情感分析[J]. 中文信息学报, 2015, 29(6): 172-178. doi: 10.3969/j.issn.1003-0077.2015.06.023.

CHEN Zhao, XU Ruifeng, GUI Lin, et al. Combining convolutional neural networks and word sentiment sequence features for Chinese text sentiment analysis[J]. Journal of Chinese Information Processing, 2015, 29(6): 172-178. doi: 10.3969/j.issn.1003-0077.2015.06.023.

刘龙飞, 杨亮, 张绍武, 等. 基于卷积神经网络的微博情感倾向性分析[J]. 中文信息学报, 2015, 29(6): 159-165. doi: 10.3969/j.issn.1003-0077.2015.06.021.

LIU Longfei, YANG Liang, ZHANG Shaowu, et al. Convolutional neural networks for Chinese micro-blog sentiment analysis[J]. Journal of Chinese Information Processing, 2015, 29(6): 159-165. doi: 10.3969/j.issn.1003- 0077.2015.06.021.

ZHANG Ye, MARSHALL Iain, and WALLACE B C. Rational-augmented convolutional neural networks for text classification[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 759-804.

MIYAMOTO Yasumasa and CHO Kyunghyun. Gated word-character recurrent language model[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 1992-1997.

TANG Duyu, QIN Bing, LIU Ting, et al. Document modeling with gated recurrent neural network for sentiment classification[C]. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2015: 1422-1432.

梁军, 柴玉梅, 原慧斌, 等. 基于极性转移和LSTM递归网络的情感分析[J]. 中文信息学报, 2015, 29(5): 160-167. doi: 10.3969/j.issn.1003-0077.2015.05.020.

LIANG Jun, CHAI Yumei, YUAN Huibin, et al. Polarity shifting and LSTM based recursive networks for sentiment analysis[J]. Journal of Chinese Information Processing, 2015, 29(5): 160-167. doi: 10.3969/j.issn.1003-0077.2015.05.020.

刘飞龙, 郝文宁, 陈刚, 等. 基于双线性函数注意力Bi-LSTM模型的机器阅读理解[J]. 计算机科学, 2017, 44(6A): 92-96. doi: 10.11896/j.issn.1002-137X.2017.6A.019.

LIU Feilong, HAO Wenning, CHEN Gang, et al. Attention of bilinear function based Bi-LSTM model for machine reading comprehension[J]. Computer Science, 2017, 44(6A): 92-96. doi: 10.11896/j.issn.1002-137X.2017.6A.019.

CHENG Jianpeng, DONG Li, and LAPATA Mirella. Long short-term memory-networks for machine reading[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 551-561.

WANG Yequan, HUANG Minlie, ZHAO Li, et al. Attention- based LSTM for aspect-level sentiment classification[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 606-615.

PARIKH Ankur P, TAKSTROM Oscar, DAS Dipanjan, et al. A decomposable attention model for natural language inference[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 2249-2255.

GOLUB David and HE Xiaodong. Character-level question answering with attention[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 1598-1607.

MI Haitao, WANG Zhiguo, and ITTYCHERIAH Abe. Supervised attentions for neural machine translation[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 2283-2288.

施引文献

资源附件(0)

访问统计