A Multi-domain Text Classification Method Based on Recurrent Convolution Multi-task Learning

Jinbao XIE; Jiahui LI; Shouqiang KANG; Qingyan WANG; Yujing WANG

doi:10.11999/JEIT200869

Volume 43 Issue 8

Aug. 2021

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2021 > 43(8): 2395-2403

Jinbao XIE, Jiahui LI, Shouqiang KANG, Qingyan WANG, Yujing WANG. A Multi-domain Text Classification Method Based on Recurrent Convolution Multi-task Learning[J]. Journal of Electronics & Information Technology, 2021, 43(8): 2395-2403. doi: 10.11999/JEIT200869

Citation:

Jinbao XIE, Jiahui LI, Shouqiang KANG, Qingyan WANG, Yujing WANG. A Multi-domain Text Classification Method Based on Recurrent Convolution Multi-task Learning[J]. Journal of Electronics & Information Technology, 2021, 43(8): 2395-2403. doi: 10.11999/JEIT200869

Citation:

PDF( 2423 KB)

A Multi-domain Text Classification Method Based on Recurrent Convolution Multi-task Learning

doi: 10.11999/JEIT200869

1.
School of Robotic, Guangdong Polytechnic of Science and Technology, Zhuhai 519090, China
2.
School of Electrical and Electronic Engineering, Harbin University of Science and Technology, Harbin 150000, China

Funds: The collaborative intelligent robot production and education integrates innovative application platform based on the industrial Internet (2020CJPT004), The Natural Science Foundation of Heilongjiang Province (LH2019E058), The open fund projects of Hubei Key Laboratory of Intelligent Robot (Wuhan Institute of Technology) (HBIR 202004), The Fundamental Research Fundation for Universities of Heilongjiang Province (LGYC2018JC027)

Received Date: 2020-10-09
Rev Recd Date: 2021-02-03

Available Online: 2021-03-01

Publish Date: 2021-08-10

Abstract

Abstract

In the text classification task, many texts in different domains are similarly expressed and have the characteristics of correlation, which can solve the problem of insufficient training data with labels. The text of different fields can be combined with the multi-task learning method, and the training accuracy and speed of the model can be improved. A Recurrent Convolution Multi-Task Learning (MTL-RC) model for text multi-classification is proposed, jointly modeling the text of multiple tasks, and taking advantage of multi-task learning, Recurrent Neural Network(RNN) and Convolutional Neural Network(CNN) models to obtain the correlation between multi-domain texts, long-term dependence of text. Local features of text are extracted. Rich experiments are carried out based on multi-domain text classification datasets, the Recurrent Convolution Multi-Task Learning(MTL-LC) proposed in this paper has an average accuracy of 90.1% for text classification in different fields, which is 6.5% higher than the single-task learning model STL-LC. Compared with mainstream multi-tasking learning models Full Shared Multi-Task Learning(FS-MTL), Adversarial Multi-Task Learninng(ASP-MTL), and Indirect Communciation for Multi-Task Learning(IC-MTL) have increased by 5.4%, 4%, and 2.8%, respectively.
- Multi-domain text classification,
- Multi-task learning,
- Recurrent Neural Netword(RNN),
- Convolutional Neural Network(CNN)

FullText(HTML)

References(18)

References

[1]	谢金宝, 侯永进, 康守强, 等. 基于语义理解注意力神经网络的多元特征融合中文文本分类[J]. 电子与信息学报, 2018, 40(5): 1258–1265. doi: 10.11999/JEIT170815 XIE Jinbao, HOU Yongjin, KANG Shouqiang, et al. Multi-feature fusion based on semantic understanding attention neural network for Chinese text categorization[J]. Journal of Electronics &Information Technology, 2018, 40(5): 1258–1265. doi: 10.11999/JEIT170815
[2]	KIM Y. Convolutional neural networks for sentence classification[C]. 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, 2014: 1746–1751. doi: 10.3115/v1/D14-1181.
[3]	BLITZER J, DREDZE M, and PEREIRA F. Biographies, Bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification[C]. The 45th Annual Meeting of the Association of Computational Linguistics (ACL), Prague, Czech Republic, 2007: 440–447.
[4]	CARUANA R. Multitask learning[J]. Machine Learning, 1997, 28(1): 41–75. doi: 10.1023/A:1007379606734
[5]	LIU Xiaodong, GAO Jianfeng, HE Xiaodong, et al. Representation learning using multi-task deep neural networks for semantic classification and information retrieval[C]. The 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), Denver, USA, 2015: 912–921.
[6]	LIU Pengfei, QIU Xipeng, and HUANG Xuanjing. Recurrent neural network for text classification with multi-task learning[C]. The Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI), New York, USA, 2016: 2873–2879.
[7]	LIU Pengfei, QIU Xipeng, and HUANG Xuanjing. Adversarial multi-task learning for text classification[C]. The 55th Annual Meeting of the Association for Computational Linguistics (ACL), Vancouver, Canada, 2017: 1–10.
[8]	HOCHREITER S and SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735–1780. doi: 10.1162/neco.1997.9.8.1735
[9]	王鑫, 李可, 宁晨, 等. 基于深度卷积神经网络和多核学习的遥感图像分类方法[J]. 电子与信息学报, 2019, 41(5): 1098–1105. doi: 10.11999/JEIT180628 WANG Xin, LI Ke, NING Chen, et al. Remote sensing image classification method based on deep convolution neural network and multi-kernel learning[J]. Journal of Electronics &Information Technology, 2019, 41(5): 1098–1105. doi: 10.11999/JEIT180628
[10]	KALCHBRENNER N, GREFENSTETTE E, and BLUNSOM P. A convolutional neural network for modelling sentences[C]. The 52nd Annual Meeting of the Association for Computational Linguistics (ACL), Baltimore, USA, 2014: 655–665.
[11]	刘宗林, 张梅山, 甄冉冉, 等. 融入罪名关键词的法律判决预测多任务学习模型[J]. 清华大学学报: 自然科学版, 2019, 59(7): 497–504. doi: 10.16511/j.cnki.qhdxxb.2019.21.020 LIU Zonglin, ZHANG Meishan, ZHEN Ranran, et al. Multi-task learning model for legal judgment predictions with charge keywords[J]. Journal of Tsinghua University:Science and Technology, 2019, 59(7): 497–504. doi: 10.16511/j.cnki.qhdxxb.2019.21.020
[12]	COLLOBERT R and WESTON J. A unified architecture for natural language processing: Deep neural networks with multitask learning[C]. The 25th International Conference on Machine Learning (ICML), Helsinki, Finland, 2008: 160–167.
[13]	LI Shoushan, HUANG Churen, and ZONG Chengqing. Multi-domain sentiment classification with classifier combination[J]. Journal of Computer Science and Technology, 2011, 26(1): 25–33. doi: 10.1007/s11390-011-9412-y
[14]	LIU Pengfei, QIU Xipeng, and HUANG Xuanjing. Deep multi-task learning with shared memory for text classification[C]. 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), Austin, USA, 2016: 118–127.
[15]	LIU Pengfei, FU Jie, DONG Yue, et al. Learning multi-task communication with message passing for sequence learning[C]. AAAI Conference on Artificial Intelligence (AAAI), Palo Alto, USA, 2019: 4360–4367.
[16]	YUAN Zhigang, WU Sixing, WU Fangzhao, et al. Domain attention model for multi-domain sentiment classification[J]. Knowledge-Based Systems, 2018, 155: 1–10. doi: 10.1016/j.knosys.2018.05.004
[17]	MAAS A L, DALY R E, PHAM P T, et al. Learning word vectors for sentiment analysis[C]. The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL), Portland, USA, 2011: 142–150.
[18]	PANG Bo and LEE L. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales[C]. The 43rd Annual Meeting on Association for Computational Linguistics (ACL), Ann Arbor, USA, 2005: 115–124.