Multi-channel Enhanced Graph Convolutional Network for Aspect-based Sentiment Analysis
-
摘要: 传统的基于单通道的特征提取方式，仅使用单一的依赖关系捕获特征，忽略单词间的语义相似性与依赖关系类型信息。尽管基于图卷积网络进行方面情感分析的方法已经取得一定成效，但始终难以同时聚合节点的语义信息和句法结构特征，在整个迭代训练过程中最初的语义特征会逐渐遗失，影响句子最终的情感分类效果。由于缺乏先验知识会导致模型对相关情感词的误解，因此需要引入外部知识来丰富文本信息。目前，如何利用图神经网络(GNN)融合句法和语义特征的方式仍值得深入研究。针对上述问题，该文提出一种多通道增强图卷积网络模型。首先，通过对情感知识和依赖类型增强的句法图进行图卷积操作，得到基于语法的两种表示，与经过多头注意力和图卷积学习到的语义表示进行融合，使多通道的特征能够互补学习。实验结果表明，在5个公开数据集上，准确率和宏F1值优于基准模型。由此可见，依赖类型和情感知识均对增强句法图有重要影响，表明融合语义信息与句法结构的有效性。Abstract: In traditional single-channel-based feature extraction approaches, features are captured solely based on dependency, while semantic similarity and dependency types between words are ignored. Although some success has been achieved through the graph convolutional network-based approach for sentiment analysis, aggregating both semantic information and syntactic structure features remains challenging, and the gradual loss of semantic features throughout the training process affects the final sentiment classification effect. To prevent the model from misinterpreting relevant sentiment words due to the absence of prior knowledge, the inclusion of external knowledge is recommended to enrich the text. Presently, how to utilize Graph Neural Networks(GNN) to fuse syntactic and semantic features still deserves further research. A multi-channel enhanced graph convolutional network model is proposed in this paper to address the above issues. First, graph convolution operations on syntactic graphs enhanced with sentiment knowledge and dependency types are performed to obtain two syntax-based representations, which are fused with the semantic representations learned through multi-head attention and graph convolution, so that the multi-channel features can be learned complementarily. It is observed from the experimental results that both the accuracy and macro F1 of our model surpass those of the benchmark model on five publicly available datasets. These findings indicate the importance of dependency types and affective knowledge to enhance syntactic graphs and highlight the effectiveness of combining semantic information with syntactic structure.
-
表 1 SenticNet 6中情感词汇示例
词汇 极性-标签 SenticNet情感得分 语义相关词汇 Great 积极 0.999 'adore', 'glory', 'devote', 'admire', 'fairly' Excellent 积极 0.795 'well_done', 'great', 'excelent', 'good_quality', 'good' Hard 消极 –0.291 'hard_nail', 'disturb', 'upset', 'perturb', 'frustrate' Belittlement 消极 –0.693 'revile', 'rude', 'vituperation', 'scurrility', 'billingsgate' Unsalable 消极 –0.934 'sorry', 'trash', 'rubbish', 'superfluous', 'otiose' 
下载: 导出CSV
表 2 数据集统计
数据集 积极 中性 消极 训练集 测试集 训练集 测试集 训练集 测试集 Twitter 1561 173 3127 346 1560 173 Lap14 994 341 464 169 870 128 Rest14 2164 728 637 196 807 196 Rest15 912 326 36 34 256 182 Rest16 1240 469 69 30 439 117
下载: 导出CSV
表 3 模型的超参数设置
超参数 GloVe参数设置 BERT参数设置 批量训练样本数Batch size 16 16 训练迭代次数Epoch 100 100 优化器Optimizer Adam Adam 学习率Learning rate 10–3 2×10–5 丢失率Dropout rate 0.3 0.2 L2正则化系数 10–5 10–3
下载: 导出CSV
表 4 不同模型的结果对比(%)
类别 模型 Twitter Lap 14 Rest 14 Rest 15 Rest16 Acc 宏F1 Acc 宏F1 Acc 宏F1 Acc 宏F1 Acc 宏F1 基线 LSTM 69.56 67.70 69.28 63.09 78.13 67.47 77.37 55.17 86.80 63.88 ATAE-LSTM 69.65 67.40 69.14 63.18 77.32 66.57 75.43 56.34 83.25 63.85 IAN 72.50 70.81 72.05 67.38 79.26 70.09 78.54 52.65 84.74 55.21 AF-LSTM 69.21 68.24 69.97 63.49 77.46 65.18 76.12 56.29 85.61 66.15 GCN模型 TD-GAT 72.70 70.45 75.63 70.64 81.32 71.72 80.38 60.50 87.71 67.87 ASGCN 72.15 70.40 75.55 71.05 80.77 72.02 79.89 61.89 88.99 67.48 BiGCN 74.16 73.35 74.49 71.84 81.97 73.48 81.16 64.79 88.96 70.84 GL-GCN 73.26 71.26 76.91 72.76 82.11 73.46 80.81 64.99 88.47 69.64 MIGCN 73.31 72.12 76.59 72.44 82.32 74.31 80.81 64.21 89.50 71.97 DA-GCN 72.78 70.93 75.58 71.20 81.42 74.62 79.70 61.88 89.76 67.81 MSD-GCN 74.52 73.68 77.59 74.18 82.67 75.44 80.44 67.74 89.03 71.95 SGAN 74.95 73.37 76.75 72.90 81.80 74.28 81.28 65.31 88.49 72.83 KSD-GCN 75.00 73.38 78.37 75.18 84.11 76.78 82.47 64.20 90.42 73.81 BERT DA-GCN-BERT 75.43 73.67 78.82 75.28 83.43 74.35 82.97 64.56 89.69 71.86 SK-GCN-BERT 75.00 73.01 79.00 75.57 83.48 75.19 83.20 66.78 87.19 72.02 KSD-GCN-BERT 76.73 75.76 80.72 76.58 86.07 80.00 84.50 68.83 91.88 74.09
下载: 导出CSV
表 5 消融实验结果(%)
模型 Twitter Lap 14 Rest 14 Rest 15 Rest16 Acc 宏F1 Acc 宏F1 Acc 宏F1 Acc 宏F1 Acc 宏F1 本文 75.00 73.38 78.37 75.18 84.11 76.78 82.47 64.20 90.42 73.81 w/o sem 72.25 70.58 75.70 71.28 81.33 71.59 81.36 62.12 88.31 68.10 w/o dsyn 73.55 71.50 74.60 70.10 81.51 71.47 80.81 61.61 87.66 67.73 w/o ksyn 73.12 71.92 74.61 70.23 81.60 72.08 80.25 63.77 88.96 68.51 w/o sem-dsyn 72.54 70.78 75.39 70.73 81.16 71.12 80.99 62.44 88.31 70.11 w/o dsyn-ksyn 72.97 71.44 75.54 70.48 81.07 70.21 80.44 62.03 87.66 68.54 w/o sem-ksyn 72.54 70.79 74.60 71.22 80.98 71.19 79.70 62.01 88.14 69.46
下载: 导出CSV
-
[1] TANG Duyu, QIN Bing, FENG Xiaocheng, et al. Effective LSTMs for target-dependent sentiment classification[C]. The 26th International Conference on Computational Linguistics: Technical Papers, Osaka, Japan, 2016: 3298–3307. [2] WANG Yequan, HUANG Minlie, ZHU Xiaoyan, et al. Attention-based LSTM for aspect-level sentiment classification[C]. The 2016 Conference on Empirical Methods in Natural Language Processing, Austin, USA, 2016: 606–615. [3] MA Dehong, LI Sujian, ZHANG Xiaodong, et al. Interactive attention networks for aspect-level sentiment classification[C]. The Twenty-Sixth International Joint Conference on Artificial Intelligence, Melbourne, Australia, 2017: 4068–4074. [4] MIAO Yuqing, LUO Ronghai, ZHU Lin, et al. Contextual graph attention network for aspect-level sentiment classification[J]. Mathematics, 2022, 10(14): 2473. doi: 10.3390/math10142473. [5] ZHANG Chen, LI Qiuchi, and SONG Dawei. Aspect-based sentiment classification with aspect-specific graph convolutional networks[C]. The 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, 2019: 4568–4578. [6] ZHANG Mi and QIAN Tieyun. Convolution over hierarchical syntactic and lexical graphs for aspect level sentiment analysis[C]. The 2020 Conference on Empirical Methods in Natural Language Processing, 2020: 3540–3549. [7] YUAN Li, WANG Jin, YU L C, et al. Syntactic graph attention network for aspect-level sentiment analysis[J]. IEEE Transactions on Artificial Intelligence, 2022: 1–15. [8] ZHAO Zefang, LIU Yuyang, GAO Junruo, et al. Multi-grained syntactic dependency-aware graph convolution for aspect-based sentiment analysis[C]. The 2022 Conference on International Joint Conference on Neural Networks, Padua, Italy, 2022: 1–8. [9] DAI Anan, HU Xiaohui, NIE Jianyun, et al. Learning from word semantics to sentence syntax by graph convolutional networks for aspect-based sentiment analysis[J]. International Journal of Data Science and Analytics, 2022, 14(1): 17–26. doi: 10.1007/s41060-022-00315-2. [10] 王汝言, 陶中原, 赵容剑, 等. 多交互图卷积网络用于方面情感分析[J]. 电子与信息学报, 2022, 44(3): 1111–1118. doi: 10.11999/JEIT210459.WANG Ruyan, TAO Zhongyuan, ZHAO Rongjian, et al. Multi-interaction graph convolutional networks for aspect-level sentiment analysis[J]. Journal of Electronics &Information Technology, 2022, 44(3): 1111–1118. doi: 10.11999/JEIT210459. [11] WANG Kai, SHEN Weizhou, YANG Yunyi, et al. Relational graph attention network for aspect-based sentiment analysis[C]. The 58th Annual Meeting of the Association for Computational Linguistics, 2020: 3229–3238. [12] LIANG Bin, SU Hang, GUI Lin, et al. Aspect-based sentiment analysis via affective knowledge enhanced graph convolutional networks[J]. Knowledge-Based Systems, 2022, 235: 107643. doi: 10.1016/j.knosys.2021.107643. [13] ZHOU Jie, HUANG J X, HU Q V, et al. SK-GCN: Modeling syntax and knowledge via graph convolutional network for aspect-level sentiment classification[J]. Knowledge-Based Systems, 2020, 205: 106292. doi: 10.1016/j.knosys.2020.106292. [14] XU Junjie, YANG Shuwen, XIAO Luwei, et al. Graph convolution over the semantic-syntactic hybrid graph enhanced by affective knowledge for aspect-level sentiment classification[C]. International Joint Conference on Neural Networks, Padua, Italy, 2022: 1–8. [15] PENNINGTON J, SOCHER R, and MANNING C. GloVe: Global vectors for word representation[C]. The 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, 2014: 1532–1543. [16] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]. The 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Minnesota, 2019: 4171–4186. [17] CAMBRIA E, LI Yang, XING F Z, et al. SenticNet 6: Ensemble application of symbolic and subsymbolic AI for sentiment analysis[C]. The 29th ACM International Conference on Information and Knowledge Management, Ireland, 2020: 105–114. [18] HUANG Binxuan and CARLEY K M. Syntax-aware aspect level sentiment classification with graph attention networks[C]. The 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, 2019: 5469–5477. [19] LI Dong, WEI Furu, TAN Chuanqi, et al. Adaptive recursive neural network for target-dependent twitter sentiment classification[C]. The 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, USA, 2014: 49–54. [20] PONTIKI M, GALANIS D, PAVLOPOULOS J, et al. SemEval-2014 task 4: Aspect based sentiment analysis[C]. The 8th International Workshop on Semantic Evaluation, Dublin, Ireland, 2014: 27–35. [21] PONTIKI M, GALANIS D, PAPAGEORGIOU H, et al. SemEval-2015 task 12: Aspect based sentiment analysis[C]. The 9th International Workshop on Semantic Evaluation, Denver, USA, 2015: 486–495. [22] PONTIKI M, GALANIS D, PAPAGEORGIOU H, et al. SemEval-2016 task 5: Aspect based sentiment analysis[C]. The 10th International Workshop on Semantic Evaluation, San Diego, USA, 2016: 19–30. [23] WANG Xin, LIU Yuanchao, SUN Chengjie, et al. Predicting polarities of tweets by composing word embeddings with long short-term memory[C]. The 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China, 2015: 1343–1353. [24] TAY Y, TUAN L A, and HUI S C. Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis[C]. The Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, USA, 2018: 5956–5963. [25] ZHU Xiaofei, ZHU Ling, GUO Jiafeng, et al. GL-GCN: Global and local dependency guided graph convolutional networks for aspect-based sentiment classification[J]. Expert Systems with Applications, 2021, 186: 115712. doi: 10.1016/j.eswa.2021.115712. [26] WANG Xue, LIU Peiyu, ZHU Zhenfang, et al. Aspect-based sentiment analysis with graph convolutional networks over dependency awareness[C]. The 26th International Conference on Pattern Recognition, Montreal, Canada, 2022: 2238–2245. -
图(7) / 表(5)
计量
- 文章访问数: 547
- HTML全文浏览量: 253
- PDF下载量: 87
- 被引次数: 0
下载:
