Self-attention Capsule Network Rate Prediction with Review Quality

Shunpan LIANG; Wei LIU; Dianlong YOU; Zeqian LIU; Fuzhi ZHANG

doi:10.11999/JEIT200932

Volume 43 Issue 12

Dec. 2021

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2021 > 43(12): 3451-3458

Shunpan LIANG, Wei LIU, Dianlong YOU, Zeqian LIU, Fuzhi ZHANG. Self-attention Capsule Network Rate Prediction with Review Quality[J]. Journal of Electronics & Information Technology, 2021, 43(12): 3451-3458. doi: 10.11999/JEIT200932

Citation:

Shunpan LIANG, Wei LIU, Dianlong YOU, Zeqian LIU, Fuzhi ZHANG. Self-attention Capsule Network Rate Prediction with Review Quality[J]. Journal of Electronics & Information Technology, 2021, 43(12): 3451-3458. doi: 10.11999/JEIT200932

Citation:

PDF( 1322 KB)

Self-attention Capsule Network Rate Prediction with Review Quality

doi: 10.11999/JEIT200932

Shunpan LIANG^{1, 2
,
,},
Wei LIU¹,
Dianlong YOU^{1, 2},
Zeqian LIU¹,
Fuzhi ZHANG^{1, 2}

1.
College of Information Science and Engineering, Yanshan University, Qinhuangdao 066004, China
2.
Key Laboratory for Software Engineering of Hebei Province, Yanshan University, Qinhuangdao 066004, China

Funds: The National Natural Science Foundation of China (62072393), The Natural Science Foundation of Hebei Province (G2021203010, F2021203038)

Received Date: 2020-11-02
Rev Recd Date: 2021-10-25

Available Online: 2021-11-10

Publish Date: 2021-12-21

Abstract

Abstract

Recommendation systems based on reviews generally use convolutional neural networks to identify the semantics. However, due to the “invariance” of convolutional neural networks, that is, they only pay attention to the existence of features and ignore the details of features. The pooling operation will also lose some important information; In addition, using all the reviews as auxiliary information will not only not improve the quality of semantics, but will be affected by the low-quality reviews, this will lead to inaccurate recommendations. In order to solve the two problems mentioned above, this paper proposes a SACR (Self-Attention Capsule network Rate prediction) model. SACR uses a self-attention capsule network that can retain feature details to mine reviews, uses user and item ID to mark low-quality reviews, and merge the two representations to predict the rate. This paper also improves the squeeze function of the capsule, which can obtain more accurate high-level capsules. The experiments show that SACR has a significant improvement in prediction accuracy compared to some classic models and the latest models.
- Recommendation system,
- Capsule network,
- Attention,
- Review quality,
- Rate prediction

FullText(HTML)

References(20)

References

[1]	TAN Yunzhi, ZHANG Min, LIU Yiqun, et al. Rating-boosted latent topics: Understanding users and items with ratings and reviews[C]. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16), New York, USA, 2016: 2640–2646.
[2]	ZHANG Wei, YUAN Quan, HAN Jiawei, et al. Collaborative multi-level embedding learning from reviews for rating prediction[C]. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16), New York, USA, 2016: 2986–2992.
[3]	BLEI D, CARIN L, and DUNSON D. Probabilistic topic models[J]. IEEE Signal Processing Magazine, 2010, 27(6): 55–65. doi: 10.1109/MSP.2010.938079
[4]	KIM D, PARK C, OH J, et al. Convolutional matrix factorization for document context-aware recommendation[C]. Proceedings of the 10th ACM Conference on Recommender Systems, Boston, USA, 2016: 233–240. doi: 10.1145/2959100.2959165.
[5]	ZHENG Lei, NOROOZI V, and YU P S. Joint deep modeling of users and items using reviews for recommendation[C]. Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, Cambridge, 2017: 425–434. doi: 10.1145/3018661.3018665.
[6]	CHEN Chong, ZHANG Min, LIU Yiqun, et al. Neural attentional rating regression with review-level explanations[C]. Proceedings of the 2018 World Wide Web Conference, Lyon, Italy, 2018: 1583–1592. doi: 10.1145/3178876.3186070.
[7]	LU Yichao, DONG Ruihai, and SMYTH B. Coevolutionary recommendation model: Mutual learning between ratings and reviews[C]. Proceedings of the 2018 World Wide Web Conference, Lyon, Italy, 2018: 773–782. doi: 10.1145/3178876.3186158.
[8]	SABOUR S, FROSST N, and HINTON G. Dynamic routing between capsules[C]. NIPS, Los Angeles, USA, 2017: 3859–3869.
[9]	ZHANG Ningyu, DENG Shumin, SUN Zhanling, et al. Attention-based capsule networks with dynamic routing for relation extraction[C]. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 2018: 986–992. doi: 10.18653/v1/d18-1120.
[10]	ZHANG Min and GENG Guohua. Capsule networks with word-attention dynamic routing for cultural relics relation extraction[J]. IEEE Access, 2020, 8: 94236–94244. doi: 10.1109/ACCESS.2020.2995447
[11]	ZHAO Wei, YE Jianbo, YANG Min, et al. Investigating capsule networks with dynamic routing for text classification[C]. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 2018: 3110–3119. doi: 10.18653/v1/D18-1350.
[12]	KIM J, JANG S, PARK E, et al. Text classification using capsules[J]. Neurocomputing, 2020, 376: 214–221. doi: 10.1016/j.neucom.2019.10.033
[13]	LI Chenliang, QUAN Cong, PENG Li, et al. A capsule network for recommendation and explaining what you like and dislike[C]. Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France, 2019: 275–284. doi: 10.1145/3331184.3331216.
[14]	LI Chao, LIU Zhiyuan, WU Mengmeng, et al. Multi-interest network with dynamic routing for recommendation at tmall[C]. The 28th ACM International Conference on Information and Knowledge Management, Beijing, China, 2019: 2615–2623. doi: 10.1145/3357384.3357814.
[15]	OLATUNJI I E, LI Xin, and LAM W. Context-aware helpfulness prediction for online product reviews[C]. 15th Asia Information Retrieval Societies Conference on Information Retrieval Technology, Hong Kong, China, 2019: 56–65. doi: 10.1007/978-3-030-42835-8_6.
[16]	丁永刚, 李石君, 付星, 等. 面向时序感知的多类别商品方面情感分析推荐模型[J]. 电子与信息学报, 2018, 40(6): 1453–1460. doi: 10.11999/JEIT170938 DING Yonggang, LI Shijun, FU Xing, et al. Temporal-aware multi-category products recommendation model based on aspect-level sentiment analysis[J]. Journal of Electronics &Information Technology, 2018, 40(6): 1453–1460. doi: 10.11999/JEIT170938
[17]	KINGMA D P and BA L J. ADAM: A method for stochastic optimization[C]. International Conference on Learning Representations (ICLR), San Diego, USA, 2015: 1–15.
[18]	SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: A simple way to prevent neural networks from overfitting[J]. The Journal of Machine Learning Research, 2014, 15(1): 1929–1958. doi: 10.5555/2627435.2670313
[19]	MCAULEY J, TARGETT C, SHI Qinfeng, et al. Image-based recommendations on styles and substitutes[C]. The 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, Santiago, USA, 2015: 43–52. doi: 10.1145/2766462.2767755.
[20]	SALAKHUTDINOV R and MNIH A. Probabilistic matrix factorization[C]. Proceedings of the 20th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2007: 1257–1264. doi: 10.5555/2981562.2981720.