Cognitive Emotional Interaction Model of Robot Based on Reinforcement Learning

Hongcheng HUANG; Jing LI; Min HU; Yang TAO; Lan KOU

doi:10.11999/JEIT191035

Volume 43 Issue 6

Jun. 2021

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2021 > 43(6): 1781-1788

Hongcheng HUANG, Jing LI, Min HU, Yang TAO, Lan KOU. Cognitive Emotional Interaction Model of Robot Based on Reinforcement Learning[J]. Journal of Electronics & Information Technology, 2021, 43(6): 1781-1788. doi: 10.11999/JEIT191035

Citation:

Hongcheng HUANG, Jing LI, Min HU, Yang TAO, Lan KOU. Cognitive Emotional Interaction Model of Robot Based on Reinforcement Learning[J]. Journal of Electronics & Information Technology, 2021, 43(6): 1781-1788. doi: 10.11999/JEIT191035

Citation:

PDF( 1055 KB)

Cognitive Emotional Interaction Model of Robot Based on Reinforcement Learning

doi: 10.11999/JEIT191035

Hongcheng HUANG^{1, 2},
Jing LI¹,
Min HU¹,
Yang TAO^{1, 2
,
,},
Lan KOU¹

1.
School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
2.
Chongqing Engineering Research Center of Communication Software, Chongqing 400065, China

Funds: The National Key Research and Development Project (2019YFB2102001), The National Natural Science Foundation of China (61871062)

Received Date: 2019-12-24
Rev Recd Date: 2021-02-22

Available Online: 2021-03-17

Publish Date: 2021-06-18

Abstract

Abstract

In order to enhance the cognitive emotional computing ability of robot, a cognitive emotional interaction model of robot based on reinforcement learning is proposed, which combines immediate feedback and long-term trend according to PAD(Pleasure-Arousal-Dominance) emotional space. Firstly, according to the psychology theory of interpersonal communication, the human emotion generation process is simulated to generate human-like emotions, and the three influencing factors of similarity, positivity and empathy are extracted. Secondly, the relationship between the response emotion+ state and the contexted long-term emotion state is established by using the global co-ordination feature of reinforcement learning, so as to model the robot emotion generation process. Then, three factors are incorporated into the model reward mechanism for the evaluate of the interactive emotion state, to update the model and get the optimal emotional strategy. Finally, the optimal emotional state corresponding to the obtained optimal emotional strategy is used to update the robot's emotional state transition probability, and based on the sentiment values of the six basic emotional states in space, them are mapped to continuous emotional space to get the optimal response emotional value of the robot. Subjective and objective comparison experiments show that the model in this paper can effectively increase the delicateness, continuity, positivity and empathy of the robot's emotional expression, and can effectively reduce the robot's dependence on external emotional stimuli, further improving the harmonious and friendly human-computer interaction.
- Pleasure-Arousal-Dominance (PAD) emotion space,
- Reinforcement learning,
- Emotional state transfer,
- Cognitive emotion generation

FullText(HTML)

References(19)

References

[1]	ZUCCO C, CALABRESE B, and CANNATARO M. Sentiment analysis and affective computing for depression monitoring[C]. 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Kansas City, USA, 2017: 1988–1995. doi: 10.1109/BIBM.2017.8217966.
[2]	杨勇, 张志瑜. 基于PAD的个性化情感模型[J]. 重庆邮电大学学报: 自然科学版, 2012, 24(1): 96–103. doi: 10.3979/j.issn.1673-825X.2012.01.019 YANG Yong and ZHANG Zhiyu. Personalized affective model based on PAD[J]. Journal of Chongqing University of Posts and Telecommunications:Natural Science Edition, 2012, 24(1): 96–103. doi: 10.3979/j.issn.1673-825X.2012.01.019
[3]	WU Dapeng, HAN Xiaojuan, YANG Zhigang, et al. Exploiting transfer learning for emotion recognition under cloud-edge-client collaborations[J]. IEEE Journal on Selected Areas in Communications, 2021, 39(2): 479–490. doi: 10.1109/JSAC.2020.3020677
[4]	丁永刚, 李石君, 付星, 等. 面向时序感知的多类别商品方面情感分析推荐模型[J]. 电子与信息学报, 2018, 40(6): 1453–1460. doi: 10.11999/JEIT170938 DING Yonggang, LI Shijun, FU Xing, et al. Temporal-aware multi-category products recommendation model based on aspect-level sentiment analysis[J]. Journal of Electronics &Information Technology, 2018, 40(6): 1453–1460. doi: 10.11999/JEIT170938
[5]	LIU Xin, XIE Lun, and WANG Zhiliang. Empathizing with emotional robot based on cognition reappraisal[J]. China Communications, 2017, 14(9): 100–113. doi: 10.1109/CC.2017.8068769
[6]	ZHANG Rui, WANG Zhenyu, and MAI Dongcheng. Building emotional conversation systems using multi-task Seq2Seq learning[C]. The 6th CCF International Conference on Natural Language Processing and Chinese Computing, Dalian, China, 2018: 612–621. doi: 10.1007/978-3-319-73618-1_51.
[7]	杨杨, 邱雪松, 孟洛明, 等. 情感驱动的自私MANETs节点协商机制[J]. 电子与信息学报, 2011, 33(6): 1294–1300. doi: 10.3724/SP.J.1146.2010.01072 YANG Yang, QIU Xuesong, MENG Luoming, et al. An emotion-driven negotiation mechanism of selfish nodes in the MANETs[J]. Journal of Electronics &Information Technology, 2011, 33(6): 1294–1300. doi: 10.3724/SP.J.1146.2010.01072
[8]	ZHOU Hao, HUANG Minlie, ZHANG Tianyang, et al. Emotional chatting machine: Emotional conversation generation with internal and external memory[C]. The 32nd AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th Innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, USA, 2018: 730–738.
[9]	WANG Ke and WAN Xiaojun. SentiGAN: Generating sentimental texts via mixture adversarial networks[C]. The 27th International Joint Conference on Artificial Intelligence, Stockholm, Sverige, 2018: 4446–4452. doi: 10.24963/ijcai.2018/618.
[10]	LI Jingyuan and SUN Xiao. A syntactically constrained bidirectional-asynchronous approach for emotional conversation generation[C]. 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 2018: 678–683. doi: 10.18653/v1/D18-1071.
[11]	孙晓, 李佳, 卫星. 基于强化学习的情感编辑约束对话内容生成[J/OL]. 自动化学报. doi: 10.16383/j.aas.c190058. SUN Xiao, LI Jia, and WEI Xing. Emotional editing constraint conversation generation based on reinforcement learning[J/OL]. Acta Automatica Sinica. doi: 10.16383/j.aas.c190058.
[12]	RUSSELL J A and MEHRABIAN A. Evidence for a three-factor theory of emotions[J]. Journal of Research in Personality, 1977, 11(3): 273–294. doi: 10.1016/0092-6566(77)90037-X
[13]	PARK J W, KIM W H, LEE W H, et al. How to completely use the PAD space for socially interactive robots[C]. 2011 IEEE International Conference on Robotics and Biomimetics, Karon Beach, Thailand, 2011: 3005–3010. doi: 10.1109/ROBIO.2011.6181762.
[14]	邵俊健. 高维数据的聚类算法及其距离度量的研究[D]. [硕士论文], 江南大学, 2019. SHAO Junjian. Research on clustering algorithm of high dimensional data and its Disatance metric[D]. [Master dissertation], Jiangnan University, 2019.
[15]	孙佩宏, 陶霖密. PAD情感空间中情感距离度量方法[C]. 第四届和谐人机环境联合学术会议论文集, 武汉, 2008: 642–649. SUN Peihong and TAO Linmi. Emotional distance measurement method in PAD emotion space[C]. The 4th Harmonious Human-Computer Environment Joint Academic Conference, Wuhan, China, 2008: 642–649.
[16]	吴伟国, 李虹漫. PAD情感空间内人工情感建模及人机交互实验[J]. 哈尔滨工业大学学报, 2019, 51(1): 29–37. doi: 10.11918/j.issn.0367-6234.201807138 WU Weiguo and LI Hongman. Artificial emotion modeling in PAD emotional space and human-robot interactive experiment[J]. Journal of Harbin Institute of Technology, 2019, 51(1): 29–37. doi: 10.11918/j.issn.0367-6234.201807138
[17]	HUPONT I, BALDASSARRI S, and CEREZO E. Facial emotional classification: From a discrete perspective to a continuous emotional space[J]. Pattern Analysis and Applications, 2013, 16(1): 41–54. doi: 10.1007/s10044-012-0286-6
[18]	GUNTHER C. ChatterBot tutorial[EB/OL]. https://chatterbot.readthedocs.io/en/stable/tutorial.html, 2019.
[19]	WU Yu, WU Wei, XING Chen, et al. Sequential matching network: A new architecture for multi-turn response selection in retrieval-based chatbots[C]. The 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Canada, 2017: 496–505.