Advances in Application of Game Brain-Computer Interface Based on ElectroEncephaloGram

HE Feng; DONG Bowen; HAN Jin; LI Yulong; XU Minpeng; MING Dong

doi:10.11999/JEIT211337

Volume 44 Issue 2

Feb. 2022

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2022 > 44(2): 415-423

HE Feng, DONG Bowen, HAN Jin, LI Yulong, XU Minpeng, MING Dong. Advances in Application of Game Brain-Computer Interface Based on ElectroEncephaloGram[J]. Journal of Electronics & Information Technology, 2022, 44(2): 415-423. doi: 10.11999/JEIT211337

Citation:

HE Feng, DONG Bowen, HAN Jin, LI Yulong, XU Minpeng, MING Dong. Advances in Application of Game Brain-Computer Interface Based on ElectroEncephaloGram[J]. Journal of Electronics & Information Technology, 2022, 44(2): 415-423. doi: 10.11999/JEIT211337

Citation:

PDF( 1840 KB)

Advances in Application of Game Brain-Computer Interface Based on ElectroEncephaloGram

doi: 10.11999/JEIT211337

HE Feng^{1, 2
,
,},
DONG Bowen¹,
HAN Jin²,
LI Yulong¹,
XU Minpeng^{1, 2},
MING Dong^{1, 2}

1.
Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, China
2.
School of Precision Instrument and Opto-electronics Engineering, Tianjin University, Tianjin 300072, China

Funds: The National Science Foundation for Distinguished Young Scholars (81925020), The National Science Foundation for Excellent Young Scientists (62122059), The National Natural Science Foundation of China (81630051, 61976152), Young Elite Scientist Sponsorship Program by CAST (2018QNRC001)

Received Date: 2021-11-25
Accepted Date: 2022-01-24
Rev Recd Date: 2022-01-16

Available Online: 2022-01-24

Publish Date: 2022-02-25

Abstract

Abstract

The game Brain-Computer Interface (game-BCI) is a novel interactive mode that the user can control and interact with the game directly by recognizing brain signals. It not only provides a new way of game interaction for healthy people, but also provides a new way of rehabilitation for the disabled. Compared with the other invasive BCIs, BCI based on scalp ElectroEncephaloGram (EEG) has a wider application prospects because there are the advantages of non-invasiveness, high time resolution, low cost, and good portability. The game-BCI technology based on scalp EEG is summarized and is divided into active, reactive, passive, and hybrid paradigms according to control signal types. Then the control strategies and application scenarios of different types of game BCI are introduced. The classification algorithms of EEG signals commonly used in game BCI are compared and analyzed. Finally, the current problems and future development directions in this filed are discussed.
- Brain-Computer Interface (BCI),
- ElectroEncephaloGram (EEG),
- Activate BCI,
- Reactive BCI,
- Passive BCI

FullText(HTML)

References(54)

References

[1]	WOLPAW J R, DEL R, MILLÁN J, and RAMSEY N F. Brain-computer interfaces: Definitions and principles[J]. Handbook of Clinical Neurology, 2020, 168: 15–23. doi: 10.1016/B978-0-444-63934-9.00002-0
[2]	WOLPAW J R, BIRBAUMER N, HEETDERKS W J, et al. Brain-computer interface technology: A review of the first international meeting[J]. IEEE Transactions on Rehabilitation Engineering, 2000, 8(2): 164–173. doi: 10.1109/tre.2000.847807
[3]	CATTAN G. The use of brain-computer interfaces in games is not ready for the general public[J]. Frontiers in Computer Science, 2021, 3: 628773. doi: 10.3389/fcomp.2021.628773
[4]	CATTAN G, ANDREEV A, and VISINONI E. Recommendations for integrating a P300-based brain–computer interface in virtual reality environments for gaming: An update[J]. Computers, 2020, 9(4): 92. doi: 10.3390/computers9040092
[5]	MERCADO J, ESCOBEDO L, and TENTORI M. A BCI video game using neurofeedback improves the attention of children with autism[J]. Journal on Multimodal User Interfaces, 2021, 15(3): 273–281. doi: 10.1007/s12193-020-00339-7
[6]	PARK K, KIHL T, PARK S, et al. Fairy tale directed game-based training system for children with ADHD using BCI and motion sensing technologies[J]. Behaviour & Information Technology, 2019, 38(6): 564–577. doi: 10.1080/0144929X.2018.1544276
[7]	SEKHAVAT Y A. Battle of minds: A new interaction approach in BCI games through competitive reinforcement[J]. Multimedia Tools and Applications, 2020, 79(5): 3449–3464. doi: 10.1007/s11042-019-07963-w
[8]	CHEN Yuliang. Design of a game control system based on Brain-computer interface: Link to a game[C]. Proceedings of the 2021 2nd International Conference on Computing and Data Science, Stanford, USA, 2021: 255–258.
[9]	TORRES E P, TORRES E A, HERNÁNDEZ-ÁLVAREZ M, et al. EEG-based BCI emotion recognition: A survey[J]. Sensors, 2020, 20(18): 5083. doi: 10.3390/s20185083
[10]	DE CASTRO-CROS M, SEBASTIAN-ROMAGOSA M, RODRÍGUEZ-SERRANO J, et al. Effects of gamification in BCI functional rehabilitation[J]. Frontiers in Neuroscience, 2020, 14: 882. doi: 10.3389/fnins.2020.00882
[11]	SINGH A K, WANG Yukai, KING J T, et al. Extended interaction with a BCI video game changes resting-state brain activity[J]. IEEE Transactions on Cognitive and Developmental Systems, 2020, 12(4): 809–823. doi: 10.1109/TCDS.2020.2985102
[12]	ARPAIA P, ESPOSITO A, MANCINO F, et al. Active and passive brain-computer interfaces integrated with extended reality for applications in health 4.0[C]. Proceedings of the 8th International Conference on Augmented Reality, Virtual Reality and Computer Graphics, Virtual Event, 2021: 392–405.
[13]	CHMURA J, ROSING J, COLLAZOS S, et al. Classification of movement and inhibition using a hybrid BCI[J]. Frontiers in Neurorobotics, 2017, 11: 38. doi: 10.3389/fnbot.2017.00038
[14]	WEN Dong, LIANG Bingbing, ZHOU Yanhong, et al. The current research of combining multi-modal brain-computer interfaces with virtual reality[J]. IEEE Journal of Biomedical and Health Informatics, 2020, 25(9): 3278–3287. doi: 10.1109/JBHI.2020.3047836
[15]	NAYAK T, KO L W, JUNG T P, et al. Target classification in a novel SSVEP-RSVP based BCI gaming system[C]. Proceedings of 2019 IEEE International Conference on Systems, Man and Cybernetics, Bari, Italy, 2019: 4194–4198.
[16]	MENG Jiayuan, XU Minpeng, WANG Kun, et al. Separable EEG features induced by timing prediction for active brain-computer interfaces[J]. Sensors, 2020, 20(12): 3588. doi: 10.3390/s20123588
[17]	TEZZA D, CAPRIO D, GARCIA S, et al. Brain-controlled drone racing game: A qualitative analysis[C]. Proceedings of the 2nd International Conference on Human-Computer Interaction, Copenhagen, Denmark, 2020: 350–360.
[18]	COYLE D, GARCIA J, SATTI A R, et al. EEG-based continuous control of a game using a 3 channel motor imagery BCI: BCI game[C]. Proceedings of 2011 IEEE Symposium on Computational Intelligence, Cognitive Algorithms, Mind, and Brain, Paris, France, 2011: 1–7.
[19]	KARÁCSONY T, HANSEN J P, IVERSEN H K, et al. Brain computer interface for neuro-rehabilitation with deep learning classification and virtual reality feedback[C]. Proceedings of the 10th Augmented Human International Conference 2019, Reims, France, 2019: 22.
[20]	VOURVOPOULOS A, FERREIRA A, and BADIA S B I. NeuRow: An immersive VR environment for motor-imagery training with the use of brain-computer interfaces and vibrotactile feedback[C]. Proceedings of the 3rd International Conference on Physiological Computing Systems, Lisbon, Portugal, 2016: 43–53.
[21]	PUTRI F, DING Hao, GARCIA A, et al. Towards successful multi-user Brain-Computer Interface (BCI) gaming: Analysis of the EEG signatures and connectivity[C]. Proceedings of the 3rd International Conference on Computer-Human Interaction Research and Applications, Vienna, Austria, 2019: 59–65.
[22]	LI Junhua, LIU Ye, LU Zhen, et al. A competitive brain computer interface: Multi-person car racing system[C]. Proceedings of the 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Osaka, Japan, 2013: 2200–2203.
[23]	KUMAR M K, PARAMESHACHARI B D, PRABU S, et al. Comparative analysis to identify efficient technique for interfacing BCI system[J]. IOP Conference Series:Materials Science and Engineering, 2020, 925: 012062. doi: 10.1088/1757-899X/925/1/012062
[24]	FILIZ E and ARSLAN R B. Design and implementation of steady state visual evoked potential based brain computer interface video game[C]. Proceedings of the 2020 IEEE 20th Mediterranean Electrotechnical Conference, Palermo, Italy, 2020: 335–338.
[25]	VIDAL J J. Real-time detection of brain events in EEG[J]. Proceedings of the IEEE, 1977, 65(5): 633–641. doi: 10.1109/PROC.1977.10542
[26]	PARAFITA R, PIRES G, NUNES U, et al. A spacecraft game controlled with a brain-computer interface using SSVEP with phase tagging[C]. Proceedings of the 2013 IEEE 2nd International Conference on Serious Games and Applications for Health, Vilamoura, Portugal, 2013: 1–6.
[27]	PEREZ-VALERO E, LOPEZ-GORDO M A, and VAQUERO-BLASCO M A. An attention-driven videogame based on steady-state motion visual evoked potentials[J]. Expert Systems, 2021, 38(4): e12682. doi: 10.1111/exsy.12682
[28]	CRUZ I, MOREIRA C, POEL M, et al. Kessel run-a cooperative multiplayer SSVEP BCI game[C]. Proceedings of the 9th International Conference on Intelligent Technologies for Interactive Entertainment, Funchal, Portugal, 2017: 77–95.
[29]	FARWELL L A and DONCHIN E. Talking off the top of your head: Toward a mental prosthesis utilizing event-related brain potentials[J]. Electroencephalography and Clinical Neurophysiology, 1988, 70(6): 510–523. doi: 10.1016/0013-4694(88)90149-6
[30]	FINKE A, LENHARDT A, and RITTER H. The MindGame: A P300-based brain–computer interface game[J]. Neural Networks, 2009, 22(9): 1329–1333. doi: 10.1016/j.neunet.2009.07.003
[31]	LI Man, LI Feng, PAN Jiahui, et al. The MindGomoku: An online P300 BCI game based on bayesian deep learning[J]. Sensors, 2021, 21(5): 1613. doi: 10.3390/s21051613
[32]	BAYLISS J D and BALLARD D H. Single trial P3 epoch recognition in a virtual environment[J]. Neurocomputing, 2000, 32/33: 637–642. doi: 10.1016/S0925-2312(00)00226-5
[33]	HUANG Zhihua, ZHENG Wenming, WU Yingjie, et al. Ensemble or pool: A comprehensive study on transfer learning for c-VEP BCI during interpersonal interaction[J]. Journal of Neuroscience Methods, 2020, 343: 108855. doi: 10.1016/j.jneumeth.2020.108855
[34]	PENG Faqiang and HUANG Zhihua. A c-VEP BCI system for psychological experiments[C]. Proceedings of the 2019 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics, Suzhou, China, 2019: 1–5.
[35]	BEVERIDGE R, WILSON S, CALLAGHAN M, et al. Neurogaming with motion-onset visual evoked potentials (mVEPs): Adults versus teenagers[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27(4): 572–581. doi: 10.1109/TNSRE.2019.2904260
[36]	RIECHMANN H, FINKE A, and RITTER H. Using a cVEP-based brain-computer interface to control a virtual agent[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2016, 24(6): 692–699. doi: 10.1109/TNSRE.2015.2490621
[37]	FRIEDRICH E V C, SUTTIE N, SIVANATHAN A, et al. Brain–computer interface game applications for combined neurofeedback and biofeedback treatment for children on the autism spectrum[J]. Frontiers in Neuroengineering, 2014, 7: 21. doi: 10.3389/fneng.2014.00021
[38]	MUÑOZ J E, LOPEZ D S, LOPEZ J F, et al. Design and creation of a BCI videogame to train sustained attention in children with ADHD[C]. Proceedings of the 2015 10th Computing Colombian Conference, Bogota, Colombia, 2015: 194–199.
[39]	HRAMOV A E, MAKSIMENKO V A, and PISARCHIK A N. Physical principles of brain-computer interfaces and their applications for rehabilitation, robotics and control of human brain states[J]. Physics Reports, 2021, 918: 1–133. doi: 10.1016/j.physrep.2021.03.002
[40]	LI Zina, ZHANG Shuqing, and PAN Jiahui. Advances in hybrid brain-computer interfaces: Principles, design, and applications[J]. Computational Intelligence and Neuroscience, 2019, 2019: 3807670. doi: 10.1155/2019/3807670
[41]	KONEČNÝ R and LIAROKAPIS F. Foresthlon: Investigating gender experience through a hybrid BCI game[C]. Proceedings of the 3rd International Conference on Human-Computer Interaction, Virtual Event, 2021: 57–74.
[42]	XU Minpeng, HAN Jin, WANG Yijun, et al. Implementing over 100 command codes for a high-speed hybrid brain-computer interface using concurrent P300 and SSVEP features[J]. IEEE Transactions on Biomedical Engineering, 2020, 67(11): 3073–3082. doi: 10.1109/TBME.2020.2975614
[43]	WANG Zhihua, YU Yang, XU Ming, et al. Towards a hybrid BCI gaming paradigm based on motor imagery and SSVEP[J]. International Journal of Human–Computer Interaction, 2019, 35(3): 197–205. doi: 10.1080/10447318.2018.1445068
[44]	MÜHL C, GÜRKÖK H, BOS D P O, et al. Bacteria hunt: Evaluating multi-paradigm BCI interaction[J]. Journal on Multimodal User Interfaces, 2010, 4(1): 11–25. doi: 10.1007/s12193-010-0046-0
[45]	KHONG A, LIN Jiangnan, THOMAS K P, et al. BCI based multi-player 3-D game control using EEG for enhancing attention and memory[C]. Proceedings of 2014 IEEE International Conference on Systems, Man, and Cybernetics, San Diego, USA, 2014: 1847–1852.
[46]	KINNEY-LANG E, MURJI S, KELLY D, et al. Designing a flexible tool for rapid implementation of brain-computer interfaces (BCI) in game development[C]. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society, Montreal, Canada, 2020: 6078–6081.
[47]	SERRANO-BARROSO A, SIUGZDAITE R, GUERRERO-CUBERO J, et al. Detecting attention levels in ADHD children with a video game and the measurement of brain activity with a single-channel BCI headset[J]. Sensors, 2021, 21(9): 3221. doi: 10.3390/s21093221
[48]	ARRAMBIDE K, FREIMAN CORMIER L, WEHBE R R, et al. The development of "orbit": The collaborative BCI game for children with AD(H)D[C]. Proceedings of Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts, Barcelona, Spain, 2019: 341–348.
[49]	MRIDHA M F, DAS S C, KABIR M M, et al. Brain-computer interface: Advancement and challenges[J]. Sensors, 2021, 21(17): 5746. doi: 10.3390/s21175746
[50]	SUBRAMANIAN R R, VARMA K Y, BALAJI K, et al. Multiplayer online car racing with BCI In VR[C]. Proceedings of the 2021 5th International Conference on Intelligent Computing and Control Systems, Madurai, India, 2021: 1835–1839.
[51]	SCOTT J A and SIMS M. Acceleration of therapeutic use of brain computer interfaces by development for gaming[C]. Proceedings of the 12th International Conference on Intelligent Technologies for Interactive Entertainment. Virtual Event, 2020: 267–281.
[52]	ZHANG Rongxiang. Virtual reality games based on brain computer interface[C]. Proceedings of 2020 International Conference on Intelligent Computing and Human-Computer Interaction, Sanya, China, 2020: 227–230.
[53]	XU Lichao, XU Minpeng, KE Yufeng, et al. Cross-dataset variability problem in EEG decoding with deep learning[J]. Frontiers in Human Neuroscience, 2020, 14: 103. doi: 10.3389/fnhum.2020.00103
[54]	XU Minpeng, XIAO Xiaolin, WANG Yijun, et al. A brain–computer interface based on miniature-event-related potentials induced by very small lateral visual stimuli[J]. IEEE Transactions on Biomedical Engineering, 2018, 65(5): 1166–1175. doi: 10.1109/TBME.2018.2799661