高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

视觉诱发响应研究中EEG与fMRI联合应用方法综述

危志伟 肖晓琳 许敏鹏 明东

危志伟, 肖晓琳, 许敏鹏, 明东. 视觉诱发响应研究中EEG与fMRI联合应用方法综述[J]. 电子与信息学报. doi: 10.11999/JEIT250781
引用本文: 危志伟, 肖晓琳, 许敏鹏, 明东. 视觉诱发响应研究中EEG与fMRI联合应用方法综述[J]. 电子与信息学报. doi: 10.11999/JEIT250781
WEI Zhiwei, XIAO Xiaolin, XU Minpeng, MING Dong. A Review of Joint EEG-fMRI Methods for Visual Evoked Response Studies[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250781
Citation: WEI Zhiwei, XIAO Xiaolin, XU Minpeng, MING Dong. A Review of Joint EEG-fMRI Methods for Visual Evoked Response Studies[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250781

视觉诱发响应研究中EEG与fMRI联合应用方法综述

doi: 10.11999/JEIT250781 cstr: 32379.14.JEIT250781
基金项目: 国家自然科学基金(W2511072, 62106170)
详细信息
    作者简介:

    危志伟:男,硕士生,研究方向为视觉脑机接口

    肖晓琳:女,博士,副教授,研究方向为视觉型脑机接口、高速高维脑操控系统、视觉脑控游戏、临床视功能检测

    许敏鹏:男,博士,教授,研究方向为脑-机接口,神经信号处理和神经调控

    明东:男,博士,教授,研究方向为脑-机接口、神经 再生与修复、神经仿生与智能、神经刺激与调节、神经传感与成像

    通讯作者:

    肖晓琳 xiaoxiao0@tju.edu.cn

  • 中图分类号: TN99; R741.044

A Review of Joint EEG-fMRI Methods for Visual Evoked Response Studies

Funds: The National Natural Science Foundation of China (W2511072, 62106170)
  • 摘要: 利用脑电图(EEG)和功能性磁共振成像(fMRI)等无创脑成像技术研究视觉诱发响应,是探索人类视觉信息加工机制的重要途径。EEG-fMRI联合技术综合了EEG的高时间分辨优势与fMRI的高空间分辨优势,从更全面的神经时空活动视角为视觉诱发响应研究提供了方法支撑。该文系统综述了视觉诱发响应研究中EEG和fMRI的经典融合方法和EEG-fMRI联合技术在神经科学领域的应用情况,最后讨论了EEG-fMRI联合应用方法在视觉诱发响应研究中面临的技术挑战和未来发展方向。
  • 图  1  1999~2024 EEG-fMRI联合应用出版文献数量与被引情况

    图  2  EEG-fMRI联合应用相关文献按WOS中主题分析的结果

    图  3  同步EEG-fMRI采集系统结构简图

    图  4  EEG与fMRI数据的对称融合和非对称融合

    图  5  基于GLM模型的EEG-informed fMRI示意图

    图  6  基于源重建的fMRI-informed EEG示意图

    图  7  基于joint ICA的EEG-fMRI对称融合示意图

  • [1] RENTON A I, KLEIN D J, LIVEZEY J A, et al. Video-evoked neuromarkers of visual function in age-related macular degeneration[J]. Frontiers in Human Neuroscience, 2025, 19: 1569282. doi: 10.3389/fnhum.2025.1569282.
    [2] KARAMI A, CASTALDI E, EGER E, et al. Distinct neural representational geometries of numerosity in early visual and association regions across visual streams[J]. Communications Biology, 2025, 8(1): 1029. doi: 10.1038/s42003-025-08395-z.
    [3] BOYLAN M R, PANITZ C, TEBBE A L, et al. Feature-based attentional amplitude modulations of the steady-state visual evoked potentials reflect blood oxygen level dependent changes in feature-sensitive visual areas[J]. Journal of Cognitive Neuroscience, 2023, 35(9): 1493–1507. doi: 10.1162/jocn_a_02030.
    [4] SCHMITT O. Relationships and representations of brain structures, connectivity, dynamics and functions[J]. Progress in Neuro-Psychopharmacology and Biological Psychiatry, 2025, 138: 111332. doi: 10.1016/j.pnpbp.2025.111332.
    [5] COOK A J, IM H Y, and GIASCHI D E. Large-scale functional networks underlying visual attention[J]. Neuroscience & Biobehavioral Reviews, 2025, 173: 106165. doi: 10.1016/j.neubiorev.2025.106165.
    [6] TU Y H, TAN A, BAI Y R, et al. Decoding subjective intensity of nociceptive pain from pre-stimulus and post-stimulus brain activities[J]. Frontiers in Computational Neuroscience, 2016, 10: 32. doi: 10.3389/fncom.2016.00032.
    [7] EGAN M K, LARSEN R, WIRSICH J, et al. Safety and data quality of EEG recorded simultaneously with multi-band fMRI[J]. PLoS One, 2021, 16(7): e0238485. doi: 10.1371/journal.pone.0238485.
    [8] 高鹏, 李海芳. 多模态神经影像技术研究进展与实践[J]. 太原理工大学学报, 2022, 53(3): 420–31. doi: 10.16355/j.cnki.issn1007-9432tyut.2022.03.007.

    GAO Peng and LI Haifang. Research progress and practical application of multimodal neuroimaging technology[J]. Journal of Taiyuan University of Technology, 2022, 53(3): 420–431. doi: 10.16355/j.cnki.issn1007-9432tyut.2022.03.007.
    [9] CHANG C and CHEN J E. Multimodal EEG-fMRI: Advancing insight into large-scale human brain dynamics[J]. Current Opinion in Biomedical Engineering, 2021, 18: 100279. doi: 10.1016/j.cobme.2021.100279.
    [10] ODUSAMI M, MASKELIŪNAS R, DAMAŠEVIČIUS R, et al. Machine learning with multimodal neuroimaging data to classify stages of Alzheimer's disease: A systematic review and meta-analysis[J]. Cognitive Neurodynamics, 2024, 18(3): 775–794. doi: 10.1007/s11571-023-09993-5.
    [11] LEI Xu, VALDES-SOSA P A, and YAO Dezhong. EEG/fMRI fusion based on independent component analysis: Integration of data-driven and model-driven methods[J]. Journal of Integrative Neuroscience, 2012, 11(3): 313–337. doi: 10.1142/s0219635212500203.
    [12] CHATZICHRISTOS C, KOFIDIS E, VAN PAESSCHEN W, et al. Early soft and flexible fusion of electroencephalography and functional magnetic resonance imaging via double coupled matrix tensor factorization for multisubject group analysis[J]. Human Brain Mapping, 2022, 43(4): 1231–1255. doi: 10.1002/hbm.25717.
    [13] VAN EYNDHOVEN S, HUNYADI B, DE LATHAUWER L, et al. Flexible fusion of electroencephalography and functional magnetic resonance imaging: Revealing neural-hemodynamic coupling through structured matrix-tensor factorization[C]. 2017 25th European Signal Processing Conference, Kos, Greece, 2017: 26–30. doi: 10.23919/EUSIPCO.2017.8081162.
    [14] JACOB L P L, BAILES S M, WILLIAMS S D, et al. Brainwide hemodynamics predict EEG neural rhythms across sleep and wakefulness in humans[J]. PLoS Computational Biology, 2025, 21(9): e1013497. doi: 10.1371/journal.pcbi.1013497.
    [15] HAUK O and CLARKE A. Patterns of language[J]. Language, Cognition and Neuroscience, 2024, 39(8): 959–961. doi: 10.1080/23273798.2024.2388329.
    [16] JONMOHAMADI Y, FORSYTH A, MCMILLAN R, et al. Constrained temporal parallel decomposition for EEG-fMRI fusion[J]. Journal of Neural Engineering, 2019, 16(1): 016017. doi: 10.1088/1741-2552/aaefda.
    [17] BéNAR C G, SCHöN D, GRIMAULT S, et al. Single‐trial analysis of oddball event‐related potentials in simultaneous EEG‐fMRI[J]. Human Brain Mapping, 2007, 28(7): 602–613. doi: 10.1002/hbm.20289.
    [18] HUSTER R J, DEBENER S, EICHELE T, et al. Methods for simultaneous EEG-fMRI: An introductory review[J]. Journal of Neuroscience, 2012, 32(18): 6053–6060. doi: 10.1523/jneurosci.0447-12.2012.
    [19] PROKOPIOU P C, XIFRA-PORXAS A, KASSINOPOULOS M, et al. Modeling the hemodynamic response function using EEG-fMRI data during eyes-open resting-state conditions and motor task execution[J]. Brain Topography, 2022, 35(3): 302–321. doi: 10.1007/s10548-022-00898-w.
    [20] GUO Qian, ZHOU Tiantong, LI Wenjie, et al. Single-trial EEG-informed fMRI analysis of emotional decision problems in hot executive function[J]. Brain and Behavior, 2017, 7(7): e00728. doi: 10.1002/brb3.728.
    [21] MURTA T, LEITE M, CARMICHAEL D W, et al. Electrophysiological correlates of the BOLD signal for EEG-informed fMRI[J]. Human Brain Mapping, 2015, 36(1): 391–414. doi: 10.1002/hbm.22623.
    [22] MORADI N, GOODYEAR B G, and SOTERO R C. Deep EEG source localization via EMD-based fMRI high spatial frequency[J]. PLoS One, 2024, 19(3): e0299284. doi: 10.1371/journal.pone.0299284.
    [23] LIU Ke, YU Zhuliang, WU Wei, et al. fMRI-SI-STBF: An fMRI-informed Bayesian electromagnetic spatio-temporal extended source imaging[J]. Neurocomputing, 2021, 462: 14–30. doi: 10.1016/j.neucom.2021.06.066.
    [24] WANG Hailing, LEI Xu, ZHAN Zhichao, et al. A new fMRI informed mixed-norm constrained algorithm for EEG source localization[J]. IEEE Access, 2018, 6: 8258–8269. doi: 10.1109/access.2018.2792442.
    [25] BRINKMANN B H. Technical considerations in EEG source imaging[J]. Journal of Clinical Neurophysiology, 2024, 41(1): 2–7. doi: 10.1097/wnp.0000000000001029.
    [26] MAGRI C, SCHRIDDE U, MURAYAMA Y, et al. The amplitude and timing of the BOLD signal reflects the relationship between local field potential power at different frequencies[J]. Journal of Neuroscience, 2012, 32(4): 1395–1407. doi: 10.1523/jneurosci.3985-11.2012.
    [27] SUN Peng, HICKS Y, and SETCHI R. Joint EEG-fMRI model for EEG source separation[C]. 2014 IEEE International Conference on Systems, Man, and Cybernetics, San Diego, USA, 2014: 2234–2239. doi: 10.1109/SMC.2014.6974257.
    [28] HEUGEL N, BEARDSLEY S A, and LIEBENTHAL E. EEG and fMRI coupling and decoupling based on joint independent component analysis (jICA)[J]. Journal of Neuroscience Methods, 2022, 369: 109477. doi: 10.1016/j.jneumeth.2022.109477.
    [29] LI Junhua, CHEN Yu, TAYA F, et al. A unified canonical correlation analysis-based framework for removing gradient artifact in concurrent EEG/fMRI recording and motion artifact in walking recording from EEG signal[J]. Medical & Biological Engineering & Computing, 2017, 55(9): 1669–1681. doi: 10.1007/s11517-017-1620-3.
    [30] JONMOHAMADI Y, MUTHUKUMARASWAMY S, CHEN J, et al. Extraction of common task features in EEG-fMRI data using coupled tensor-tensor decomposition[J]. Brain Topography, 2020, 33(5): 636–650. doi: 10.1007/s10548-020-00787-0.
    [31] ZAFAR R, KAMEL N, NAUFAL M, et al. A study of decoding human brain activities from simultaneous data of EEG and fMRI using MVPA[J]. Australasian Physical & Engineering Sciences in Medicine, 2018, 41(3): 633–645. doi: 10.1007/s13246-018-0656-5.
    [32] SCRIVENER C L, TEED J A, and SILSON E H. Visual imagery of familiar people and places in category selective cortex[J]. Neuroscience of Consciousness, 2025, 2025(1): niaf006. doi: 10.1093/nc/niaf006.
    [33] PITZALIS S, STRAPPINI F, BULTRINI A, et al. Detailed spatiotemporal brain mapping of chromatic vision combining high‐resolution VEP with fMRI and retinotopy[J]. Human Brain Mapping, 2018, 39(7): 2868–2886. doi: 10.1002/hbm.24046.
    [34] ITTHIPURIPAT S, SPRAGUE T C, and SERENCES J T. Functional MRI and EEG index complementary attentional modulations[J]. Journal of Neuroscience, 2019, 39(31): 6162–6179. doi: 10.1523/JNEUROSCI.2519-18.2019.
    [35] HEDGE C, STOTHART G, TODD JONES J, et al. A frontal attention mechanism in the visual mismatch negativity[J]. Behavioural Brain Research, 2015, 293: 173–181. doi: 10.1016/j.bbr.2015.07.022.
    [36] FRANCO R M, D'ALMEIDA O C, RAIMUNDO M, et al. The clinical use of retinotopy in functional hemianopia[J]. Neuro-Ophthalmology, 2025. doi: 10.1080/01658107.2025.2474542. (查阅网上资料,未找到对应的卷期页码信息,请确认).
    [37] HUANG Yujing, CROMARTY R, JIA Lina, et al. Attention network dysfunctions in lewy body dementia and alzheimer's disease[J]. Journal of Clinical Medicine, 2024, 13(22): 6691. doi: 10.3390/jcm13226691.
    [38] WANG Xinmei, CUI Dongmei, ZHENG Ling, et al. Combination of blood oxygen level-dependent functional magnetic resonance imaging and visual evoked potential recordings for abnormal visual cortex in two types of amblyopia[J]. Molecular Vision, 2012, 18: 909–919.
    [39] BARTOLINI E, PESARESI I, FABBRI S, et al. Abnormal response to photic stimulation in Juvenile Myoclonic Epilepsy: An EEG-fMRI study[J]. Epilepsia, 2014, 55(7): 1038–1047. doi: 10.1111/epi.12634.
    [40] ANWAR M N, BONZANO L, ROSSI-SEBASTIANO D, et al. Simultaneous recording of VEP and fMRI to study optic neuritis in multiple sclerosis[C]. Proceedings of the Fifth IASTED International Conference: Biomedical Engineering, Innsbruck, Austria, 2007: 197–202.
    [41] SINGH S P, MISHRA S, GUPTA S, et al. Functional mapping of the brain for brain-computer interfacing: A review[J]. Electronics, 2023, 12(3): 604. doi: 10.3390/electronics12030604.
    [42] AHMAD R F, MALIK A S, KAMEL N, et al. Visual brain activity patterns classification with simultaneous EEG-fMRI: A multimodal approach[J]. Technology Health Care, 2017, 25(3): 471–485. doi: 10.3233/thc-161286.
    [43] WATANABE N, MIYOSHI K, JIMURA K, et al. Multimodal deep neural decoding reveals highly resolved spatiotemporal profile of visual object representation in humans[J]. NeuroImage, 2023, 275: 120164. doi: 10.1016/j.neuroimage.2023.120164.
    [44] MAZIERO D, STENGER V A, and CARMICHAEL D W. Unified retrospective EEG motion educated artefact suppression for EEG-fMRI to suppress magnetic field gradient artefacts during motion[J]. Brain Topography, 2021, 34(6): 745–761. doi: 10.1007/s10548-021-00870-0.
    [45] JORGE J, GROUILLER F, IPEK Ö, et al. Simultaneous EEG-fMRI at ultra-high field: Artifact prevention and safety assessment[J]. NeuroImage, 2015, 105: 132–144. doi: 10.1016/j.neuroimage.2014.10.055.
    [46] CICERO N G, FULTZ N E, JEONG H, et al. High-quality multimodal MRI with simultaneous EEG using conductive ink and polymer-thick film nets[J]. Journal of Neural Engineering, 2024, 21(6): 066004. doi: 10.1088/1741-2552/ad8837.
    [47] WARBRICK T. Simultaneous EEG-fMRI: What have we learned and what does the future hold?[J]. Sensors, 2022, 22(6): 2262. doi: 10.3390/s22062262.
    [48] ROSA M J, KILNER J, BLANKENBURG F, et al. Estimating the transfer function from neuronal activity to BOLD using simultaneous EEG-fMRI[J]. NeuroImage, 2010, 49(2): 1496–1509. doi: 10.1016/j.neuroimage.2009.09.011.
    [49] PHADIKAR S, PUSULURI K, IRAJI A, et al. Integrating fMRI spatial network dynamics and EEG spectral power: Insights into resting state connectivity[J]. Frontiers in Neuroscience, 2025, 19: 1484954. doi: 10.3389/fnins.2025.1484954.
    [50] HAXBY J V, GUNTUPALLI J S, NASTASE S A, et al. Hyperalignment: Modeling shared information encoded in idiosyncratic cortical topographies[J]. eLife, 2020, 9: e56601. doi: 10.7554/eLife.56601.
    [51] PAN Jiadong, XIA Jie, ZHANG Fan, et al. 7T magnetic compatible multimodality electrophysiological signal recording system[J]. Electronics, 2023, 12(17): 3648. doi: 10.3390/electronics12173648.
    [52] CAETANO G, ESTEVES I, VOURVOPOULOS A, et al. NeuXus open-source tool for real-time artifact reduction in simultaneous EEG-fMRI[J]. NeuroImage, 2023, 280: 120353. doi: 10.1016/j.neuroimage.2023.120353.
    [53] LEVITT J, YANG Z N, WILLIAMS S D, et al. EEG-LLAMAS: A low-latency neurofeedback platform for artifact reduction in EEG-fMRI[J]. NeuroImage, 2023, 273: 120092. doi: 10.1016/j.neuroimage.2023.120092.
    [54] PERNET C R, APPELHOFF S, GORGOLEWSKI K J, et al. EEG-BIDS, an extension to the brain imaging data structure for electroencephalography[J]. Scientific Data, 2019, 6(1): 103. doi: 10.1038/s41597-019-0104-8.
    [55] LIU Junwei, CEN Xiaoping, YI Chenxin, et al. Challenges in AI-driven biomedical multimodal data fusion and analysis[J]. Genomics, Proteomics & Bioinformatics, 2025, 23(1): qzaf011. doi: 10.1093/gpbjnl/qzaf011.
    [56] LIFANOV-CARR J, GRIFFITHS B J, LINDE-DOMINGO J, et al. Reconstructing spatiotemporal trajectories of visual object memories in the human brain[J]. eNeuro, 2024, 11(9): ENEURO. 0091-24. doi: 10.1523/eneuro.0091-24.2024. (查阅网上资料,请核对页码信息).
    [57] GUENTHER S, KOSMYNA N, and MAES P. Image classification and reconstruction from low-density EEG[J]. Scientific Reports, 2024, 14(1): 16436. doi: 10.1038/s41598-024-66228-1.
    [58] PAN Hongguang, LI Zhuoyi, FU Yunpeng, et al. Reconstructing visual stimulus representation from EEG signals based on deep visual representation model[J]. IEEE Transactions on Human-Machine Systems, 2024, 54(6): 711–722. doi: 10.1109/thms.2024.3407875.
    [59] YIN Xu, WU Zhengping, and WANG Haixian. A novel DRL-guided sparse voxel decoding model for reconstructing perceived images from brain activity[J]. Journal of Neuroscience Methods, 2024, 412: 110292. doi: 10.1016/j.jneumeth.2024.110292.
    [60] ZHANG Yuanyuan, LI Xinyu, and LIU Baolin. An fMRI-based visual decoding framework combined with two-stage learning and asynchronous iterative update strategy[J]. Pattern Analysis and Applications, 2024, 27(4): 131. doi: 10.1007/s10044-024-01347-z.
  • 加载中
图(7)
计量
  • 文章访问数:  21
  • HTML全文浏览量:  10
  • PDF下载量:  2
  • 被引次数: 0
出版历程
  • 修回日期:  2025-12-02
  • 录用日期:  2025-12-02
  • 网络出版日期:  2026-01-10

目录

    /

    返回文章
    返回