A Review of Joint EEG-fMRI Methods for Visual Evoked Response Studies

WEI Zhiwei; XIAO Xiaolin; XU Minpeng; MING Dong

doi:10.11999/JEIT250781

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 >

WEI Zhiwei, XIAO Xiaolin, XU Minpeng, MING Dong. A Review of Joint EEG-fMRI Methods for Visual Evoked Response Studies[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250781

Citation:

WEI Zhiwei, XIAO Xiaolin, XU Minpeng, MING Dong. A Review of Joint EEG-fMRI Methods for Visual Evoked Response Studies[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250781

WEI Zhiwei, XIAO Xiaolin, XU Minpeng, MING Dong. A Review of Joint EEG-fMRI Methods for Visual Evoked Response Studies[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250781

Citation:

WEI Zhiwei, XIAO Xiaolin, XU Minpeng, MING Dong. A Review of Joint EEG-fMRI Methods for Visual Evoked Response Studies[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250781

PDF( 2564 KB)

A Review of Joint EEG-fMRI Methods for Visual Evoked Response Studies

doi: 10.11999/JEIT250781 cstr: 32379.14.JEIT250781

WEI Zhiwei¹,
XIAO Xiaolin^{1, 2
,
,},
XU Minpeng^{1, 2},
MING Dong^{1, 2}

1.
School of Precision Instrument and Opto-electronics Engineering, Tianjin University, Tianjin 300072, China
2.
Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, China

Funds: The National Natural Science Foundation of China (W2511072, 62106170)

Accepted Date: 2025-12-02
Rev Recd Date: 2025-12-02

Available Online: 2026-01-10

Abstract

Abstract

Significance The study of visual evoked responses (VERs) using non-invasive neuroimaging techniques is a cornerstone of neuroscience, providing critical insights into the mechanisms of human visual information processing. Among the available modalities, electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) are paramount. EEG captures neural electrical activity with millisecond-level temporal resolution but is fundamentally limited by its poor spatial localization capabilities. Conversely, fMRI provides millimeter-level spatial precision by measuring the blood-oxygen-level-dependent (BOLD) signal, yet its temporal resolution is inherently constrained by the sluggish nature of hemodynamic responses. This intrinsic trade-off between temporal and spatial resolution significantly hampers the ability of any single modality to fully elucidate complex visual processes such as attentional modulation, motion perception, and multi-sensory integration. To overcome this bottleneck, the joint application of EEG and fMRI has emerged as a powerful multimodal approach. By synchronously acquiring both datasets, this integrated technique synergistically combines the distinct strengths of each modality, offering a comprehensive spatiotemporal perspective on the complex dynamics of visual neural networks. Despite its growing adoption, existing literature often lacks a focused, systematic review that specifically details the core methodologies, illustrates key applications, and outlines the persistent challenges and future trends of joint EEG-fMRI in VER research. This review aims to fill this gap by providing a comprehensive and structured overview of the field, serving as a foundational reference for researchers seeking to leverage this advanced technique to explore the visual system. Progress This review first elaborates on the foundational technologies that enable joint EEG-fMRI studies, starting with the synchronous acquisition of data. This is addressed through MR-compatible EEG systems and dedicated synchronization hardware. The core of the review then systematically analyzes data fusion methodologies, which are categorized into asymmetric and symmetric approaches. Asymmetric fusion uses one modality to constrain the analysis of the other, exemplified by EEG-informed fMRI analysis, which uses single-trial EEG features to model fMRI data, and fMRI-informed EEG source imaging, which uses fMRI activation maps as spatial priors to enhance source localization accuracy. In contrast, symmetric fusion treats both modalities equally, with data-driven techniques like joint independent component analysis (joint ICA) being widely adopted to reveal shared underlying neural sources without strong biophysical assumptions. The application of these methodologies has yielded significant breakthroughs across multiple domains. In visual mechanism analysis, the technique has been instrumental in dissecting the complex feedforward and feedback dynamics of cortical areas involved in vision. In clinical diagnosis and evaluation, joint EEG-fMRI provides objective neurophysiological biomarkers for visual disorders like amblyopia and epilepsy by identifying distinct patterns of cortical activation deficits and network dysfunctions. In the field of brain-computer interfaces (BCIs), the fusion of multimodal features has significantly improved the accuracy and robustness of decoding visual intentions. Conclusions This review critically examines the joint EEG-fMRI landscape for VER studies, systematically classifying the key data acquisition and fusion methodologies and highlighting their representative applications. The analysis reveals that the choice of an optimal fusion strategy—be it asymmetric or symmetric, data-driven or model-driven—is highly dependent on the specific research question, available data quality, and underlying assumptions. While the technique has proven useful in advancing basic neuroscience, clinical diagnostics, and BCI development, its broader adoption is still hindered by persistent challenges. At the system level, hardware-induced artifacts, particularly the severe electromagnetic interference in ultra-high-field MRI environments, remain a major technical obstacle that compromises data quality. At the algorithmic level, the inherent mismatch in spatiotemporal scales between the fast, transient EEG signals and the slow, delayed BOLD response continues to pose a core fusion challenge. This is further complicated by high inter-subject variability in neural responses, which limits the generalizability of analytical models and decoding algorithms across individuals. These limitations underscore the need for continued innovation in both hardware engineering and computational methods to unlock the full potential of this powerful multimodal technique. Prospects Looking ahead, the research landscape for joint EEG-fMRI methods in VER studies is poised for significant evolution, constituting a long-term and complex process. With the integration of emerging technologies such as artificial intelligence, the methodological frameworks in this domain will evolve toward greater intelligence and automation. System-level trends point toward the development of next-generation hardware, including ultra-high-field MRI systems combined with artifact-immune EEG sensors and real-time artifact correction algorithms. Furthermore, the establishment of open-access, multi-center EEG-fMRI databases (following standards like BIDS) and standardized analysis pipelines will be crucial for improving the reproducibility and comparability of research findings, fostering a collaborative ecosystem. Algorithm-level trends are increasingly centered on the integration of artificial intelligence and deep learning. End-to-end neural network architectures, such as those incorporating spatiotemporal attention mechanisms, hold the promise of learning the complex, non-linear transformations between EEG and fMRI data directly, thus overcoming the limitations of traditional linear models. Moreover, leveraging transfer learning and personalized modeling frameworks can address the challenge of inter-subject variability, leading to the development of adaptive and robust models for visual decoding and clinical applications. Concurrently, as clinical and BCI applications accelerate, the critical challenge of balancing model complexity with interpretive clarity and computational efficiency warrants in-depth investigation. Ultimately, these synergistic advancements in hardware and algorithms will deepen our understanding of the visual system’s computational principles, refine the diagnosis and treatment of visual disorders, and propel the development of more intuitive and powerful brain-computer interfaces.
- EEG-fMRI integration,
- Visual evoked response (VER),
- Electroencephalography (EEG),
- Functional Magnetic Resonance Imaging (fMRI),
- Data fusion and analysis

FullText(HTML)

References(60)

References

[1]	RENTON A I, KLEIN D J, LIVEZEY J A, et al. Video-evoked neuromarkers of visual function in age-related macular degeneration[J]. Frontiers in Human Neuroscience, 2025, 19: 1569282. doi: 10.3389/fnhum.2025.1569282.
[2]	KARAMI A, CASTALDI E, EGER E, et al. Distinct neural representational geometries of numerosity in early visual and association regions across visual streams[J]. Communications Biology, 2025, 8(1): 1029. doi: 10.1038/s42003-025-08395-z.
[3]	BOYLAN M R, PANITZ C, TEBBE A L, et al. Feature-based attentional amplitude modulations of the steady-state visual evoked potentials reflect blood oxygen level dependent changes in feature-sensitive visual areas[J]. Journal of Cognitive Neuroscience, 2023, 35(9): 1493–1507. doi: 10.1162/jocn_a_02030.
[4]	SCHMITT O. Relationships and representations of brain structures, connectivity, dynamics and functions[J]. Progress in Neuro-Psychopharmacology and Biological Psychiatry, 2025, 138: 111332. doi: 10.1016/j.pnpbp.2025.111332.
[5]	COOK A J, IM H Y, and GIASCHI D E. Large-scale functional networks underlying visual attention[J]. Neuroscience & Biobehavioral Reviews, 2025, 173: 106165. doi: 10.1016/j.neubiorev.2025.106165.
[6]	TU Y H, TAN A, BAI Y R, et al. Decoding subjective intensity of nociceptive pain from pre-stimulus and post-stimulus brain activities[J]. Frontiers in Computational Neuroscience, 2016, 10: 32. doi: 10.3389/fncom.2016.00032.
[7]	EGAN M K, LARSEN R, WIRSICH J, et al. Safety and data quality of EEG recorded simultaneously with multi-band fMRI[J]. PLoS One, 2021, 16(7): e0238485. doi: 10.1371/journal.pone.0238485.
[8]	高鹏, 李海芳. 多模态神经影像技术研究进展与实践[J]. 太原理工大学学报, 2022, 53(3): 420–31. doi: 10.16355/j.cnki.issn1007-9432tyut.2022.03.007. GAO Peng and LI Haifang. Research progress and practical application of multimodal neuroimaging technology[J]. Journal of Taiyuan University of Technology, 2022, 53(3): 420–431. doi: 10.16355/j.cnki.issn1007-9432tyut.2022.03.007.
[9]	CHANG C and CHEN J E. Multimodal EEG-fMRI: Advancing insight into large-scale human brain dynamics[J]. Current Opinion in Biomedical Engineering, 2021, 18: 100279. doi: 10.1016/j.cobme.2021.100279.
[10]	ODUSAMI M, MASKELIŪNAS R, DAMAŠEVIČIUS R, et al. Machine learning with multimodal neuroimaging data to classify stages of Alzheimer's disease: A systematic review and meta-analysis[J]. Cognitive Neurodynamics, 2024, 18(3): 775–794. doi: 10.1007/s11571-023-09993-5.
[11]	LEI Xu, VALDES-SOSA P A, and YAO Dezhong. EEG/fMRI fusion based on independent component analysis: Integration of data-driven and model-driven methods[J]. Journal of Integrative Neuroscience, 2012, 11(3): 313–337. doi: 10.1142/s0219635212500203.
[12]	CHATZICHRISTOS C, KOFIDIS E, VAN PAESSCHEN W, et al. Early soft and flexible fusion of electroencephalography and functional magnetic resonance imaging via double coupled matrix tensor factorization for multisubject group analysis[J]. Human Brain Mapping, 2022, 43(4): 1231–1255. doi: 10.1002/hbm.25717.
[13]	VAN EYNDHOVEN S, HUNYADI B, DE LATHAUWER L, et al. Flexible fusion of electroencephalography and functional magnetic resonance imaging: Revealing neural-hemodynamic coupling through structured matrix-tensor factorization[C]. 2017 25th European Signal Processing Conference, Kos, Greece, 2017: 26–30. doi: 10.23919/EUSIPCO.2017.8081162.
[14]	JACOB L P L, BAILES S M, WILLIAMS S D, et al. Brainwide hemodynamics predict EEG neural rhythms across sleep and wakefulness in humans[J]. PLoS Computational Biology, 2025, 21(9): e1013497. doi: 10.1371/journal.pcbi.1013497.
[15]	HAUK O and CLARKE A. Patterns of language[J]. Language, Cognition and Neuroscience, 2024, 39(8): 959–961. doi: 10.1080/23273798.2024.2388329.
[16]	JONMOHAMADI Y, FORSYTH A, MCMILLAN R, et al. Constrained temporal parallel decomposition for EEG-fMRI fusion[J]. Journal of Neural Engineering, 2019, 16(1): 016017. doi: 10.1088/1741-2552/aaefda.
[17]	BéNAR C G, SCHöN D, GRIMAULT S, et al. Single‐trial analysis of oddball event‐related potentials in simultaneous EEG‐fMRI[J]. Human Brain Mapping, 2007, 28(7): 602–613. doi: 10.1002/hbm.20289.
[18]	HUSTER R J, DEBENER S, EICHELE T, et al. Methods for simultaneous EEG-fMRI: An introductory review[J]. Journal of Neuroscience, 2012, 32(18): 6053–6060. doi: 10.1523/jneurosci.0447-12.2012.
[19]	PROKOPIOU P C, XIFRA-PORXAS A, KASSINOPOULOS M, et al. Modeling the hemodynamic response function using EEG-fMRI data during eyes-open resting-state conditions and motor task execution[J]. Brain Topography, 2022, 35(3): 302–321. doi: 10.1007/s10548-022-00898-w.
[20]	GUO Qian, ZHOU Tiantong, LI Wenjie, et al. Single-trial EEG-informed fMRI analysis of emotional decision problems in hot executive function[J]. Brain and Behavior, 2017, 7(7): e00728. doi: 10.1002/brb3.728.
[21]	MURTA T, LEITE M, CARMICHAEL D W, et al. Electrophysiological correlates of the BOLD signal for EEG-informed fMRI[J]. Human Brain Mapping, 2015, 36(1): 391–414. doi: 10.1002/hbm.22623.
[22]	MORADI N, GOODYEAR B G, and SOTERO R C. Deep EEG source localization via EMD-based fMRI high spatial frequency[J]. PLoS One, 2024, 19(3): e0299284. doi: 10.1371/journal.pone.0299284.
[23]	LIU Ke, YU Zhuliang, WU Wei, et al. fMRI-SI-STBF: An fMRI-informed Bayesian electromagnetic spatio-temporal extended source imaging[J]. Neurocomputing, 2021, 462: 14–30. doi: 10.1016/j.neucom.2021.06.066.
[24]	WANG Hailing, LEI Xu, ZHAN Zhichao, et al. A new fMRI informed mixed-norm constrained algorithm for EEG source localization[J]. IEEE Access, 2018, 6: 8258–8269. doi: 10.1109/access.2018.2792442.
[25]	BRINKMANN B H. Technical considerations in EEG source imaging[J]. Journal of Clinical Neurophysiology, 2024, 41(1): 2–7. doi: 10.1097/wnp.0000000000001029.
[26]	MAGRI C, SCHRIDDE U, MURAYAMA Y, et al. The amplitude and timing of the BOLD signal reflects the relationship between local field potential power at different frequencies[J]. Journal of Neuroscience, 2012, 32(4): 1395–1407. doi: 10.1523/jneurosci.3985-11.2012.
[27]	SUN Peng, HICKS Y, and SETCHI R. Joint EEG-fMRI model for EEG source separation[C]. 2014 IEEE International Conference on Systems, Man, and Cybernetics, San Diego, USA, 2014: 2234–2239. doi: 10.1109/SMC.2014.6974257.
[28]	HEUGEL N, BEARDSLEY S A, and LIEBENTHAL E. EEG and fMRI coupling and decoupling based on joint independent component analysis (jICA)[J]. Journal of Neuroscience Methods, 2022, 369: 109477. doi: 10.1016/j.jneumeth.2022.109477.
[29]	LI Junhua, CHEN Yu, TAYA F, et al. A unified canonical correlation analysis-based framework for removing gradient artifact in concurrent EEG/fMRI recording and motion artifact in walking recording from EEG signal[J]. Medical & Biological Engineering & Computing, 2017, 55(9): 1669–1681. doi: 10.1007/s11517-017-1620-3.
[30]	JONMOHAMADI Y, MUTHUKUMARASWAMY S, CHEN J, et al. Extraction of common task features in EEG-fMRI data using coupled tensor-tensor decomposition[J]. Brain Topography, 2020, 33(5): 636–650. doi: 10.1007/s10548-020-00787-0.
[31]	ZAFAR R, KAMEL N, NAUFAL M, et al. A study of decoding human brain activities from simultaneous data of EEG and fMRI using MVPA[J]. Australasian Physical & Engineering Sciences in Medicine, 2018, 41(3): 633–645. doi: 10.1007/s13246-018-0656-5.
[32]	SCRIVENER C L, TEED J A, and SILSON E H. Visual imagery of familiar people and places in category selective cortex[J]. Neuroscience of Consciousness, 2025, 2025(1): niaf006. doi: 10.1093/nc/niaf006.
[33]	PITZALIS S, STRAPPINI F, BULTRINI A, et al. Detailed spatiotemporal brain mapping of chromatic vision combining high‐resolution VEP with fMRI and retinotopy[J]. Human Brain Mapping, 2018, 39(7): 2868–2886. doi: 10.1002/hbm.24046.
[34]	ITTHIPURIPAT S, SPRAGUE T C, and SERENCES J T. Functional MRI and EEG index complementary attentional modulations[J]. Journal of Neuroscience, 2019, 39(31): 6162–6179. doi: 10.1523/JNEUROSCI.2519-18.2019.
[35]	HEDGE C, STOTHART G, TODD JONES J, et al. A frontal attention mechanism in the visual mismatch negativity[J]. Behavioural Brain Research, 2015, 293: 173–181. doi: 10.1016/j.bbr.2015.07.022.
[36]	FRANCO R M, D'ALMEIDA O C, RAIMUNDO M, et al. The clinical use of retinotopy in functional hemianopia[J]. Neuro-Ophthalmology, 2025. doi: 10.1080/01658107.2025.2474542. (查阅网上资料,未找到对应的卷期页码信息,请确认).
[37]	HUANG Yujing, CROMARTY R, JIA Lina, et al. Attention network dysfunctions in lewy body dementia and alzheimer's disease[J]. Journal of Clinical Medicine, 2024, 13(22): 6691. doi: 10.3390/jcm13226691.
[38]	WANG Xinmei, CUI Dongmei, ZHENG Ling, et al. Combination of blood oxygen level-dependent functional magnetic resonance imaging and visual evoked potential recordings for abnormal visual cortex in two types of amblyopia[J]. Molecular Vision, 2012, 18: 909–919.
[39]	BARTOLINI E, PESARESI I, FABBRI S, et al. Abnormal response to photic stimulation in Juvenile Myoclonic Epilepsy: An EEG-fMRI study[J]. Epilepsia, 2014, 55(7): 1038–1047. doi: 10.1111/epi.12634.
[40]	ANWAR M N, BONZANO L, ROSSI-SEBASTIANO D, et al. Simultaneous recording of VEP and fMRI to study optic neuritis in multiple sclerosis[C]. Proceedings of the Fifth IASTED International Conference: Biomedical Engineering, Innsbruck, Austria, 2007: 197–202.
[41]	SINGH S P, MISHRA S, GUPTA S, et al. Functional mapping of the brain for brain-computer interfacing: A review[J]. Electronics, 2023, 12(3): 604. doi: 10.3390/electronics12030604.
[42]	AHMAD R F, MALIK A S, KAMEL N, et al. Visual brain activity patterns classification with simultaneous EEG-fMRI: A multimodal approach[J]. Technology Health Care, 2017, 25(3): 471–485. doi: 10.3233/thc-161286.
[43]	WATANABE N, MIYOSHI K, JIMURA K, et al. Multimodal deep neural decoding reveals highly resolved spatiotemporal profile of visual object representation in humans[J]. NeuroImage, 2023, 275: 120164. doi: 10.1016/j.neuroimage.2023.120164.
[44]	MAZIERO D, STENGER V A, and CARMICHAEL D W. Unified retrospective EEG motion educated artefact suppression for EEG-fMRI to suppress magnetic field gradient artefacts during motion[J]. Brain Topography, 2021, 34(6): 745–761. doi: 10.1007/s10548-021-00870-0.
[45]	JORGE J, GROUILLER F, IPEK Ö, et al. Simultaneous EEG-fMRI at ultra-high field: Artifact prevention and safety assessment[J]. NeuroImage, 2015, 105: 132–144. doi: 10.1016/j.neuroimage.2014.10.055.
[46]	CICERO N G, FULTZ N E, JEONG H, et al. High-quality multimodal MRI with simultaneous EEG using conductive ink and polymer-thick film nets[J]. Journal of Neural Engineering, 2024, 21(6): 066004. doi: 10.1088/1741-2552/ad8837.
[47]	WARBRICK T. Simultaneous EEG-fMRI: What have we learned and what does the future hold?[J]. Sensors, 2022, 22(6): 2262. doi: 10.3390/s22062262.
[48]	ROSA M J, KILNER J, BLANKENBURG F, et al. Estimating the transfer function from neuronal activity to BOLD using simultaneous EEG-fMRI[J]. NeuroImage, 2010, 49(2): 1496–1509. doi: 10.1016/j.neuroimage.2009.09.011.
[49]	PHADIKAR S, PUSULURI K, IRAJI A, et al. Integrating fMRI spatial network dynamics and EEG spectral power: Insights into resting state connectivity[J]. Frontiers in Neuroscience, 2025, 19: 1484954. doi: 10.3389/fnins.2025.1484954.
[50]	HAXBY J V, GUNTUPALLI J S, NASTASE S A, et al. Hyperalignment: Modeling shared information encoded in idiosyncratic cortical topographies[J]. eLife, 2020, 9: e56601. doi: 10.7554/eLife.56601.
[51]	PAN Jiadong, XIA Jie, ZHANG Fan, et al. 7T magnetic compatible multimodality electrophysiological signal recording system[J]. Electronics, 2023, 12(17): 3648. doi: 10.3390/electronics12173648.
[52]	CAETANO G, ESTEVES I, VOURVOPOULOS A, et al. NeuXus open-source tool for real-time artifact reduction in simultaneous EEG-fMRI[J]. NeuroImage, 2023, 280: 120353. doi: 10.1016/j.neuroimage.2023.120353.
[53]	LEVITT J, YANG Z N, WILLIAMS S D, et al. EEG-LLAMAS: A low-latency neurofeedback platform for artifact reduction in EEG-fMRI[J]. NeuroImage, 2023, 273: 120092. doi: 10.1016/j.neuroimage.2023.120092.
[54]	PERNET C R, APPELHOFF S, GORGOLEWSKI K J, et al. EEG-BIDS, an extension to the brain imaging data structure for electroencephalography[J]. Scientific Data, 2019, 6(1): 103. doi: 10.1038/s41597-019-0104-8.
[55]	LIU Junwei, CEN Xiaoping, YI Chenxin, et al. Challenges in AI-driven biomedical multimodal data fusion and analysis[J]. Genomics, Proteomics & Bioinformatics, 2025, 23(1): qzaf011. doi: 10.1093/gpbjnl/qzaf011.
[56]	LIFANOV-CARR J, GRIFFITHS B J, LINDE-DOMINGO J, et al. Reconstructing spatiotemporal trajectories of visual object memories in the human brain[J]. eNeuro, 2024, 11(9): ENEURO. 0091-24. doi: 10.1523/eneuro.0091-24.2024. (查阅网上资料,请核对页码信息).
[57]	GUENTHER S, KOSMYNA N, and MAES P. Image classification and reconstruction from low-density EEG[J]. Scientific Reports, 2024, 14(1): 16436. doi: 10.1038/s41598-024-66228-1.
[58]	PAN Hongguang, LI Zhuoyi, FU Yunpeng, et al. Reconstructing visual stimulus representation from EEG signals based on deep visual representation model[J]. IEEE Transactions on Human-Machine Systems, 2024, 54(6): 711–722. doi: 10.1109/thms.2024.3407875.
[59]	YIN Xu, WU Zhengping, and WANG Haixian. A novel DRL-guided sparse voxel decoding model for reconstructing perceived images from brain activity[J]. Journal of Neuroscience Methods, 2024, 412: 110292. doi: 10.1016/j.jneumeth.2024.110292.
[60]	ZHANG Yuanyuan, LI Xinyu, and LIU Baolin. An fMRI-based visual decoding framework combined with two-stage learning and asynchronous iterative update strategy[J]. Pattern Analysis and Applications, 2024, 27(4): 131. doi: 10.1007/s10044-024-01347-z.