A Review of Joint EEG-fMRI Methods for Visual Evoked Response Studies

WEI Zhiwei; XIAO Xiaolin; XU Minpeng; MING Dong

doi:10.11999/JEIT250781

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2026 >

WEI Zhiwei, XIAO Xiaolin, XU Minpeng, MING Dong. A Review of Joint EEG-fMRI Methods for Visual Evoked Response Studies[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250781

Citation:

WEI Zhiwei, XIAO Xiaolin, XU Minpeng, MING Dong. A Review of Joint EEG-fMRI Methods for Visual Evoked Response Studies[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250781

WEI Zhiwei, XIAO Xiaolin, XU Minpeng, MING Dong. A Review of Joint EEG-fMRI Methods for Visual Evoked Response Studies[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250781

Citation:

WEI Zhiwei, XIAO Xiaolin, XU Minpeng, MING Dong. A Review of Joint EEG-fMRI Methods for Visual Evoked Response Studies[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250781

PDF( 2932 KB)

A Review of Joint EEG-fMRI Methods for Visual Evoked Response Studies

doi: 10.11999/JEIT250781 cstr: 32379.14.JEIT250781

WEI Zhiwei¹,
XIAO Xiaolin^{1, 2
,
,},
XU Minpeng^{1, 2},
MING Dong^{1, 2}

1.
School of Precision Instrument and Opto-electronics Engineering, Tianjin University, Tianjin 300072, China
2.
Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072, China

Funds: The National Natural Science Foundation of China (W2511072, 62106170)

Received Date: 2025-08-21
Accepted Date: 2025-12-02
Rev Recd Date: 2025-12-02

Available Online: 2026-01-10

Abstract

Abstract

Significance The study of Visual Evoked Responses (VERs) using non-invasive neuroimaging is central to understanding human visual information processing. Electroencephalography (EEG) provides millisecond temporal resolution but has limited spatial precision. Functional Magnetic Resonance Imaging (fMRI) offers millimeter spatial resolution based on the blood-oxygen-level-dependent signal, although its temporal resolution is constrained by delayed hemodynamic responses. This trade-off limits the ability of any single modality to characterize complex visual processes such as attentional modulation, motion perception, and multisensory integration. Joint EEG-fMRI acquisition has therefore become an effective multimodal approach. By recording both modalities synchronously, this technique combines their complementary strengths and yields a unified spatiotemporal representation of visual neural dynamics. Despite increasing use, the literature lacks a focused review that summarizes core methods, representative applications, and continuing challenges in joint EEG-fMRI research on VERs. This review addresses this need by providing a structured overview for researchers working on visual system investigation. Progress The review first introduces the foundational technologies that support joint EEG-fMRI studies, beginning with synchronous data acquisition using MR-compatible EEG systems and dedicated synchronization hardware. The core data fusion methods are grouped into asymmetric and symmetric approaches. Asymmetric strategies use one modality to constrain analyses of the other. EEG-informed fMRI analysis models fMRI activity using single-trial EEG features, whereas fMRI-informed EEG source imaging uses fMRI activation maps as spatial priors to improve source localization. Symmetric fusion treats both modalities equally. Data-driven methods such as joint independent component analysis identify shared neural sources without imposing strong biophysical assumptions. These methods have contributed to advances in several areas. In visual mechanism studies, joint EEG-fMRI has clarified feedforward and feedback interactions in visual cortical networks. In clinical diagnosis and evaluation, it offers objective physiological markers for disorders such as amblyopia and epilepsy by revealing altered activation patterns and network dysfunction. In Brain-Computer Interface (BCI) research, multimodal feature fusion improves the accuracy and robustness of decoding visual intentions. Conclusions This review examines joint EEG-fMRI methods for VER studies, classifying major acquisition and fusion strategies and summarizing representative applications. The choice of fusion framework depends on the research objective, data quality, and underlying assumptions. Although joint EEG-fMRI benefits basic neuroscience, clinical diagnosis, and BCI development, several issues limit broader use. System-level obstacles include hardware-induced artifacts, particularly severe electromagnetic interference in ultra-high-field MRI, which degrades EEG data quality. Algorithmic challenges arise from the mismatch in spatiotemporal scales between rapid EEG signals and delayed hemodynamic responses. Inter-subject variability further reduces the generalizability of analytical and decoding models. Continued innovation in hardware engineering and computational methods is required to address these limitations. Prospects Future work in joint EEG-fMRI for VER studies is expected to progress gradually and will be shaped by advances in artificial intelligence. System-level developments include next-generation hardware combining ultra-high-field MRI systems with artifact-resilient EEG sensors and real-time correction algorithms. The creation of open, multi-center EEG-fMRI databases (following standards like BIDS) based on standardized formats and analysis pipelines will improve reproducibility and comparability. Algorithmic progress is likely to focus on artificial intelligence and deep learning. End-to-end neural architectures with spatiotemporal attention mechanisms may learn nonlinear transformations between EEG and fMRI directly, addressing limitations of conventional linear models. Transfer learning and personalized modeling may mitigate inter-subject variability and support adaptive decoding and clinical applications. As clinical and BCI uses expand, balancing model complexity with interpretability and computational efficiency will remain essential. These developments are expected to advance understanding of visual neural computation, improve diagnostic and therapeutic strategies, and support more effective BCI systems.
- EEG-fMRI integration,
- Visual Evoked Response (VER),
- Electroencephalography (EEG),
- Functional Magnetic Resonance Imaging (fMRI),
- Data fusion and analysis

FullText(HTML)

References(60)

References

[1]	RENTON A I, KLEIN D J, LIVEZEY J A, et al. Video-evoked neuromarkers of visual function in age-related macular degeneration[J]. Frontiers in Human Neuroscience, 2025, 19: 1569282. doi: 10.3389/fnhum.2025.1569282.
[2]	KARAMI A, CASTALDI E, EGER E, et al. Distinct neural representational geometries of numerosity in early visual and association regions across visual streams[J]. Communications Biology, 2025, 8(1): 1029. doi: 10.1038/s42003-025-08395-z.
[3]	BOYLAN M R, PANITZ C, TEBBE A L, et al. Feature-based attentional amplitude modulations of the steady-state visual evoked potentials reflect blood oxygen level dependent changes in feature-sensitive visual areas[J]. Journal of Cognitive Neuroscience, 2023, 35(9): 1493–1507. doi: 10.1162/jocn_a_02030.
[4]	SCHMITT O. Relationships and representations of brain structures, connectivity, dynamics and functions[J]. Progress in Neuro-Psychopharmacology and Biological Psychiatry, 2025, 138: 111332. doi: 10.1016/j.pnpbp.2025.111332.
[5]	COOK A J, IM H Y, and GIASCHI D E. Large-scale functional networks underlying visual attention[J]. Neuroscience & Biobehavioral Reviews, 2025, 173: 106165. doi: 10.1016/j.neubiorev.2025.106165.
[6]	TU Y H, TAN A, BAI Y R, et al. Decoding subjective intensity of nociceptive pain from pre-stimulus and post-stimulus brain activities[J]. Frontiers in Computational Neuroscience, 2016, 10: 32. doi: 10.3389/fncom.2016.00032.
[7]	EGAN M K, LARSEN R, WIRSICH J, et al. Safety and data quality of EEG recorded simultaneously with multi-band fMRI[J]. PLoS One, 2021, 16(7): e0238485. doi: 10.1371/journal.pone.0238485.
[8]	高鹏, 李海芳. 多模态神经影像技术研究进展与实践[J]. 太原理工大学学报, 2022, 53(3): 420–431. doi: 10.16355/j.cnki.issn1007-9432tyut.2022.03.007. GAO Peng and LI Haifang. Research progress and practical application of multimodal neuroimaging technology[J]. Journal of Taiyuan University of Technology, 2022, 53(3): 420–431. doi: 10.16355/j.cnki.issn1007-9432tyut.2022.03.007.
[9]	CHANG C and CHEN J E. Multimodal EEG-fMRI: Advancing insight into large-scale human brain dynamics[J]. Current Opinion in Biomedical Engineering, 2021, 18: 100279. doi: 10.1016/j.cobme.2021.100279.
[10]	ODUSAMI M, MASKELIŪNAS R, DAMAŠEVIČIUS R, et al. Machine learning with multimodal neuroimaging data to classify stages of Alzheimer's disease: A systematic review and meta-analysis[J]. Cognitive Neurodynamics, 2024, 18(3): 775–794. doi: 10.1007/s11571-023-09993-5.
[11]	LEI Xu, VALDES-SOSA P A, and YAO Dezhong. EEG/fMRI fusion based on independent component analysis: Integration of data-driven and model-driven methods[J]. Journal of Integrative Neuroscience, 2012, 11(3): 313–337. doi: 10.1142/s0219635212500203.
[12]	CHATZICHRISTOS C, KOFIDIS E, VAN PAESSCHEN W, et al. Early soft and flexible fusion of electroencephalography and functional magnetic resonance imaging via double coupled matrix tensor factorization for multisubject group analysis[J]. Human Brain Mapping, 2022, 43(4): 1231–1255. doi: 10.1002/hbm.25717.
[13]	VAN EYNDHOVEN S, HUNYADI B, DE LATHAUWER L, et al. Flexible fusion of electroencephalography and functional magnetic resonance imaging: Revealing neural-hemodynamic coupling through structured matrix-tensor factorization[C]. 2017 25th European Signal Processing Conference, Kos, Greece, 2017: 26–30. doi: 10.23919/EUSIPCO.2017.8081162.
[14]	JACOB L P L, BAILES S M, WILLIAMS S D, et al. Brainwide hemodynamics predict EEG neural rhythms across sleep and wakefulness in humans[J]. PLoS Computational Biology, 2025, 21(9): e1013497. doi: 10.1371/journal.pcbi.1013497.
[15]	HAUK O and CLARKE A. Patterns of language[J]. Language, Cognition and Neuroscience, 2024, 39(8): 959–961. doi: 10.1080/23273798.2024.2388329.
[16]	JONMOHAMADI Y, FORSYTH A, MCMILLAN R, et al. Constrained temporal parallel decomposition for EEG-fMRI fusion[J]. Journal of Neural Engineering, 2019, 16(1): 016017. doi: 10.1088/1741-2552/aaefda.
[17]	BéNAR C G, SCHöN D, GRIMAULT S, et al. Single‐trial analysis of oddball event‐related potentials in simultaneous EEG‐fMRI[J]. Human Brain Mapping, 2007, 28(7): 602–613. doi: 10.1002/hbm.20289.
[18]	HUSTER R J, DEBENER S, EICHELE T, et al. Methods for simultaneous EEG-fMRI: An introductory review[J]. Journal of Neuroscience, 2012, 32(18): 6053–6060. doi: 10.1523/jneurosci.0447-12.2012.
[19]	PROKOPIOU P C, XIFRA-PORXAS A, KASSINOPOULOS M, et al. Modeling the hemodynamic response function using EEG-fMRI data during eyes-open resting-state conditions and motor task execution[J]. Brain Topography, 2022, 35(3): 302–321. doi: 10.1007/s10548-022-00898-w.
[20]	GUO Qian, ZHOU Tiantong, LI Wenjie, et al. Single-trial EEG-informed fMRI analysis of emotional decision problems in hot executive function[J]. Brain and Behavior, 2017, 7(7): e00728. doi: 10.1002/brb3.728.
[21]	MURTA T, LEITE M, CARMICHAEL D W, et al. Electrophysiological correlates of the BOLD signal for EEG-informed fMRI[J]. Human Brain Mapping, 2015, 36(1): 391–414. doi: 10.1002/hbm.22623.
[22]	MORADI N, GOODYEAR B G, and SOTERO R C. Deep EEG source localization via EMD-based fMRI high spatial frequency[J]. PLoS One, 2024, 19(3): e0299284. doi: 10.1371/journal.pone.0299284.
[23]	LIU Ke, YU Zhuliang, WU Wei, et al. fMRI-SI-STBF: An fMRI-informed Bayesian electromagnetic spatio-temporal extended source imaging[J]. Neurocomputing, 2021, 462: 14–30. doi: 10.1016/j.neucom.2021.06.066.
[24]	WANG Hailing, LEI Xu, ZHAN Zhichao, et al. A new fMRI informed mixed-norm constrained algorithm for EEG source localization[J]. IEEE Access, 2018, 6: 8258–8269. doi: 10.1109/access.2018.2792442.
[25]	BRINKMANN B H. Technical considerations in EEG source imaging[J]. Journal of Clinical Neurophysiology, 2024, 41(1): 2–7. doi: 10.1097/wnp.0000000000001029.
[26]	MAGRI C, SCHRIDDE U, MURAYAMA Y, et al. The amplitude and timing of the BOLD signal reflects the relationship between local field potential power at different frequencies[J]. Journal of Neuroscience, 2012, 32(4): 1395–1407. doi: 10.1523/jneurosci.3985-11.2012.
[27]	SUN Peng, HICKS Y, and SETCHI R. Joint EEG-fMRI model for EEG source separation[C]. 2014 IEEE International Conference on Systems, Man, and Cybernetics, San Diego, USA, 2014: 2234–2239. doi: 10.1109/SMC.2014.6974257.
[28]	HEUGEL N, BEARDSLEY S A, and LIEBENTHAL E. EEG and fMRI coupling and decoupling based on joint Independent Component Analysis (jICA)[J]. Journal of Neuroscience Methods, 2022, 369: 109477. doi: 10.1016/j.jneumeth.2022.109477.
[29]	LI Junhua, CHEN Yu, TAYA F, et al. A unified canonical correlation analysis-based framework for removing gradient artifact in concurrent EEG/fMRI recording and motion artifact in walking recording from EEG signal[J]. Medical & Biological Engineering & Computing, 2017, 55(9): 1669–1681. doi: 10.1007/s11517-017-1620-3.
[30]	JONMOHAMADI Y, MUTHUKUMARASWAMY S, CHEN J, et al. Extraction of common task features in EEG-fMRI data using coupled tensor-tensor decomposition[J]. Brain Topography, 2020, 33(5): 636–650. doi: 10.1007/s10548-020-00787-0.
[31]	ZAFAR R, KAMEL N, NAUFAL M, et al. A study of decoding human brain activities from simultaneous data of EEG and fMRI using MVPA[J]. Australasian Physical & Engineering Sciences in Medicine, 2018, 41(3): 633–645. doi: 10.1007/s13246-018-0656-5.
[32]	SCRIVENER C L, TEED J A, and SILSON E H. Visual imagery of familiar people and places in category selective cortex[J]. Neuroscience of Consciousness, 2025, 2025(1): niaf006. doi: 10.1093/nc/niaf006.
[33]	PITZALIS S, STRAPPINI F, BULTRINI A, et al. Detailed spatiotemporal brain mapping of chromatic vision combining high‐resolution VEP with fMRI and retinotopy[J]. Human Brain Mapping, 2018, 39(7): 2868–2886. doi: 10.1002/hbm.24046.
[34]	ITTHIPURIPAT S, SPRAGUE T C, and SERENCES J T. Functional MRI and EEG index complementary attentional modulations[J]. Journal of Neuroscience, 2019, 39(31): 6162–6179. doi: 10.1523/JNEUROSCI.2519-18.2019.
[35]	HEDGE C, STOTHART G, TODD JONES J, et al. A frontal attention mechanism in the visual mismatch negativity[J]. Behavioural Brain Research, 2015, 293: 173–181. doi: 10.1016/j.bbr.2015.07.022.
[36]	FRANCO R M, D'ALMEIDA O C, RAIMUNDO M, et al. The clinical use of retinotopy in functional hemianopia[J]. Neuro-Ophthalmology, 2025. doi: 10.1080/01658107.2025.2474542.
[37]	HUANG Yujing, CROMARTY R, JIA Lina, et al. Attention network dysfunctions in lewy body dementia and alzheimer's disease[J]. Journal of Clinical Medicine, 2024, 13(22): 6691. doi: 10.3390/jcm13226691.
[38]	WANG Xinmei, CUI Dongmei, ZHENG Ling, et al. Combination of blood oxygen level-dependent functional magnetic resonance imaging and visual evoked potential recordings for abnormal visual cortex in two types of amblyopia[J]. Molecular Vision, 2012, 18: 909–919.
[39]	BARTOLINI E, PESARESI I, FABBRI S, et al. Abnormal response to photic stimulation in Juvenile Myoclonic Epilepsy: An EEG-fMRI study[J]. Epilepsia, 2014, 55(7): 1038–1047. doi: 10.1111/epi.12634.
[40]	ANWAR M N, BONZANO L, ROSSI-SEBASTIANO D, et al. Simultaneous recording of VEP and fMRI to study optic neuritis in multiple sclerosis[C]. The Fifth IASTED International Conference: Biomedical Engineering, Innsbruck, Austria, 2007: 197–202.
[41]	SINGH S P, MISHRA S, GUPTA S, et al. Functional mapping of the brain for brain-computer interfacing: A review[J]. Electronics, 2023, 12(3): 604. doi: 10.3390/electronics12030604.
[42]	AHMAD R F, MALIK A S, KAMEL N, et al. Visual brain activity patterns classification with simultaneous EEG-fMRI: A multimodal approach[J]. Technology Health Care, 2017, 25(3): 471–485. doi: 10.3233/thc-161286.
[43]	WATANABE N, MIYOSHI K, JIMURA K, et al. Multimodal deep neural decoding reveals highly resolved spatiotemporal profile of visual object representation in humans[J]. NeuroImage, 2023, 275: 120164. doi: 10.1016/j.neuroimage.2023.120164.
[44]	MAZIERO D, STENGER V A, and CARMICHAEL D W. Unified retrospective EEG motion educated artefact suppression for EEG-fMRI to suppress magnetic field gradient artefacts during motion[J]. Brain Topography, 2021, 34(6): 745–761. doi: 10.1007/s10548-021-00870-0.
[45]	JORGE J, GROUILLER F, IPEK Ö, et al. Simultaneous EEG-fMRI at ultra-high field: Artifact prevention and safety assessment[J]. NeuroImage, 2015, 105: 132–144. doi: 10.1016/j.neuroimage.2014.10.055.
[46]	CICERO N G, FULTZ N E, JEONG H, et al. High-quality multimodal MRI with simultaneous EEG using conductive ink and polymer-thick film nets[J]. Journal of Neural Engineering, 2024, 21(6): 066004. doi: 10.1088/1741-2552/ad8837.
[47]	WARBRICK T. Simultaneous EEG-fMRI: What have we learned and what does the future hold?[J]. Sensors, 2022, 22(6): 2262. doi: 10.3390/s22062262.
[48]	ROSA M J, KILNER J, BLANKENBURG F, et al. Estimating the transfer function from neuronal activity to BOLD using simultaneous EEG-fMRI[J]. NeuroImage, 2010, 49(2): 1496–1509. doi: 10.1016/j.neuroimage.2009.09.011.
[49]	PHADIKAR S, PUSULURI K, IRAJI A, et al. Integrating fMRI spatial network dynamics and EEG spectral power: Insights into resting state connectivity[J]. Frontiers in Neuroscience, 2025, 19: 1484954. doi: 10.3389/fnins.2025.1484954.
[50]	HAXBY J V, GUNTUPALLI J S, NASTASE S A, et al. Hyperalignment: Modeling shared information encoded in idiosyncratic cortical topographies[J]. eLife, 2020, 9: e56601. doi: 10.7554/eLife.56601.
[51]	PAN Jiadong, XIA Jie, ZHANG Fan, et al. 7T magnetic compatible multimodality electrophysiological signal recording system[J]. Electronics, 2023, 12(17): 3648. doi: 10.3390/electronics12173648.
[52]	CAETANO G, ESTEVES I, VOURVOPOULOS A, et al. NeuXus open-source tool for real-time artifact reduction in simultaneous EEG-fMRI[J]. NeuroImage, 2023, 280: 120353. doi: 10.1016/j.neuroimage.2023.120353.
[53]	LEVITT J, YANG Z N, WILLIAMS S D, et al. EEG-LLAMAS: A low-latency neurofeedback platform for artifact reduction in EEG-fMRI[J]. NeuroImage, 2023, 273: 120092. doi: 10.1016/j.neuroimage.2023.120092.
[54]	PERNET C R, APPELHOFF S, GORGOLEWSKI K J, et al. EEG-BIDS, an extension to the brain imaging data structure for electroencephalography[J]. Scientific Data, 2019, 6(1): 103. doi: 10.1038/s41597-019-0104-8.
[55]	LIU Junwei, CEN Xiaoping, YI Chenxin, et al. Challenges in AI-driven biomedical multimodal data fusion and analysis[J]. Genomics, Proteomics & Bioinformatics, 2025, 23(1): qzaf011. doi: 10.1093/gpbjnl/qzaf011.
[56]	LIFANOV-CARR J, GRIFFITHS B J, LINDE-DOMINGO J, et al. Reconstructing spatiotemporal trajectories of visual object memories in the human brain[J]. eNeuro, 2024, 11(9): ENEURO. 0091-24. doi: 10.1523/eneuro.0091-24.2024.
[57]	GUENTHER S, KOSMYNA N, and MAES P. Image classification and reconstruction from low-density EEG[J]. Scientific Reports, 2024, 14(1): 16436. doi: 10.1038/s41598-024-66228-1.
[58]	PAN Hongguang, LI Zhuoyi, FU Yunpeng, et al. Reconstructing visual stimulus representation from EEG signals based on deep visual representation model[J]. IEEE Transactions on Human-Machine Systems, 2024, 54(6): 711–722. doi: 10.1109/thms.2024.3407875.
[59]	YIN Xu, WU Zhengping, and WANG Haixian. A novel DRL-guided sparse voxel decoding model for reconstructing perceived images from brain activity[J]. Journal of Neuroscience Methods, 2024, 412: 110292. doi: 10.1016/j.jneumeth.2024.110292.
[60]	ZHANG Yuanyuan, LI Xinyu, and LIU Baolin. An fMRI-based visual decoding framework combined with two-stage learning and asynchronous iterative update strategy[J]. Pattern Analysis and Applications, 2024, 27(4): 131. doi: 10.1007/s10044-024-01347-z.