Research Progress of Electromagnetic Neural Network Based on Metamaterials

MA Qian; FENG Zirui; GAO Xinxin; GU Ze; YOU Jianwei; CUI Tiejun

doi:10.11999/JEIT231285

Volume 46 Issue 5

May 2024

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2024 > 46(5): 1529-1545

MA Qian, FENG Zirui, GAO Xinxin, GU Ze, YOU Jianwei, CUI Tiejun. Research Progress of Electromagnetic Neural Network Based on Metamaterials[J]. Journal of Electronics & Information Technology, 2024, 46(5): 1529-1545. doi: 10.11999/JEIT231285

Citation:

MA Qian, FENG Zirui, GAO Xinxin, GU Ze, YOU Jianwei, CUI Tiejun. Research Progress of Electromagnetic Neural Network Based on Metamaterials[J]. Journal of Electronics & Information Technology, 2024, 46(5): 1529-1545. doi: 10.11999/JEIT231285

Citation:

PDF( 22309 KB)

Research Progress of Electromagnetic Neural Network Based on Metamaterials

doi: 10.11999/JEIT231285

MA Qian^{1, 2},
FENG Zirui^{1, 2},
GAO Xinxin³,
GU Ze^{1, 2},
YOU Jianwei^{1, 2},
CUI Tiejun^{1, 2
,
,}

1.
State Key Laboratory of Millimeter Waves, Southeast University, Nanjing 210096, China
2.
Institute of Electromagnetic Space, Southeast University, Nanjing 210096, China
3.
State Key Laboratory of Terahertz and Millimeter Waves, City University of Hong Kong, Hong Kong 999077, China

Funds: The National Natural Science Foundation of China (62301147, 62288101, 92167202), The Natural Science Foundation of Jiangsu Province (BK20230822), The Major Project of Natural Science Foundation of Jiangsu Province (BK20212002), The Fundamental Research Funds for the Central Universities (2242023K5002, 2242018R30001, 2242022R20017)

Received Date: 2023-11-20
Rev Recd Date: 2024-04-30

Available Online: 2024-05-12

Publish Date: 2024-05-30

Abstract

Abstract

With the widespread application of artificial intelligence technology, the demand for computing power for intelligent computing has grown exponentially. At present, the rapid development of chips has approached the bottleneck of its manufacturing process, and power consumption is also increasing. Therefore, research on high-speed, energy-efficient intelligent computing hardware is an important direction. Computing architectures represented by photonic circuit neural networks and all-optical diffraction neural networks have received widespread attention due to their advantages such as fast calculation and low power consumption. This article reviews the representative work of optical neural networks, and introduces it through the two main lines of development of three-dimensional diffractive neural networks and optical neural network chips. At the same time, it focuses on the bottlenecks and challenges faced by optical diffractive neural networks and photonic neural network chips, such as network scale and Integration degree, etc., analyzes and compares their characteristics, performance and respective advantages and disadvantages. Secondly, taking into account the development needs of generalization, this article further discusses the programmable design of neuromorphic computing hardware, and introduces some representative work on programmable neural networks to each part. In addition to intelligent neural networks in the optical band, this article also discusses the development and application of microwave diffraction neural networks and demonstrates their programmability. Finally, the future direction and development trends of intelligent neuromorphic computing are introduced, as well as its potential applications in wireless communications, information processing and sensing.
- Metamaterials,
- Optical diffractive neural network,
- Electromagnetic neural network,
- Intelligent computing.

FullText(HTML)

References(66)

References

[1]	SILVER D, HUANG A, MADDISON C J, et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016, 529(7587): 484–489. doi: 10.1038/nature16961.
[2]	VAN DIS E A M, BOLLEN J, ZUIDEMA W, et al. ChatGPT: Five priorities for research[J]. Nature, 2023, 614(7947): 224–226. doi: 10.1038/d41586-023-00288-7.
[3]	BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[C]. The 34th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2020: 159.
[4]	XU Xingyuan, REN Guanghui, FELEPPA T, et al. Self-calibrating programmable photonic integrated circuits[J]. Nature Photonics, 2022, 16(8): 595–602. doi: 10.1038/s41566-022-01020-z.
[5]	FELDMANN J, YOUNGBLOOD N, KARPOV M, et al. Parallel convolutional processing using an integrated photonic tensor core[J]. Nature, 2021, 589(7840): 52–58. doi: 10.1038/s41586-020-03070-1.
[6]	FELDMANN J, YOUNGBLOOD N, WRIGHT C D, et al. All-optical spiking neurosynaptic networks with self-learning capabilities[J]. Nature, 2019, 569(7755): 208–214. doi: 10.1038/s41586-019-1157-8.
[7]	WETZSTEIN G, OZCAN A, GIGAN S, et al. Inference in artificial intelligence with deep optics and photonics[J]. Nature, 2020, 588(7836): 39–47. doi: 10.1038/s41586-020-2973-6.
[8]	LI Yuhang, LUO Yi, MENGU D, et al. Quantitative Phase Imaging (QPI) through random diffusers using a diffractive optical network[J]. Light: Advanced Manufacturing, 2023, 4(3): 17. doi: 10.37188/lam.2023.017.
[9]	MENGU D, LUO Yi, RIVENSON Y, et al. Analysis of diffractive optical neural networks and their integration with electronic neural networks[J]. IEEE Journal of Selected Topics in Quantum Electronics, 2020, 26(1): 3700114. doi: 10.1109/JSTQE.2019.2921376.
[10]	WANG Tianyu, SOHONI M M, WRIGHT L G, et al. Image sensing with multilayer nonlinear optical neural networks[J]. Nature Photonics, 2023, 17(5): 408–415. doi: 10.1038/s41566-023-01170-8.
[11]	HUGHES T W, MINKOV M, SHI Yu, et al. Training of photonic neural networks through in situ backpropagation and gradient measurement[J]. Optica, 2018, 5(7): 864–871. doi: 10.1364/optica.5.000864.
[12]	CHEN Yitong, ZHOU Tiankuang, WU Jiamin, et al. Photonic unsupervised learning variational autoencoder for high-throughput and low-latency image transmission[J]. Science Advances, 2023, 9(7): eadf8437. doi: 10.1126/SCIADV.ADF8437.
[13]	LUO Yi, MENGU D, YARDIMCI N T, et al. Design of task-specific optical systems using broadband diffractive neural networks[J]. Light: Science & Applications, 2019, 8: 112. doi: 10.1038/s41377-019-0223-1.
[14]	WILLIAMSON I A D, HUGHES T W, MINKOV M, et al. Reprogrammable electro-optic nonlinear activation functions for optical neural networks[J]. IEEE Journal of Selected Topics in Quantum Electronics, 2020, 26(1): 7700412. doi: 10.1109/jstqe.2019.2930455.
[15]	SHEN Yichen, HARRIS N C, SKIRLO S, et al. Deep learning with coherent nanophotonic circuits[J]. Nature Photonics, 2017, 11(7): 441–446. doi: 10.1038/nphoton.2017.93.
[16]	SHOKRANEH F, GEOFFROY-GAGNON S, NEZAMI M S, et al. A single layer neural network implemented by a 4×4 MZI-Based Optical Processor[J]. IEEE Photonics Journal, 2019, 11(6): 4501612. doi: 10.1109/jphot.2019.2952562.
[17]	POUR FARD M M, WILLIAMSON I A D, EDWARDS M, et al. Experimental realization of arbitrary activation functions for optical neural networks[J]. Optics Express, 2020, 28(8): 12138–12148. doi: 10.1364/OE.391473.
[18]	LIAO Kun, CHEN Ye, YU Zhongcheng, et al. All-optical computing based on convolutional neural networks[J]. Opto-Electronic Advances, 2021, 4(11): 200060. doi: 10.29026/oea.2021.200060.
[19]	HAMERLY R, BERNSTEIN L, SLUDDS A, et al. Large- Scale Optical Neural Network Based on Photoelectric Multiplication[J]. Physical Review X, 2019, 9(2): 021032 doi: 10.1103/PhysRevX.9.021032.
[20]	FU Tingzhao, ZANG Yubin, HUANG Honghao, et al. On-chip photonic diffractive optical neural network based on a spatial domain electromagnetic propagation model[J]. Optics Express, 2021, 29(20): 31924–31940. doi: 10.1364/OE.435183.
[21]	ZAREI S, MARZBAN M R, and KHAVASI A. Integrated photonic neural network based on silicon metalines[J]. Optics Express, 2020, 28(24): 36668–36684. doi: 10.1364/OE.404386.
[22]	FU Tingzhao, ZANG Yubin, HUANG Yuyao, et al. Photonic machine learning with on-chip diffractive optics[J]. Nature Communications, 2023, 14(1): 70. doi: 10.1038/s41467-022-35772-7.
[23]	WANG Zi, CHANG L, WANG Feifan, et al. Integrated photonic metasystem for image classifications at telecommunication wavelength[J]. Nature Communications, 2022, 13(1): 2131. doi: 10.1038/s41467-022-29856-7.
[24]	KHORAM E, CHEN Ang, LIU Dianjing, et al. Nanophotonic media for artificial neural inference[J]. Photonics Research, 2019, 7(8): 823–827. doi: 10.1364/prj.7.000823.
[25]	LIN Xing, RIVENSON Y, YARDIMCI N T, et al. All-optical machine learning using diffractive deep neural networks[J]. Science, 2018, 361(6406): 1004–1008. doi: 10.1126/science.aat8084.
[26]	LI Jingxi, GAN Tianyi, ZHAO Yifan, et al. Unidirectional imaging using deep learning–designed materials[J]. Science Advances, 2023, 9(17): eadg1505. doi: 10.1126/sciadv.adg1505.
[27]	YAN Tao, WU Jiamin, ZHOU Tiankuang, et al. Fourier-space diffractive deep neural network[J]. Physical Review Letters, 2019, 123(2): 023901. doi: 10.1103/PhysRevLett.123.023901.
[28]	RAHMAN M S S, LI Jingxi, MENGU D, et al. Ensemble learning of diffractive optical networks[J]. Light: Science & Applications, 2021, 10(1): 14. doi: 10.1038/s41377-020-00446-w.
[29]	ZHANG Luhe, LI Caiyun, HE Jiangyong, et al. Optical machine learning using time-lens deep neural NetWorks[J]. Photonics, 2021, 8(3): 78. doi: 10.3390/photonics8030078.
[30]	CHANG Julie, SITZMANN V, DUN Xiong, et al. Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification[J]. Scientific Reports, 2018, 8(1): 12324. doi: 10.1038/s41598-018-30619-y.
[31]	QU Geyang, CAI Guiyi, SHA Xinbo, et al. All-dielectric metasurface empowered optical-electronic hybrid neural networks[J]. Laser & Photonics Reviews, 2022, 16(10): 2100732. doi: 10.1002/lpor.202100732.
[32]	XIAO Yongliang, LI Sikun, SITU Guohai, et al. Unitary learning for diffractive deep neural network[J]. Optics and Lasers in Engineering, 2021, 139: 106499. doi: 10.1016/j.optlaseng.2020.106499.
[33]	LI Jingxi, MENGU D, YARDIMCI N T, et al. Spectrally encoded single-pixel machine vision using diffractive networks[J]. Science Advances, 2021, 7(13): eabd7690. doi: 10.1126/sciadv.abd7690.
[34]	LI Jingxi, MENGU D, LUO Yi, et al. Class-specific differential detection in diffractive optical neural networks improves inference accuracy[J]. Advanced Photonics, 2019, 1(4): 046001. doi: 10.1117/1.Ap.1.4.046001.
[35]	ZHOU Tiankuang, LIN Xin, WU Jiamin, et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit[J]. Nature Photonics, 2021, 15(5): 367–373. doi: 10.1038/s41566-021-00796-w.
[36]	CUI Tiejun, QI Meiqing, WAN Xiang, et al. Coding metamaterials, digital metamaterials and programmable metamaterials[J]. Light: Science & Applications, 2014, 3(10): e218. doi: 10.1038/lsa.2014.99.
[37]	CUI Tiejun, LIU Shuo, BAI Guodong, et al. Direct transmission of digital message via programmable coding metasurface[J]. Research, 2019, 2019: 2584509. doi: 10.34133/2019/2584509.
[38]	DAI Junyan, TANG Wankai, CHEN Mingzheng, et al. Wireless communication based on information metasurfaces[J]. IEEE Transactions on Microwave Theory and Techniques, 2021, 69(3): 1493–1510. doi: 10.1109/tmtt.2021.3054662.
[39]	ZHANG Lei, CHEN Mingzheng, TANG Wankai, et al. A wireless communication scheme based on space- and frequency-division multiplexing using digital metasurfaces[J]. Nature Electronics, 2021, 4(3): 218–227. doi: 10.1038/s41928-021-00554-4.
[40]	CHEN Lei, MA Qian, LUO Sisi, et al. Touch-programmable metasurface for various electromagnetic manipulations and encryptions[J]. Small, 2022, 18(45): 2203871. doi: 10.1002/smll.202203871.
[41]	LI Lianlin, SHUANG Ya, MA Qian, et al. Intelligent metasurface imager and recognizer[J]. Light: Science & Applications, 2019, 8: 97. doi: 10.1038/s41377-019-0209-z.
[42]	MA Qian, BAI Guodong, JING Hongbo, et al. Smart metasurface with self-adaptively reprogrammable functions[J]. Light: Science & Applications, 2019, 8: 98. doi: 10.1038/s41377-019-0205-3.
[43]	MA Qian, GAO Wei, XIAO Qiang, et al. Directly wireless communication of human minds via non-invasive brain-computer-metasurface platform[J]. eLight, 2022, 2: 11. doi: 10.1186/s43593-022-00019-x.
[44]	GAO Xinxin, MA Qian, GU Ze, et al. Programmable surface plasmonic neural networks for microwave detection and processing[J]. Nature Electronics, 2023, 6(4): 319–328. doi: 10.1038/s41928-023-00951-x.
[45]	LIU Che, MA Qian, LUO Zhangjie, et al. A programmable diffractive deep neural network based on a digital-coding metasurface array[J]. Nature Electronics, 2022, 5(2): 113–122. doi: 10.1038/s41928-022-00719-9.
[46]	QIAN Chao, WANG Zhedong, QIAN Haoliang, et al. Dynamic recognition and mirage using neuro-metamaterials[J]. Nature Communications, 2022, 13(1): 2694. doi: 10.1038/s41467-022-30377-6.
[47]	GAO Xinxin, MA Qian, GU Ze, et al. Reconfigurable spoof localized surface plasmonic for frequency detections[J]. Laser & Photonics Reviews, 2023, 17(11): 2300267. doi: 10.1002/lpor.202300267.
[48]	GAO Xinxin, CUI Wenyi, GU Ze, et al. Multimode and reconfigurable phase shifter of spoof surface plasmons[J]. IEEE Transactions on Antennas and Propagation, 2023, 71(6): 5361–5369. doi: 10.1109/tap.2023.3262348.
[49]	GAO Xinxin, CHEN Baojie, SHUM K M, et al. Multifunctional terahertz spoof plasmonic devices[J]. Advanced Materials Technologies, 2023, 8(12): 2202050. doi: 10.1002/admt.202202050.
[50]	GAO Xinxin, GU Ze, MA Qian, et al. Reprogrammable spoof plasmonic modulator[J]. Advanced Functional Materials, 2023, 33(18): 2212328. doi: 10.1002/adfm.202212328.
[51]	GAO Xinxin, ZHANG Haochi, WU Liangwei, et al. Programmable multifunctional device based on spoof surface Plasmon polaritons[J]. IEEE Transactions on Antennas and Propagation, 2020, 68(5): 3770–3779. doi: 10.1109/tap.2020.2969745.
[52]	GAO Xinxin, ZHANG Jingjing, MA Qian, et al. Nonmagnetic spoof plasmonic isolator based on parametric amplification[J]. Laser & Photonics Reviews, 2022, 16(4): 2100578. doi: 10.1002/lpor.202100578.
[53]	GAO Xinxin, ZHANG Jingjing, LUO Yu, et al. Reconfigurable parametric amplifications of spoof surface plasmons[J]. Advanced Science, 2021, 8(17): 2100795. doi: 10.1002/advs.202100795.
[54]	GAO Xinxin, ZHANG Jingjing, ZHANG Haochi, et al. Dynamic controls of second‐harmonic generations in both forward and backward modes using reconfigurable plasmonic metawaveguide[J]. Advanced Optical Materials, 2020, 8(8): 1902058. doi: 10.1002/adom.201902058.
[55]	GAO Xinxin, ZHANG Haochi, HE Peihang, et al. Crosstalk suppression based on mode mismatch between spoof SPP transmission line and microstrip[J]. IEEE Transactions on Components, Packaging and Manufacturing Technology, 2019, 9(11): 2267–2275. doi: 10.1109/tcpmt.2019.2931373.
[56]	ZHANG Haochi, ZHANG Lepeng, HE Peihang, et al. A plasmonic route for the integrated wireless communication of subdiffraction-limited signals[J]. Light: Science & Applications, 2020, 9: 113. doi: 10.1038/s41377-020-00355-y.
[57]	JOY S R, EREMENTCHOUK M, YU Hao, et al. Spoof Plasmon interconnects—communications beyond RC limit[J]. IEEE Transactions on Communications, 2019, 67(1): 599–610. doi: 10.1109/tcomm.2018.2874242.
[58]	GOI E, SCHOENHARDT S, and GU Ming. Direct retrieval of Zernike-based pupil functions using integrated diffractive deep neural networks[J]. Nature Communications, 2022, 13(1): 7531. doi: 10.1038/s41467-022-35349-4.
[59]	QIAN Chao, LIN Xiao, LIN Xiaobin, et al. Performing optical logic operations by a diffractive neural network[J]. Light: Science & Applications, 2020, 9: 59. doi: 10.1038/s41377-020-0303-2.
[60]	GAO Xinxin and CUI Tiejun. Using surface plasmons to create programmable neural networks[J]. Nature Electronics, 2023, 6(4): 266–267. doi: 10.1038/s41928-023-00952-w.
[61]	ZHANG H, GU M, JIANG X D, et al. An optical neural chip for implementing complex-valued neural network[J]. Nature Communications, 2021, 12(1): 457. doi: 10.1038/s41467-020-20719-7.
[62]	ZHU H H, ZOU J, ZHANG H, et al. Space-efficient optical computing with an integrated chip diffractive neural network[J]. Nature Communications, 2022, 13(1): 1044. doi: 10.1038/s41467-022-28702-0.
[63]	LUO Xuhao, HU Yueqiang, OU Xiangnian et al. Metasurface-enabled on-chip multiplexed diffractive neural networks in the visible[J]. Light: Science & Applications, 2022, 11(1): 158. doi: 10.1038/s41377-022-00844-2.
[64]	HUANG Zebin, WANG Peipei, LIU Junmin, et al. All-optical signal processing of vortex beams with diffractive deep neural networks[J]. Physical Review Applied, 2021, 15(1): 014037. doi: 10.1103/PhysRevApplied.15.014037.
[65]	WANG Zhedong, QIAN Chao, FAN Zhixiang, et al. Arbitrary polarization readout with dual-channel neuro-metasurfaces[J]. Advanced Science, 2023, 10(5): 2204699. doi: 10.1002/advs.202204699.
[66]	MA Qian, GAO Xinxin, GU Ze, et al. Intelligent neuromorphic computing based on nanophotonics and metamaterials[J]. MRS Communications. doi: 10.1557/s43579-024-00520-z.