A Processing-In-Memory Neural Network Inference System Design for Infrared Gesture Recognition

SHI Xiangyang; LIU Jinchang; LU Qiulin; WANG Ziang; HAN Yongkang; GAO Gen; SUN Haoran; JIANG Xiaoyong; SHI Tuo; LI Qing; MIAO Jinshui

doi:10.11999/JEIT260122

Volume 48 Issue 7

Jul. 2026

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2026 > 48(7): 2805-2814

SHI Xiangyang, LIU Jinchang, LU Qiulin, WANG Ziang, HAN Yongkang, GAO Gen, SUN Haoran, JIANG Xiaoyong, SHI Tuo, LI Qing, MIAO Jinshui. A Processing-In-Memory Neural Network Inference System Design for Infrared Gesture Recognition[J]. Journal of Electronics & Information Technology, 2026, 48(7): 2805-2814. doi: 10.11999/JEIT260122

Citation:

SHI Xiangyang, LIU Jinchang, LU Qiulin, WANG Ziang, HAN Yongkang, GAO Gen, SUN Haoran, JIANG Xiaoyong, SHI Tuo, LI Qing, MIAO Jinshui. A Processing-In-Memory Neural Network Inference System Design for Infrared Gesture Recognition[J]. Journal of Electronics & Information Technology, 2026, 48(7): 2805-2814. doi: 10.11999/JEIT260122

Citation:

SHI Xiangyang, LIU Jinchang, LU Qiulin, WANG Ziang, HAN Yongkang, GAO Gen, SUN Haoran, JIANG Xiaoyong, SHI Tuo, LI Qing, MIAO Jinshui. A Processing-In-Memory Neural Network Inference System Design for Infrared Gesture Recognition[J]. Journal of Electronics & Information Technology, 2026, 48(7): 2805-2814. doi: 10.11999/JEIT260122

PDF( 3978 KB)

A Processing-In-Memory Neural Network Inference System Design for Infrared Gesture Recognition

doi: 10.11999/JEIT260122 cstr: 32379.14.JEIT260122

1.
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou 310024, China
2.
Zhejiang Lab, Hangzhou 311121, China
3.
Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai 200083, China

Funds: The National Natural Science Foundation of China (62334011, 62327812), The National Key Research and Development Program of China(2022YFB4501700)

Received Date: 2026-02-02
Accepted Date: 2026-02-06
Rev Recd Date: 2026-02-05

Available Online: 2026-02-16

Publish Date: 2026-07-10

Abstract

Abstract

Objective Infrared (IR) image recognition is widely used in security monitoring, autonomous driving, and human-machine interaction, where stable perception under low illumination and noisy backgrounds is required. However, IR images often exhibit low contrast, blurred edges, and weak texture representation, which reduce recognition accuracy. Conventional Deep Neural Networks (DNNs) further aggravate this issue in edge environments because the von Neumann architecture requires intensive computation, frequent memory-processor data transfer, and high power consumption. Memristor devices provide an alternative because they support in-memory computing. However, most existing implementations still depend on CPUs or GPUs to execute nonlinear operations such as activation layers, which reintroduces data transfer overhead. To address this issue, a nearly all-memristor neural network framework for IR image recognition is proposed. In this framework, computationally dominant linear operations are executed entirely on memristor arrays, whereas CPU participation is limited to the final activation step. Methods The proposed system maps two fully connected layers of a neural network directly onto memristor crossbar arrays. This mapping enables large-scale matrix-vector multiplications to be executed in memory with high parallelism and low energy consumption. Network weights are encoded as memristor conductance states, and inference is performed by applying voltage inputs and summing output currents. Intermediate nonlinear activations and final classification are computed on the host computer. Because these operations require minimal computation, the CPU overhead in both latency and energy remains negligible. Based on this design, an IR gesture recognition system is constructed to distinguish two hand gestures and a no-gesture state. System evaluation considers recognition accuracy, inference latency, energy consumption, and the amount of data transferred between the memristor arrays and the CPU. A CPU-only neural network is used as the baseline. The framework provides a practical approach for near all-hardware neural network computing while maintaining recognition accuracy and reducing energy consumption and data transfer for edge-scale IR applications. Results and Discussions The system is evaluated on a three-class IR gesture recognition task that includes two gestures and a no-gesture state. The memristor-based network achieves approximately 95% accuracy (Table 2), which is close to the 97% obtained with a GPU implementation (Table 2). This result indicates that conductance variation has limited effect on recognition performance. The average inference latency per frame is substantially lower than that of the GPU baseline, and the achieved frame rate satisfies real-time requirements. Power measurements indicate that the memristor array consumes only milliwatts, whereas the GPU requires approximately 25 W (Table 2). Although peripheral circuit consumption is not fully included, the results demonstrate the inherent energy efficiency of in-memory computing. Intermediate and final activation outputs are transferred to the CPU, which removes most memory-processor interactions. This reduction in data movement, combined with array-level parallelism, accounts for the observed improvements in latency and energy consumption. Overall, the framework maintains high recognition accuracy while significantly improving computational efficiency, which indicates strong potential for edge-scale IR recognition. Conclusions This study presents a nearly all-memristor infrared neural network framework in which two fully connected layers are executed on memristor arrays and intermediate nonlinear activations are processed on the host computer. When applied to a three-class IR gesture recognition task, the system achieves recognition accuracy comparable to that of a GPU platform while significantly reducing inference latency and energy consumption. By minimizing memory-processor data transfer and exploiting in-memory computing, the framework provides clear advantages for edge applications. The results confirm the feasibility of deploying memristor-based neural networks in practical infrared recognition systems. Future research will focus on integrating memristor-based activation functions, scaling the system to larger circuits, and extending the approach to more complex network architectures and datasets.
- Infrared image recognition,
- Memristor neural network,
- In-memory computing,
- Edge computing,
- Gesture recognition

FullText(HTML)

References(22)

References

[1]	WANG Qingwang, JIN Pengcheng, WU Yuhang, et al. Infrared image enhancement: A review[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2025, 18: 3281–3299. doi: 10.1109/JSTARS.2024.3523418.
[2]	HU Lei, HU Long, and CHEN Minghui. Edge-enhanced infrared image super-resolution reconstruction model under transformer[J]. Scientific Reports, 2024, 14(1): 15585. doi: 10.1038/s41598-024-66302-8.
[3]	LIU Zhenhui. Intelligent classification of computer vulnerabilities and network security management system: Combining memristor neural network and improved TCNN model[J]. PLoS One, 2025, 20(1): e0318075. doi: 10.1371/journal.pone.0318075.
[4]	AGUIRRE F, SEBASTIAN A, LE GALLO M, et al. Hardware implementation of memristor-based artificial neural networks[J]. Nature Communications, 2024, 15(1): 1974. doi: 10.1038/s41467-024-45670-9.
[5]	DING Chenchen, REN Yuan, LIU Zhengwu, et al. Transforming memristor noises into computational innovations[J]. Communications Materials, 2025, 6(1): 149. doi: 10.1038/s43246-025-00876-2.
[6]	张睿, 李允臣, 王家宝, 等. 基于深度学习的红外目标检测综述[J]. 计算机技术与发展, 2023, 33(11): 1–8. doi: 10.3969/j.issn.1673-629X.2023.11.001. ZHANG Rui, LI Yunchen, WANG Jiabao, et al. Survey on infrared object detection based on deep learning[J]. Computer Technology and Development, 2023, 33(11): 1–8. doi: 10.3969/j.issn.1673-629X.2023.11.001.
[7]	郭昕婕, 王光燿, 王绍迪. 存内计算芯片研究进展及应用[J]. 电子与信息学报, 2023, 45(5): 1888–1898. doi: 10.11999/JEIT220420. GUO Xinjie, WANG Guangyao, and WANG Shaodi. Technology developments and applications of in-memory computing processors[J]. Journal of Electronics & Information Technology, 2023, 45(5): 1888–1898. doi: 10.11999/JEIT220420.
[8]	KHOR W, CHEN Y K, ROBERTS M, et al. Automated detection and classification of concealed objects using infrared thermography and convolutional neural networks[J]. Scientific Reports, 2024, 14(1): 8353. doi: 10.1038/s41598-024-56636-8.
[9]	YE Longcheng, GAO Zhixuan, FU Jinke, et al. Overview of memristor-based neural network design and applications[J]. Frontiers in Physics, 2022, 10: 839243. doi: 10.3389/fphy.2022.839243.
[10]	JIN Boyan, WANG Zhenlong, WANG Tianyu, et al. Memristor-based artificial neural networks for hardware neuromorphic computing[J]. Research, 2025, 8: 0758. doi: 10.34133/research.0758.
[11]	FENG Weibo, QIN Tianling, and TANG Xin. Advances in infrared detectors for in-memory sensing and computing[J]. Photonics, 2024, 11(12): 1138. doi: 10.3390/photonics11121138.
[12]	WANG Jiejun, PAN Xinqiang, ZHAO Zebin, et al. An infrared near-sensor reservoir computing system based on large-dynamic-space memristor with tens of thousands of states for dynamic gesture perception[J]. Advanced Science, 2024, 11(6): 2307359. doi: 10.1002/advs.202307359.
[13]	ZHANG Wenbin, YAO Peng, GAO Bin, et al. Edge learning using a fully integrated neuro-inspired memristor chip[J]. Science, 2023, 381(6663): 1205–1211. doi: 10.1126/science.ade3483.
[14]	BAE B, PARK M, LEE D, et al. Hetero-integrated InGaAs photodiode and oxide memristor-based artificial optical nerve for in-sensor NIR image processing[J]. Advanced Optical Materials, 2023, 11(3): 2201905. doi: 10.1002/adom.202201905.
[15]	PAN Youjiang, WU Chunyan, BU Guohao, et al. All-in-one neuromorphic light-emitting memristor for infrared image visualization and recognition[J]. Advanced Functional Materials, 2025, 35(37): 2505174. doi: 10.1002/adfm.202505174.
[16]	SHI Tuo, GAO Lili, TIAN Yang, et al. Memristor-based feature learning for pattern classification[J]. Nature Communications, 2025, 16(1): 913. doi: 10.1038/s41467-025-56286-y.
[17]	AZIZA H. Image classification in memristor-based neural networks: A comparative study of software and hardware models using RRAM crossbars[J]. Electronics, 2025, 14(6): 1125. doi: 10.3390/electronics14061125.
[18]	SHI Tuo, ZHANG Hui, CUI Shiyu, et al. Stochastic neuro-fuzzy system implemented in memristor crossbar arrays[J]. Science Advances, 2024(10): eadl3135. doi: 10.1126/sciadv.adl3135.
[19]	LO Y C, WU Junshen, WANG C C, et al. ISSA: Architecting CNN accelerators using input-skippable, set-associative computing-in-memory[J]. IEEE Transactions on Computers, 2024, 73(9): 2136–2149. doi: 10.1109/TC.2024.3404060.
[20]	FORNT J, FONTOVA-MUSTÉ P, CARO M, et al. An energy-efficient GeMM-based convolution accelerator with on-the-fly im2col[J]. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2023, 31(11): 1874–1878. doi: 10.1109/TVLSI.2023.3286122.
[21]	刘津畅, 时拓. 一种多值阻变存储器的振荡收敛编程方法及装置[P]. 中国, 118760422B, 2024. LIU Jinchang and SHI Tuo. Oscillation convergence programming method and device of multi-valued resistive random access memory[P]. CN, 118760422B, 2024.
[22]	JOSHI V, LE GALLO M, HAEFELI S, et al. Accurate deep neural network inference using computational phase-change memory[J]. Nature Communications, 2020, 11(1): 2473. doi: 10.1038/s41467-020-16108-9.