Diffusion Model and Edge Information Guided Single-photon Image Reconstruction Algorithm

ZHANG Dan; LIAN Qiusheng; YANG Yuchi

doi:10.11999/JEIT241063

Volume 47 Issue 7

Jul. 2025

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 > 47(7): 2237-2248

ZHANG Dan, LIAN Qiusheng, YANG Yuchi. Diffusion Model and Edge Information Guided Single-photon Image Reconstruction Algorithm[J]. Journal of Electronics & Information Technology, 2025, 47(7): 2237-2248. doi: 10.11999/JEIT241063

Citation:

ZHANG Dan, LIAN Qiusheng, YANG Yuchi. Diffusion Model and Edge Information Guided Single-photon Image Reconstruction Algorithm[J]. Journal of Electronics & Information Technology, 2025, 47(7): 2237-2248. doi: 10.11999/JEIT241063

Citation:

PDF( 4541 KB)

Diffusion Model and Edge Information Guided Single-photon Image Reconstruction Algorithm

doi: 10.11999/JEIT241063 cstr: 32379.14.JEIT241063

ZHANG Dan^{1, 3},
LIAN Qiusheng^{1, 2
,
,},
YANG Yuchi⁴

1.
School of Information Science and Engineering, Yanshan University, QinhuangDao, 066004, China
2.
Hebei Key Laboratory of Information Transmission and Signal Processing, Qinhuangdao, 066004, China
3.
School of Information Science and Engineering, Hebei University of Science and Technology, Shijiazhuang, 050018, China
4.
School of Management Science and Information Engineering, Hebei University of Economics and Business, Shijiazhuang, 050061, China

Funds: Hebei Natural Science Foundation (F2022203030)

Received Date: 2024-12-25
Rev Recd Date: 2025-03-31

Available Online: 2025-04-23

Publish Date: 2025-07-22

Abstract

Abstract

Objective Quanta Image Sensors (QIS) are solid-state sensors that encode scene information into binary bit-streams. The reconstruction for QIS consists of recovering the original scenes from these bit-streams, which is an ill-posed problem characterized by incomplete measurements. Existing reconstruction algorithms based on physical sensors primarily use maximum-likelihood estimation, which may introduce noise-like components and result in insufficient sharpness, especially under low oversampling factors. Model-based optimization algorithms for QIS generally combine the likelihood function with an explicit or implicit image prior in a cost function. Although these methods provide superior quality, they are computationally intensive due to the use of iterative solvers. Additionally, intrinsic readout noise in QIS circuits can degrade the binary response, complicating the imaging process. To address these challenges, an image reconstruction algorithm, Edge-Guided Diffusion Model (EGDM), is proposed for single-photon sensors. This algorithm utilizes a diffusion model guided by edge information to achieve high-speed, high-quality imaging for QIS while improving robustness to readout noise. Methods The proposed EGDM algorithm incorporates a measurement subspace constrained by binary measurements into the unconditional diffusion model sampling framework. This constraint ensures that the generated images satisfy both data consistency and the natural image distribution. Due to high noise intensity in latent variables during the initial reverse diffusion stages of diffusion models, texture details may be lost, and structural components may become blurred. To enhance reconstruction quality while minimizing the number of sampling steps, a bilateral filter is applied to extract edge information from images generated by maximum likelihood estimation. Additionally, the integration of jump sampling with a measurement subspace projection termination strategy reduces inference time and computational complexity, while preserving visual quality. Results and Discussions Experimental results on both the benchmark datasets, Set10 and BSD68 (Fig. 6, Fig. 7, Table 2), and the real video frame (Fig. 8) demonstrate that the proposed EGDM method outperforms several state-of-the-art reconstruction algorithms for QIS and diffusion-based methods in both objective metrics and visual perceptual quality. Notably, EGDM achieves an improvement of approximately 0.70 dB to 3.00 dB compared to diffusion-based methods for QIS in terms of Peak Signal-to-Noise Ratio (PSNR) across all oversampling factors. For visualization, the proposed EGDM produces significantly finer textures and preserves image sharpness. In the case of real QIS video sequences (Fig. 8), EGDM preserves more detailed information while mitigating blur artifacts commonly found in low-light video capture. Furthermore, to verify the robustness of the reconstruction algorithm to readout noise, the reconstruction of the original scene from the measurements is conducted under various readout noise levels. The experimental results (Table 3, Fig. 9, Fig. 10) demonstrate the effectiveness of the proposed EGDM method in suppressing readout noise, as it achieves the lowest average Mean Squared Error (MSE) and superior quality compared to other algorithms in terms of PSNR, particularly at higher noise levels. Visually, EGDM produces the best results, with sharp edges and clear texture patterns even under severe noise conditions. Compared to the EGDM algorithm without acceleration strategies, the implementation of jump sampling and measurement subspace projection termination strategies reduces the execution time by 5 seconds and 1.9 seconds, respectively (Table 4). Moreover, EGDM offers faster computation speeds than other methods, including deep learning-based reconstruction algorithms that rely on GPU-accelerated computing. After thorough evaluation, these experimental findings confirm that the high-performance reconstruction and rapid imaging speed make the proposed EGDM method an excellent choice for practical applications. Conclusions This paper proposes a single-photon image reconstruction algorithm, EGDM, based on a diffusion model and edge information guidance, overcoming the limitations of traditional algorithms that produce suboptimal solutions in the presence of low oversampling factors and readout noise. The measurement subspace defined by binary measurements is introduced as a constraint in the diffusion model sampling process, ensuring that the reconstructed images satisfy both data consistency and the characteristics of natural image distribution. The bilateral filter is applied to extract edge components from the MLE-generated image as auxiliary information. Furthermore, a hybrid sampling strategy combining jump sampling with measurement subspace projection termination is introduced, significantly reducing the number of sampling steps while improving reconstruction quality. Experimental results on both benchmark datasets and real video frames demonstrate that: (1) Compared with conventional image reconstruction algorithms for QIS, EGDM achieves excellent performance in both average PSNR and SSIM. (2) Under different oversampling factors, EGDM outperforms existing diffusion-based reconstruction methods by a large margin. (3) Compared with existing techniques, the EGDM algorithm requires less computational time while exhibiting strong robustness against readout noise, confirming its effectiveness in practical applications. Future research could focus on developing parameter-free reconstruction frameworks that preserve imaging quality and extending EGDM to address more complex environmental challenges, such as dynamic low-light or high dynamic range imaging for QIS.
- Quanta Image Sensor (QIS),
- Single-photon imaging,
- Diffusion model,
- Edge information

FullText(HTML)

References(30)

References

[1]	FOSSUM E R, MA Jiaju, MASOODIAN S, et al. The quanta image sensor: Every photon counts[J]. Sensors, 2016, 16(8): 1260. doi: 10.3390/s16081260.
[2]	GYONGY I, AL ABBAS T, DUTTON N, et al. Object tracking and reconstruction with a quanta image sensor[C]. The International Image Sensor Workshop, Hiroshima, Japan, 2017: 4.
[3]	POLAND S P, KRSTAJIĆ N, MONYPENNY J, et al. A high speed multifocal multiphoton fluorescence lifetime imaging microscope for live-cell FRET imaging[J]. Biomedical Optics Express, 2015, 6(2): 277–296. doi: 10.1364/BOE.6.000277.
[4]	SEITZ P and THEUWISSEN A J P. Single-Photon Imaging[M]. Berlin, Heidelberg: Springer, 2011. doi: 10.1007/978-3-642-18443-7.
[5]	YANG Feng, LU Y M, SBAIZ L, et al. Bits from photons: Oversampled image acquisition using binary Poisson statistics[J]. IEEE Transactions on Image Processing, 2012, 21(4): 1421–1436. doi: 10.1109/TIP.2011.2179306.
[6]	CHAN S H, ELGENDY O A, and WANG Xiran. Images from bits: Non-iterative image reconstruction for quanta image sensors[J]. Sensors, 2016, 16(11): 1961. doi: 10.3390/s16111961.
[7]	YANG Feng, SBAIZ L, CHARBON E, et al. Image reconstruction in the gigavision camera[C]. The 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops, Kyoto, Japan, 2009: 2212–2219. doi: 10.1109/ICCVW.2009.5457554.
[8]	YANG Feng, LU Y M, SBAIZ L, et al. An optimal algorithm for reconstructing images from binary measurements[C]. SPIE 7533, Computational Imaging VIII, San Jose, USA, 2010: 75330K. doi: 10.1117/12.850887.
[9]	CHAN S H and LU Y M. Efficient image reconstruction for gigapixel quantum image sensors[C]. 2014 IEEE Global Conference on Signal and Information Processing, Atlanta, USA, 2014: 312–316. doi: 10.1109/GlobalSIP.2014.7032129.
[10]	CHAN S H, WANG Xiran, and ELGENDY O A. Plug-and-play ADMM for image restoration: Fixed-point convergence and applications[J]. IEEE Transactions on Computational Imaging, 2017, 3(1): 84–98. doi: 10.1109/TCI.2016.2629286.
[11]	RYU E, LIU Jialin, WANG Sicheng, et al. Plug-and-play methods provably converge with properly trained denoisers[C]. The 36th International Conference on Machine Learning, Long Beach, USA, 2019: 5546–5557.
[12]	WEI Kaixuan, AVILES-RIVERO A, LIANG Jingwei, et al. TFPnP: Tuning-free plug-and-play proximal algorithms with applications to inverse imaging problems[J]. The Journal of Machine Learning Research, 2022, 23(1): 16.
[13]	WANG Xingzheng . Single-photon cameras image reconstruction using vision transformer[C]. 2023 IEEE 3rd International Conference on Computer Communication and Artificial Intelligence, Taiyuan, China, 2023: 296–300. doi: 10.1109/CCAI57533.2023.10201259.
[14]	胡铭菲, 左信, 刘建伟. 深度生成模型综述[J]. 自动化学报, 2022, 48(1): 40–74. doi: 10.16383/j.aas.c190866. HU Mingfei, ZUO Xin, and LIU Jianwei. Survey on deep generative model[J]. Acta Automatica Sinica, 2022, 48(1): 40–74. doi: 10.16383/j.aas.c190866.
[15]	XIANG Xuezhi, ABDEIN R, LV Ning, et al. InvFlow: Involution and multi-scale interaction for unsupervised learning of optical flow[J]. Pattern Recognition, 2024, 145: 109918. doi: 10.1016/j.patcog.2023.109918.
[16]	厉行, 樊养余, 郭哲, 等. 基于边缘领域自适应的立体匹配算法[J]. 电子与信息学报, 2024, 46(7): 2970–2980. doi: 10.11999/JEIT231113. LI Xing, FAN Yangyu, GUO Zhe, et al. Edge domain adaptation for stereo matching[J]. Journal of Electronics & Information Technology, 2024, 46(7): 2970–2980. doi: 10.11999/JEIT231113.
[17]	LIN Xinmiao, LI Yikang, HSIAO J, et al. Catch missing details: Image reconstruction with frequency augmented variational autoencoder[C]. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, 2023: 1736–1745. doi: 10.1109/CVPR52729.2023.00173.
[18]	HO J, JAIN A, and ABBEEL P. Denoising diffusion probabilistic models[C]. The 34th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2020: 574.
[19]	XIA Bin, ZHANG Yulun, WANG Shiyin, et al. Diffir: Efficient diffusion model for image restoration[C]. Tthe 2023 IEEE/CVF International Conference on Computer Vision, Paris, France, 2023: 13049–13059. doi: 10.1109/ICCV51070.2023.01204.
[20]	CHUNG H, KIM J, MCCANN M T, et al. Diffusion posterior sampling for general noisy inverse problems[C]. 2023 International Conference on Learning Representations, Kigali, Rwanda, 2023: 1–30.
[21]	FEI Ben, LYU Zhaoyang, PAN Liang, et al. Generative diffusion prior for unified image restoration and enhancement[C]. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, 2023: 9935–9946. doi: 10.1109/CVPR52729.2023.00958.
[22]	SONG Jiaming, MENG Chenlin, and ERMON S. Denoising diffusion implicit models[C]. The 9th International Conference on Learning Representations, Vienna, Austria, 2021: 1–20.
[23]	SONG Yang, SOHL-DICKSTEIN J, KINGMA D P, et al. Score-based generative modeling through stochastic differential equations[C]. The 9th International Conference on Learning Representations, Vienna, Austria, 2021: 1–36.
[24]	ZHU Yuanzhi, ZHANG Kai, LIANG Jingyun, et al. Denoising diffusion models for plug-and-play image restoration[C]. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Vancouver, Canada, 2023: 1219–1229. doi: 10.1109/CVPRW59228.2023.00129.
[25]	BYERS R. A bisection method for measuring the distance of a stable matrix to the unstable matrices[J]. SIAM Journal on Scientific and Statistical Computing, 1988, 9(5): 875–881. doi: 10.1137/0909059.
[26]	WANG Yinhuai, YU Jiwen, and ZHANG Jian. Zero-shot image restoration using denoising diffusion null-space model[C]. 2023 International Conference on Learning Representations, Kigali, Rwanda, 2023: 1–31.
[27]	BURRI S, MARUYAMA Y, MICHALET X, et al. Architecture and applications of a high resolution gated SPAD image sensor[J]. Optics Express, 2014, 22(14): 17573–17589. doi: 10.1364/OE.22.017573.
[28]	DHARIWAL P and NICHOL A. Diffusion models beat GANs on image synthesis[C]. The 35th International Conference on Neural Information Processing Systems, 2021: 672.
[29]	CHI Yiheng, GNANASAMBANDAM A, KOLTUN V, et al. Dynamic low-light imaging with quanta image sensors[C]. The 16th European Conference on Computer Vision, Glasgow, UK, 2020: 122–138. doi: 10.1007/978-3-030-58589-1_8.
[30]	GNANASAMBANDAM A and CHAN S H. Image classification in the dark using quanta image sensors[C]. 2020 European Conference on Computer Vision, Glasgow, UK, 2020: 484–501. doi: 10.1007/978-3-030-58598-3_29.