Decision Learning Correction Network for HSI and LiDAR Fusion Classification

WANG Haoyu; LIU Nuofei; CHENG Yuhu; LIU Xiaomin; WANG Xuesong

doi:10.11999/JEIT260362

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2026 >

WANG Haoyu, LIU Nuofei, CHENG Yuhu, LIU Xiaomin, WANG Xuesong. Decision Learning Correction Network for HSI and LiDAR Fusion Classification[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260362

Citation:

WANG Haoyu, LIU Nuofei, CHENG Yuhu, LIU Xiaomin, WANG Xuesong. Decision Learning Correction Network for HSI and LiDAR Fusion Classification[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260362

Citation:

PDF( 4744 KB)

Decision Learning Correction Network for HSI and LiDAR Fusion Classification

doi: 10.11999/JEIT260362 cstr: 32379.14.JEIT260362

School of Information and Control Engineering, China University of Mining and Technology, Xuzhou 221116, China

Funds: National Natural Science Foundation of China (62303468), Science and Technology Program of Xuzhou (KC2025123), National Natural Science Foundation of China (62373364, 62573416, 62303469)

Received Date: 2026-03-30
Accepted Date: 2026-07-03
Rev Recd Date: 2026-07-03

Available Online: 2026-07-14

Abstract

Abstract

Objective Hyperspectral images(HSI) and LiDAR data provide complementary information for land-cover classification. HSI contains rich spectral responses for material discrimination, while LiDAR supplies elevation and structural information for spatial perception. However, most existing fusion methods treat multimodal fusion as a static aggregation process, assuming that a fixed fusion rule is suitable for all pixels and regions. This assumption is difficult to satisfy in complex remote sensing scenes, where class boundaries and cross-modal heterogeneous regions show higher information density but occupy only a small proportion of samples (Fig.1). To address this problem, this paper proposes a Decision Learning Correction Network (DLCN), which transforms static HSI-LiDAR fusion into a context-dependent sequential decision-making process. Methods The proposed DLCN consists of feature extraction, fusion decision learning, and final classification. First, HSI and LiDAR are processed by two parallel branches to extract spectral-spatial features and elevation-structural features, respectively. Then, the extracted features are concatenated as the current fusion state and input into an Actor-Critic framework. The Actor network generates fusion actions to dynamically adjust modal contributions, while the Critic network evaluates the long-term value of each action for final classification. To improve the learning of difficult samples, a key-sample-oriented sampling module assigns higher sampling probabilities to samples with larger modal fidelity losses. Meanwhile, a modal fidelity constraint mechanism evaluates spectral fidelity, feature consistency, structural preservation, and resolution matching, and corrects destructive actions during fusion. Through this closed-loop structure, DLCN realizes dynamic fusion action generation, evaluation, and correction (Fig. 2). Results and Discussions Experiments are conducted on Houston2013, Trento, and MUUFL datasets. DLCN achieves the best OA values of 97.85%, 99.58%, and 94.38% on the three datasets, respectively, outperforming CHNet, DSymFuser, mPMCL, MEDFN, S3F2Net, and MSAF. The classification maps show that DLCN effectively reduces misclassification in class-boundary, mixed land-cover, and structurally complex regions, producing results closer to the ground-truth maps on the three datasets (Fig. 3-Fig. 5). Ablation results demonstrate that the value-guided policy optimization mechanism, key-sample-oriented sampling module, and modal fidelity constraint mechanism all contribute to performance improvement. Compared with the baseline model, the complete DLCN obtains consistent OA gains on Houston2013, Trento, and MUUFL, verifying the effectiveness of the proposed decision-learning-correction framework. The temporal analysis shows that DLCN gradually improves class accuracy while maintaining stable spectral-angle variation during sequential decision steps (Fig. 6). In addition, DLCN achieves inference times of 1.32 s, 0.86 s, and 2.23 s on the three datasets, respectively, ranking first among the compared methods. This indicates that the introduced Actor-Critic decision mechanism and modal fidelity constraints can be effectively converted into classification gains without causing excessive computational burden. Conclusions This paper proposes DLCN for HSI and LiDAR fusion classification. Different from static fusion methods, DLCN models multimodal fusion as a sequential decision-making process and dynamically adjusts fusion strategies according to local context. The closed-loop design enables the model to generate, evaluate, and correct fusion actions. Experimental results show that DLCN achieves more accurate classification maps in heterogeneous scenes, and the temporal analysis further confirms the stability of the sequential decision process. Future work will focus on more fine-grained feature representation and more robust policy optimization to improve generalization in complex remote sensing scenes.
- Hyperspectral images,
- LiDAR,
- Multimodal fusion,
- Sequential decision optimization,
- Classification

FullText(HTML)

References(22)

References

[1]	ZHAO Xudong, ZHANG Mengmeng, TAO Ran, et al. Fractional Fourier image transformer for multimodal remote sensing data classification[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(2): 2314–2326. doi: 10.1109/TNNLS.2022.3189994.
[2]	WU Xin, HONG Danfeng, and CHANUSSOT J. Convolutional neural networks for multimodal remote sensing data classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5517010. doi: 10.1109/TGRS.2021.3124913.
[3]	VIVONE G, DENG Liangjian, DENG Shangqi, et al. Deep learning in remote sensing image fusion: Methods, protocols, data, and future perspectives[J]. IEEE Geoscience and Remote Sensing Magazine, 2025, 13(1): 269–310. doi: 10.1109/MGRS.2024.3495516.
[4]	LUO Fulin, HUA Yiyan, FU Chuan, et al. MMD-MLP: LiDAR-guided hyperspectral data classification using local-global directional-MLP with multiresolution multiscale representation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 5508414. doi: 10.1109/TGRS.2025.3550370.
[5]	DUAN Puhong, LUO Yichen, KANG Xudong, et al. LaMamba: Linear attention mamba for hyperspectral image denoising[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 5527113. doi: 10.1109/TGRS.2025.3613739.
[6]	FU Chuan, DU Bo, and ZHANG Liangpei. ReSC-net: Hyperspectral image classification based on attention-enhanced residual module and spatial-channel attention[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5518615. doi: 10.1109/TGRS.2024.3402364.
[7]	YU Chunyan, WANG Hande, SONG Meiping, et al. Interactive graph-based distillation integrated meta-learning network for hyperspectral image incremental classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2026, 64: 5500316. doi: 10.1109/TGRS.2025.3647656.
[8]	DONG Wenqian, YANG Teng, QU Jiahui, et al. Joint contextual representation model-informed interpretable network with dictionary aligning for hyperspectral and LiDAR classification[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(11): 6804–6818. doi: 10.1109/TCSVT.2023.3268757.
[9]	YANG J X, ZHOU Jun, WANG Jing, et al. LiDAR-guided cross-attention fusion for hyperspectral band selection and image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5515815. doi: 10.1109/TGRS.2024.3389651.
[10]	YANG Bin, WANG Xuan, XING Ying, et al. Modality fusion vision transformer for hyperspectral and LiDAR data collaborative classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2024, 17: 17052–17065. doi: 10.1109/JSTARS.2024.3415729.
[11]	HE Ziping, ZHU Qianglin, WANG Wei, et al. Multilevel fusion network based on mix hybrid attention for hyperspectral and LiDAR image classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2026, 19: 470–483. doi: 10.1109/JSTARS.2025.3628896.
[12]	WANG Minhui, SUN Yaxiu, XIANG Jianhong, et al. Joint classification of hyperspectral and LiDAR data based on adaptive gating mechanism and learnable transformer[J]. Remote Sensing, 2024, 16(6): 1080. doi: 10.3390/rs16061080.
[13]	WANG Haoyu, CHENG Yuhu, LIU Xiaomin, et al. Reinforcement learning based Markov edge decoupled fusion network for fusion classification of hyperspectral and LiDAR[J]. IEEE Transactions on Multimedia, 2024, 26: 7174–7187. doi: 10.1109/TMM.2024.3360717.
[14]	SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms[J]. arXiv preprint arXiv: 1707.06347, 2017. doi: 10.48550/arXiv.1707.06347.
[15]	DEBES C, MERENTITIS A, HEREMANS R, et al. Hyperspectral and LiDAR data fusion: Outcome of the 2013 GRSS data fusion contest[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6): 2405–2418. doi: 10.1109/JSTARS.2014.2305441.
[16]	RASTI B, GHAMISI P, and GLOAGUEN R. Hyperspectral and LiDAR fusion using extinction profiles and total variation component analysis[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7): 3997–4007. doi: 10.1109/TGRS.2017.2686450.
[17]	ZHANG Mengmeng, LI Wei, TAO Ran, et al. Information fusion for classification of hyperspectral and LiDAR data using IP-CNN[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5506812. doi: 10.1109/TGRS.2021.3093334.
[18]	NI Kang, XIE Yunan, ZHAO Guofeng, et al. Coarse-to-fine high-order network for hyperspectral and LiDAR classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 5509716. doi: 10.1109/TGRS.2025.3554802.
[19]	CHANG Honghao, BI Haixia, LI Fan, et al. Deep symmetric fusion transformer for multimodal remote sensing data classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5644115. doi: 10.1109/TGRS.2024.3476975.
[20]	LIU Hui, HUANG Chenjia, XIE Tao, et al. Positive matching benefits fusion: A novel contrastive learning framework for hyperspectral and LiDAR data classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2026, 64: 5502218. doi: 10.1109/TGRS.2026.3654168.
[21]	WANG Xianghai, SONG Liyang, FENG Yining, et al. S3F2Net: Spatial-spectral-structural feature fusion network for hyperspectral image and LiDAR data classification[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2025, 35(5): 4801–4815. doi: 10.1109/TCSVT.2025.3525734.
[22]	SHI Lulu, LI Chunchao, ZENG Zhengchao, et al. Masked self-attention fusion network for joint classification of hyperspectral and LiDAR data[J]. IEEE Transactions on Image Processing, 2026, 35: 346–360. doi: 10.1109/TIP.2025.3648926.