Decision Learning Correction Network: Fusion Classification of Hyperspectral Images and LiDAR Data

WANG Haoyu; LIU Nuofei; CHENG Yuhu; LIU Xiaomin; WANG Xuesong

doi:10.11999/JEIT260362

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 >

WANG Haoyu, LIU Nuofei, CHENG Yuhu, LIU Xiaomin, WANG Xuesong. Decision Learning Correction Network: Fusion Classification of Hyperspectral Images and LiDAR Data[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260362

Citation:

WANG Haoyu, LIU Nuofei, CHENG Yuhu, LIU Xiaomin, WANG Xuesong. Decision Learning Correction Network: Fusion Classification of Hyperspectral Images and LiDAR Data[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT260362

Citation:

PDF( 4850 KB)

Decision Learning Correction Network: Fusion Classification of Hyperspectral Images and LiDAR Data

doi: 10.11999/JEIT260362 cstr: 32379.14.JEIT260362

School of Information and Control Engineering, China University of Mining and Technology, Xuzhou 221116, China

Funds: The National Natural Science Foundation of China (62303468, 62373364, 62573416, 62303469), The Science and Technology Program of Xuzhou (KC2025123)

Received Date: 2026-03-30
Accepted Date: 2026-07-03
Rev Recd Date: 2026-07-02

Available Online: 2026-07-14

Abstract

Abstract

Objective HyperSpectral Images (HSI) and Light Detection and Ranging (LiDAR) provide complementary information for land-cover classification. HSI captures rich spectral information for material discrimination, while LiDAR provides elevation and structural information for spatial characterization. However, most existing fusion methods treat multimodal fusion as a one-shot static aggregation process, implicitly assuming that a fixed fusion strategy is applicable to all pixels and regions. This assumption is difficult to satisfy in complex remote sensing scenes, where class-boundary and cross-modal heterogeneous regions exhibit high information density but account for only a small proportion of samples (Fig. 1). To address this limitation, this paper proposes a Decision Learning Correction Network (DLCN) that reformulates static HSI-LiDAR fusion as a context-dependent sequential decision-making process. Methods The proposed DLCN consists of feature extraction, fusion decision learning, and classification. First, HSI and LiDAR are processed through two parallel branches to extract spectral and spatial features and elevation and structural features, respectively. The extracted features are then concatenated to form the current state and are fed into an Actor-Critic framework. The Actor network generates fusion actions to adaptively adjust modality contributions, while the Critic network evaluates the long-term value of each action for classification. To improve learning from difficult samples, a key-sample-oriented sampling module assigns higher sampling probabilities to samples with larger modal fidelity loss. Meanwhile, a modal fidelity constraint mechanism evaluates spectral fidelity, feature consistency, structural preservation, and resolution matching, and corrects destructive actions during fusion. Through this closed-loop framework, DLCN performs dynamic generation, evaluation, and correction of fusion actions, thereby producing high-quality fusion features for classification (Fig. 2). Results and Discussions Experiments are conducted on the Houston2013, Trento, and MUUFL datasets. DLCN achieves the highest Overall Accuracy (OA) of 97.85%, 99.58%, and 94.38% on the three datasets, respectively, outperforming CHNet, DSymFuser, mPMCL, MEDFN, S3F2Net, and MSAF. The classification maps demonstrate that DLCN effectively reduces misclassification in class-boundary, mixed land-cover, and structurally complex regions, producing results that more closely match the ground-truth maps across all three datasets (Figs. 3～5). Ablation studies further demonstrate that the value-guided policy optimization mechanism, key-sample-oriented sampling module, and modal fidelity constraint mechanism each improve classification performance. Compared with the baseline models, the complete DLCN consistently increases OA on Houston2013, Trento, and MUUFL, validating the effectiveness of the proposed decision-learning-correction framework. Time-step analysis shows that DLCN progressively improves classification accuracy while maintaining stable spectral-angle variation during sequential decision making (Fig. 6). Furthermore, DLCN achieves inference times of 1.32 s, 0.86 s, and 2.23 s on the three datasets, respectively, ranking first among the compared methods. These results indicate that the additional computation introduced by the Actor-Critic decision framework and modal fidelity constraint mechanism is effectively translated into improved classification performance without imposing excessive computational cost. Conclusions This paper proposes a DLCN for HSI and LiDAR fusion classification. Unlike conventional static fusion methods, DLCN formulates multimodal fusion as a sequential decision-making process and adaptively adjusts fusion strategies according to the local context. Its closed-loop framework enables fusion actions to be generated, evaluated, and corrected throughout the decision process, thereby producing high-quality fusion features for classification. Experimental results demonstrate that DLCN produces more accurate classification maps in heterogeneous remote sensing scenes, and the time-step analysis further confirms the stability of the sequential decision-making process. Future work will focus on more fine-grained feature representation and more robust policy optimization to improve model generalization in complex remote sensing scenes.
- Hyperspectral images,
- LiDAR,
- Multimodal fusion,
- Sequential decision optimization,
- Classification

FullText(HTML)

References(22)

References

[1]	马谋, 蔡明娇, 沈雨, 周芳, 蒋俊正. 融合低秩张量分解与乘积图建模的高光谱图像去噪算法[J]. 电子与信息学报, 2025, 47(10): 3951–3966. doi: 10.11999/JEIT250130. MA Mou, CAI Mingjiao, SHEN Yu, ZHOU Fang, JIANG Junzheng. Hyperspectral Image Denoising Algorithm via Joint Low-Rank Tensor Decomposition and Product Graph Modeling[J]. Journal of Electronics Information Technology, 2025, 47(10): 3951–3966. doi: 10.11999/JEIT250130.
[2]	廖帝灵, 赖涛, 黄海风, 王青松. LightMamba: 一种轻量级Mamba用于高光谱图形和激光雷达数据联合分类网络[J]. 电子与信息学报, 2025, 47(12): 4937–4947. doi: 10.11999/JEIT250981. LIAO Diling, LAI Tao, HUANG Haifeng, WANG Qingsong. LightMamba: A Lightweight Mamba Network for the Joint Classification of HSI and LiDAR Data[J]. Journal of Electronics Information Technology, 2025, 47(12): 4937–4947. doi: 10.11999/JEIT250981.
[3]	刁文辉, 龚铄, 辛林霖, 申志平, 孙超. 针对多模态遥感数据的自监督策略模型预训练方法[J]. 电子与信息学报, 2025, 47(6): 1658–1668. doi: 10.11999/JEIT241016. DIAO Wenhui, GONG Shuo, XIN Linlin, SHEN Zhiping, SUN Chao. A Model Pre-training Method with Self-Supervised Strategies for Multimodal Remote Sensing Data[J]. Journal of Electronics Information Technology, 2025, 47(6): 1658–1668. doi: 10.11999/JEIT241016.
[4]	LUO Fulin, HUA Yiyan, FU Chuan, et al. MMD-MLP: LiDAR-guided hyperspectral data classification using local-global directional-MLP with multiresolution multiscale representation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 5508414. doi: 10.1109/TGRS.2025.3550370.
[5]	DUAN Puhong, LUO Yichen, KANG Xudong, et al. LaMamba: Linear attention mamba for hyperspectral image denoising[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 5527113. doi: 10.1109/TGRS.2025.3613739.
[6]	FU Chuan, DU Bo, and ZHANG Liangpei. ReSC-net: Hyperspectral image classification based on attention-enhanced residual module and spatial-channel attention[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5518615. doi: 10.1109/TGRS.2024.3402364.
[7]	YU Chunyan, WANG Hande, SONG Meiping, et al. Interactive graph-based distillation integrated meta-learning network for hyperspectral image incremental classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2026, 64: 5500316. doi: 10.1109/TGRS.2025.3647656.
[8]	DONG Wenqian, YANG Teng, QU Jiahui, et al. Joint contextual representation model-informed interpretable network with dictionary aligning for hyperspectral and LiDAR classification[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(11): 6804–6818. doi: 10.1109/TCSVT.2023.3268757.
[9]	YANG J X, ZHOU Jun, WANG Jing, et al. LiDAR-guided cross-attention fusion for hyperspectral band selection and image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5515815. doi: 10.1109/TGRS.2024.3389651.
[10]	YANG Bin, WANG Xuan, XING Ying, et al. Modality fusion vision transformer for hyperspectral and LiDAR data collaborative classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2024, 17: 17052–17065. doi: 10.1109/JSTARS.2024.3415729.
[11]	HE Ziping, ZHU Qianglin, WANG Wei, et al. Multilevel fusion network based on mix hybrid attention for hyperspectral and LiDAR image classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2026, 19: 470–483. doi: 10.1109/JSTARS.2025.3628896.
[12]	WANG Minhui, SUN Yaxiu, XIANG Jianhong, et al. Joint classification of hyperspectral and LiDAR data based on adaptive gating mechanism and learnable transformer[J]. Remote Sensing, 2024, 16(6): 1080. doi: 10.3390/rs16061080.
[13]	WANG Haoyu, CHENG Yuhu, LIU Xiaomin, et al. Reinforcement learning based Markov edge decoupled fusion network for fusion classification of hyperspectral and LiDAR[J]. IEEE Transactions on Multimedia, 2024, 26: 7174–7187. doi: 10.1109/TMM.2024.3360717.
[14]	SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms[J]. arXiv preprint arXiv: 1707.06347, 2017. doi: 10.48550/arXiv.1707.06347.
[15]	DEBES C, MERENTITIS A, HEREMANS R, et al. Hyperspectral and LiDAR data fusion: Outcome of the 2013 GRSS data fusion contest[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6): 2405–2418. doi: 10.1109/JSTARS.2014.2305441.
[16]	RASTI B, GHAMISI P, and GLOAGUEN R. Hyperspectral and LiDAR fusion using extinction profiles and total variation component analysis[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7): 3997–4007. doi: 10.1109/TGRS.2017.2686450.
[17]	ZHANG Mengmeng, LI Wei, TAO Ran, et al. Information fusion for classification of hyperspectral and LiDAR data using IP-CNN[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5506812. doi: 10.1109/TGRS.2021.3093334.
[18]	CHANG Honghao, BI Haixia, LI Fan, et al. Deep symmetric fusion transformer for multimodal remote sensing data classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5644115. doi: 10.1109/TGRS.2024.3476975.
[19]	NI Kang, XIE Yunan, ZHAO Guofeng, et al. Coarse-to-fine high-order network for hyperspectral and LiDAR classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 5509716. doi: 10.1109/TGRS.2025.3554802.
[20]	LIU Hui, HUANG Chenjia, XIE Tao, et al. Positive matching benefits fusion: A novel contrastive learning framework for hyperspectral and LiDAR data classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2026, 64: 5502218. doi: 10.1109/TGRS.2026.3654168.
[21]	WANG Xianghai, SONG Liyang, FENG Yining, et al. S3F2Net: Spatial-spectral-structural feature fusion network for hyperspectral image and LiDAR data classification[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2025, 35(5): 4801–4815. doi: 10.1109/TCSVT.2025.3525734.
[22]	SHI Lulu, LI Chunchao, ZENG Zhengchao, et al. Masked self-attention fusion network for joint classification of hyperspectral and LiDAR data[J]. IEEE Transactions on Image Processing, 2026, 35: 346–360. doi: 10.1109/TIP.2025.3648926.