决策-学习-修正网络：高光谱图像与激光雷达融合分类

王浩宇; 刘诺菲; 程玉虎; 刘晓敏; 王雪松

doi:10.11999/JEIT260362

决策-学习-修正网络：高光谱图像与激光雷达融合分类

doi: 10.11999/JEIT260362 cstr: 32379.14.JEIT260362

中国矿业大学信息与控制工程学院徐州 221116

基金项目: 国家自然科学基金 (62303468, 62373364, 62573416, 62303469)，徐州科技项目(KC2025123)

详细信息

作者简介:
王浩宇：男，副教授，研究方向为高光谱图像分类、多模态融合、机器学习

刘诺菲：女，硕士生，研究方向为强化学习、多模态融合、深度学习

程玉虎：男，教授，研究方向为强化学习、图像处理、深度学习

刘晓敏：女，副教授，研究方向为强化学习、多模态融合、图像处理

王雪松：女，教授，研究方向为强化学习、图像处理、深度学习

通讯作者:
王雪松　wangxuesongcumt@163.com

中图分类号: TN911.73; TP18
计量
- 文章访问数: 308
- HTML全文浏览量: 139
- PDF下载量: 17
- 被引次数: 0
出版历程
- 收稿日期: 2026-03-30
- 修回日期: 2026-07-02
- 录用日期: 2026-07-03
- 网络出版日期: 2026-07-14

Decision Learning Correction Network: Fusion Classification of Hyperspectral Images and LiDAR Data

School of Information and Control Engineering, China University of Mining and Technology, Xuzhou 221116, China

Funds: The National Natural Science Foundation of China (62303468, 62373364, 62573416, 62303469), The Science and Technology Program of Xuzhou (KC2025123)

摘要

摘要: 高光谱图像与激光雷达在光谱表征和空间结构刻画方面具有天然互补性，其有效融合被认为是提升遥感地物分类性能的重要途径。然而，现有方法多将融合过程建模为固定映射下的一次性静态聚合，隐含地假设同一融合策略能够适用于不同区域，因而难以应对遥感影像中普遍存在的空间异质性和样本复杂性。为此，该文提出一种决策-学习-修正网络，将传统静态融合重构为面向局部上下文的序列决策过程，使模型能够在连续交互中学习上下文依赖的自适应融合策略。针对关键样本易被忽视的问题，构建关键样本导向采样模块，依据样本决策困难度提高复杂区域样本的训练参与度。针对融合动作可能破坏模态特性的问题，设计模态保真约束机制，对不合理动作进行修正，以保证融合特征质量。实验结果表明，所提方法在Houston2013, Trento和MUUFL 3个基准数据集上的总体精度分别提升0.82%, 0.38%和1.76%，验证了该方法在复杂场景下的有效性与优越性。
- 高光谱图像 /
- 激光雷达 /
- 多模态融合 /
- 序贯决策优化 /
- 分类
Abstract: Objective HyperSpectral Images (HSI) and Light Detection And Ranging (LiDAR) provide complementary information for land-cover classification. HSI captures rich spectral information for material discrimination, while LiDAR provides elevation and structural information for spatial characterization. However, most existing fusion methods treat multimodal fusion as a one-shot static aggregation process, implicitly assuming that a fixed fusion strategy is applicable to all pixels and regions. This assumption is difficult to satisfy in complex remote sensing scenes, where class-boundary and cross-modal heterogeneous regions exhibit high information density but account for only a small proportion of samples (Fig. 1). To address this limitation, this paper proposes a Decision Learning Correction Network (DLCN) that reformulates static HSI-LiDAR fusion as a context-dependent sequential decision-making process. Methods The proposed DLCN consists of feature extraction, fusion decision learning, and classification. First, HSI and LiDAR are processed through two parallel branches to extract spectral and spatial features and elevation and structural features, respectively. The extracted features are then concatenated to form the current state and are fed into an Actor-Critic framework. The Actor network generates fusion actions to adaptively adjust modality contributions, while the Critic network evaluates the long-term value of each action for classification. To improve learning from difficult samples, a key-sample-oriented sampling module assigns higher sampling probabilities to samples with larger modal fidelity loss. Meanwhile, a modal fidelity constraint mechanism evaluates spectral fidelity, feature consistency, structural preservation, and resolution matching, and corrects destructive actions during fusion. Through this closed-loop framework, DLCN performs dynamic generation, evaluation, and correction of fusion actions, thereby producing high-quality fusion features for classification (Fig. 2). Results and Discussions Experiments are conducted on the Houston2013, Trento, and MUUFL datasets. DLCN achieves the highest Overall Accuracy (OA) of 97.85%, 99.58%, and 94.38% on the three datasets, respectively, outperforming CHNet, DSymFuser, mPMCL, MEDFN, S3F2Net, and MSAF. The classification maps demonstrate that DLCN effectively reduces misclassification in class-boundary, mixed land-cover, and structurally complex regions, producing results that more closely match the ground-truth maps across all three datasets (Figs. 3～5). Ablation studies further demonstrate that the value-guided policy optimization mechanism, key-sample-oriented sampling module, and modal fidelity constraint mechanism each improve classification performance. Compared with the baseline models, the complete DLCN consistently increases OA on Houston2013, Trento, and MUUFL, validating the effectiveness of the proposed decision-learning-correction framework. Time-step analysis shows that DLCN progressively improves classification accuracy while maintaining stable spectral-angle variation during sequential decision making (Fig. 6). Furthermore, DLCN achieves inference times of 1.32 s, 0.86 s, and 2.23 s on the three datasets, respectively, ranking first among the compared methods. These results indicate that the additional computation introduced by the Actor-Critic decision framework and modal fidelity constraint mechanism is effectively translated into improved classification performance without imposing excessive computational cost. Conclusions This paper proposes a DLCN for HSI and LiDAR fusion classification. Unlike conventional static fusion methods, DLCN formulates multimodal fusion as a sequential decision-making process and adaptively adjusts fusion strategies according to the local context. Its closed-loop framework enables fusion actions to be generated, evaluated, and corrected throughout the decision process, thereby producing high-quality fusion features for classification. Experimental results demonstrate that DLCN produces more accurate classification maps in heterogeneous remote sensing scenes, and the time-step analysis further confirms the stability of the sequential decision-making process. Future work will focus on more fine-grained feature representation and more robust policy optimization to improve model generalization in complex remote sensing scenes.
- HyperSpectral Images (HSI) /
- Light Detection And Ranging (LiDAR) /
- Multimodal fusion /
- Sequential decision optimization /
- Classification

HTML全文

图 1 遥感数据的信息密度不均(以Houston2013数据集为例)

下载: 全尺寸图片幻灯片

图 2 决策-学习-修正网络

下载: 全尺寸图片幻灯片

图 3 分类图(Houston2013)

下载: 全尺寸图片幻灯片

图 4 分类图(MUUFL)

下载: 全尺寸图片幻灯片

图 5 分类图(Trento)

下载: 全尺寸图片幻灯片

图 6 DLCN在一段时间步长上

下载: 全尺寸图片幻灯片

表 1 Houston2013数据集的分类精度(%)

类别	CHNet	DSymFuser	mPMCL	MEDFN	S3F2Net	MSAF	DLCN
Healthy grass	100	88.98	100	84.25	98.39	98.29	97.15
Stressed grass	86.00	98.68	98.21	98.33	93.89	86.47	97.65
Synthetic grass	100	99.60	100	99.86	100	100	100
Trees	98.30	99.43	94.13	99.52	99.91	97.82	97.73
Soil	100	100	100	100	99.91	99.81	100
Water	100	100	100	92.62	100	99.30	100
Residential	98.13	99.81	98.69	92.82	99.91	98.41	99.72
Commercial	96.39	90.60	94.59	84.81	96.87	98.96	88.32
Road	99.81	87.82	89.24	68.69	43.34	94.33	98.68
Highway	90.44	94.21	100	84.51	100	97.97	100
Railway	96.58	97.15	92.13	91.58	100	98.96	99.43
Parking lot 1	100	96.06	98.27	81.02	99.52	97.31	97.60
Parking lot 2	100	92.98	95.08	95.95	100	99.65	96.14
Tennis court	100	98.79	100	99.77	100	100	100
Running track	100	100	100	100	100	99.36	99.79
OA	97.03	95.72	96.88	90.23	94.07	97.21	97.85
AA	97.71	96.28	97.36	91.58	95.44	97.78	98.15
Kappa	96.78	95.35	96.61	89.44	93.56	96.97	97.67

下载: 导出CSV

表 3 MUUFL数据集的分类精度(%)

类别	CHNet	DSymFuser	mPMCL	MEDFN	S3F2Net	MSAF	DLCN
Trees	91.70	93.99	92.20	79.11	91.71	92.87	96.62
Mostly grass	86.41	88.13	87.79	92.57	87.62	86.60	89.25
Mixed ground surface	84.85	87.12	81.61	53.30	83.02	79.93	90.54
Dirt and sand	96.36	96.84	97.37	96.12	95.40	96.90	95.05
Road	89.20	91.19	86.40	82.33	90.21	89.00	93.24
Water	100	99.68	100	99.68	99.37	99.68	99.68
Building shadow	93.13	96.74	93.33	96.45	95.54	93.52	97.36
Building	95.34	95.42	97.72	86.81	90.03	95.75	96.29
Sidewalk	84.21	90.36	85.10	65.91	77.48	84.78	79.51
Yellow curb	100	96.97	100	81.82	96.97	93.94	90.91
Cloth panels	99.16	99.16	100	100	100	99.16	99.16
OA	90.61	92.62	90.51	79.25	89.88	90.57	94.38
AA	92.76	94.15	92.87	84.92	91.58	92.01	93.42
Kappa	87.68	90.27	87.55	73.61	86.74	87.60	92.52

下载: 导出CSV

表 2 Trento数据集的分类精度(%)

类别	CHNet	DSymFuser	mPMCL	MEDFN	S3F2Net	MSAF	DLCN
Apple trees	99.46	96.72	94.55	96.12	100	99.90	99.28
Buildings	95.93	99.24	99.78	98.41	99.68	99.46	98.70
Ground	95.19	97.86	99.73	92.86	87.97	98.13	97.06
Woods	99.98	100	100	100	100	100	100
Vineyard	100	100	100	99.86	98.35	99.72	100
Roads	97.05	97.44	97.71	98.36	94.00	95.28	98.46
OA	99.17	99.20	99.01	99.01	98.62	99.32	99.58
AA	97.94	98.55	98.63	97.60	96.67	98.75	98.92
Kappa	98.89	98.93	98.68	98.67	98.15	99.09	99.44

下载: 导出CSV

表 4 消融模型

组件	Baseline-A	Baseline-B	Baseline-C	Baseline-D	DLCN
OMVGS	×	×	√	√	√
KSOSM	×	×	×	√	√
MFCM	×	×	×	×	√

下载: 导出CSV

表 5 不同组件对OA的影响(%)

数据集	Baseline-A	Baseline-B	Baseline-C	Baseline-D	DLCN
Houston2013	95.71	94.61	96.15	97.38	97.85
Trento	96.46	97.28	98.27	99.07	99.58
MUUFL	91.09	90.21	92.54	93.86	94.38

下载: 导出CSV

表 6 不同方法对每个数据集的计算代价

数据集	计算代价	CHNet	DSymFuser	mPMCL	MEDFN	S3F2Net	MSAF	DLCN
Houston2013	Time(s)	3.07	6.42	15.36	2.34	1.99	1.61	1.32
	FLOPs(M)	334.28	75.43	7.59	165.76	57.50	3.92	161.42
	params(M)	23.78	0.85	0.17	1.45	0.28	0.05	0.79
Trento	Time(s)	7.83	5.35	25.96	2.14	0.93	7.167	0.86
	FLOPs(M)	334.18	41.36	27.69	164.98	10.89	3.76	160.25
	params(M)	23.73	0.47	0.18	1.45	0.27	0.05	0.79
MUUFL	Time(s)	14.31	9.40	34.36	3.56	2.56	14.75	2.23
	FLOPs(M)	334.23	41.85	7.57	164.54	8.68	3.77	160.40
	params(M)	23.75	0.48	0.17	1.45	0.28	0.05	0.79

下载: 导出CSV

参考文献(22)

[1]	马谋, 蔡明娇, 沈雨, 周芳, 蒋俊正. 融合低秩张量分解与乘积图建模的高光谱图像去噪算法[J]. 电子与信息学报, 2025, 47(10): 3951–3966. doi: 10.11999/JEIT250130. MA Mou, CAI Mingjiao, SHEN Yu, ZHOU Fang, JIANG Junzheng. Hyperspectral Image Denoising Algorithm via Joint Low-Rank Tensor Decomposition and Product Graph Modeling[J]. Journal of Electronics & Information Technology, 2025, 47(10): 3951–3966. doi: 10.11999/JEIT250130.
[2]	廖帝灵, 赖涛, 黄海风, 王青松. LightMamba: 一种轻量级Mamba用于高光谱图形和激光雷达数据联合分类网络[J]. 电子与信息学报, 2025, 47(12): 4937–4947. doi: 10.11999/JEIT250981. LIAO Diling, LAI Tao, HUANG Haifeng, WANG Qingsong. LightMamba: A Lightweight Mamba Network for the Joint Classification of HSI and LiDAR Data[J]. Journal of Electronics & Information Technology, 2025, 47(12): 4937–4947. doi: 10.11999/JEIT250981.
[3]	刁文辉, 龚铄, 辛林霖, 申志平, 孙超. 针对多模态遥感数据的自监督策略模型预训练方法[J]. 电子与信息学报, 2025, 47(6): 1658–1668. doi: 10.11999/JEIT241016. DIAO Wenhui, GONG Shuo, XIN Linlin, SHEN Zhiping, SUN Chao. A Model Pre-training Method with Self-Supervised Strategies for Multimodal Remote Sensing Data[J]. Journal of Electronics & Information Technology, 2025, 47(6): 1658–1668. doi: 10.11999/JEIT241016.
[4]	LUO Fulin, HUA Yiyan, FU Chuan, et al. MMD-MLP: LiDAR-guided hyperspectral data classification using local-global directional-MLP with multiresolution multiscale representation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 5508414. doi: 10.1109/TGRS.2025.3550370.
[5]	DUAN Puhong, LUO Yichen, KANG Xudong, et al. LaMamba: Linear attention mamba for hyperspectral image denoising[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 5527113. doi: 10.1109/TGRS.2025.3613739.
[6]	FU Chuan, DU Bo, and ZHANG Liangpei. ReSC-net: Hyperspectral image classification based on attention-enhanced residual module and spatial-channel attention[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5518615. doi: 10.1109/TGRS.2024.3402364.
[7]	YU Chunyan, WANG Hande, SONG Meiping, et al. Interactive graph-based distillation integrated meta-learning network for hyperspectral image incremental classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2026, 64: 5500316. doi: 10.1109/TGRS.2025.3647656.
[8]	DONG Wenqian, YANG Teng, QU Jiahui, et al. Joint contextual representation model-informed interpretable network with dictionary aligning for hyperspectral and LiDAR classification[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(11): 6804–6818. doi: 10.1109/TCSVT.2023.3268757.
[9]	YANG J X, ZHOU Jun, WANG Jing, et al. LiDAR-guided cross-attention fusion for hyperspectral band selection and image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5515815. doi: 10.1109/TGRS.2024.3389651.
[10]	YANG Bin, WANG Xuan, XING Ying, et al. Modality fusion vision transformer for hyperspectral and LiDAR data collaborative classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2024, 17: 17052–17065. doi: 10.1109/JSTARS.2024.3415729.
[11]	HE Ziping, ZHU Qianglin, WANG Wei, et al. Multilevel fusion network based on mix hybrid attention for hyperspectral and LiDAR image classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2026, 19: 470–483. doi: 10.1109/JSTARS.2025.3628896.
[12]	WANG Minhui, SUN Yaxiu, XIANG Jianhong, et al. Joint classification of hyperspectral and LiDAR data based on adaptive gating mechanism and learnable transformer[J]. Remote Sensing, 2024, 16(6): 1080. doi: 10.3390/rs16061080.
[13]	WANG Haoyu, CHENG Yuhu, LIU Xiaomin, et al. Reinforcement learning based Markov edge decoupled fusion network for fusion classification of hyperspectral and LiDAR[J]. IEEE Transactions on Multimedia, 2024, 26: 7174–7187. doi: 10.1109/TMM.2024.3360717.
[14]	SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms[J]. arXiv preprint arXiv: 1707.06347, 2017. doi: 10.48550/arXiv.1707.06347.
[15]	DEBES C, MERENTITIS A, HEREMANS R, et al. Hyperspectral and LiDAR data fusion: Outcome of the 2013 GRSS data fusion contest[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6): 2405–2418. doi: 10.1109/JSTARS.2014.2305441.
[16]	RASTI B, GHAMISI P, and GLOAGUEN R. Hyperspectral and LiDAR fusion using extinction profiles and total variation component analysis[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7): 3997–4007. doi: 10.1109/TGRS.2017.2686450.
[17]	ZHANG Mengmeng, LI Wei, TAO Ran, et al. Information fusion for classification of hyperspectral and LiDAR data using IP-CNN[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5506812. doi: 10.1109/TGRS.2021.3093334.
[18]	CHANG Honghao, BI Haixia, LI Fan, et al. Deep symmetric fusion transformer for multimodal remote sensing data classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5644115. doi: 10.1109/TGRS.2024.3476975.
[19]	NI Kang, XIE Yunan, ZHAO Guofeng, et al. Coarse-to-fine high-order network for hyperspectral and LiDAR classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 5509716. doi: 10.1109/TGRS.2025.3554802.
[20]	LIU Hui, HUANG Chenjia, XIE Tao, et al. Positive matching benefits fusion: A novel contrastive learning framework for hyperspectral and LiDAR data classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2026, 64: 5502218. doi: 10.1109/TGRS.2026.3654168.
[21]	WANG Xianghai, SONG Liyang, FENG Yining, et al. S3F2Net: Spatial-spectral-structural feature fusion network for hyperspectral image and LiDAR data classification[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2025, 35(5): 4801–4815. doi: 10.1109/TCSVT.2025.3525734.
[22]	SHI Lulu, LI Chunchao, ZENG Zhengchao, et al. Masked self-attention fusion network for joint classification of hyperspectral and LiDAR data[J]. IEEE Transactions on Image Processing, 2026, 35: 346–360. doi: 10.1109/TIP.2025.3648926.