LightMamba：一种轻量级Mamba用于高光谱图形和激光雷达数据联合分类网络

廖帝灵; 赖涛; 黄海风; 王青松

doi:10.11999/JEIT250981

LightMamba：一种轻量级Mamba用于高光谱图形和激光雷达数据联合分类网络

doi: 10.11999/JEIT250981 cstr: 32379.14.JEIT250981

中山大学电子与通信工程学院深圳 518107

基金项目: 国家自然科学基金(62273365)，“小米青年学者”项目

详细信息

作者简介:
廖帝灵：男，博士，研究方向为多模态图像融合深度学习、高光谱图像处理、遥感场景图像分类

赖涛：男，副教授，研究方向为超宽带轨道式SAR成像雷达系统设计与信息处理、地基形变监测SAR系统设计与信息处理

黄海风：男，教授，研究方向空间电子学和智能传感领域的基础理论及关键技术

王青松：男，教授，研究方向为遥感图像精化处理、协同探测感知与信息融合

通讯作者:
王青松　wangqs5@mail.sysu.edu.cn

1¹⁾https://www.grss-ieee.org/technical-committees/image-analysis-and-data-fusion/?tab=past-data-fusion-contests.²⁾https://dataserv.ub.tum.de/index.php/s/m1657312.
中图分类号: TN911.73; TP751
计量
- 文章访问数: 71
- HTML全文浏览量: 35
- PDF下载量: 6
- 被引次数: 0
出版历程
- 收稿日期: 2025-09-25
- 修回日期: 2025-12-30
- 录用日期: 2025-12-30
- 网络出版日期: 2026-01-09

LightMamba: A Lightweight Mamba Network for the Joint Classification of HSI and LiDAR Data

College of Electronic and Communication Engineering, Sun Yat-sen University, Shenzhen 518107, China

Funds: The National Natural Science Foundation of China (62273365), Xiaomi Young Talents Program

摘要

摘要: 高光谱图像(HSI)和激光雷达(LiDAR)数据的联合分类是遥感领域的一项关键任务，它通过融合丰富的光谱信息和精确的三维结构信息，显著提升了对地物识别的精度。然而，现有的基于深度学习(DL)的联合分类方法依然受限于高模型计算复杂度。因此，该文提出一种新颖的轻量级Mamba网络。该网络的核心是引入了先进的状态空间模型(SSM)，其线性计算复杂度特性使其能够高效地建模遥感数据中的长距离上下文依赖关系。首先，多源对齐模块被用于对异构的HSI和LiDAR数据进行特征提取与空间-光谱维对齐，以提供一致的特征表示；其次，多源轻量Mamba模块以LiDAR的高程信息作为引导，采用轻量化设计融合双流序列，高效建模长距离依赖；最后，设计了一种基于MLP的分类器，并输出分类结果。在多个公开基准数据集上的实验结果表明，与当前先进方法相比，LightMamba在分类精度上取得了显著提升，同时保持了更低的计算复杂度，证明了基于Mamba的架构在遥感多源数据融合与分类任务中的巨大潜力。LightMamba的代码可访问https://www.scidb.cn/detail?dataSetId=064dc4ac5350418e87a8b82dd324737b&version=V1&code=j00173。
- 高光谱图像分类 /
- 多源数据融合 /
- 状态空间模型 /
- 轻量级网络 /
- Mamba
Abstract: Objective The joint classification of HyperSpectral Imagery (HSI) and Light Detection And Ranging (LiDAR) data is a critical task in remote sensing, where complementary spectral and spatial information is exploited to improve land cover recognition accuracy. However, mainstream deep learning approaches, particularly those based on Convolutional Neural Networks (CNNs) and Transformers, are constrained by high computational cost and limited efficiency in modeling long-range dependencies. CNN-based methods are effective for local feature extraction but suffer from limited receptive fields and increased parameter counts when scaled. Transformer architectures provide global context modeling but incur quadratic computational complexity due to self-attention mechanisms, which leads to prohibitive costs when processing high-dimensional remote sensing data. To address these limitations, a lightweight network architecture named LightMamba is proposed. The model leverages an advanced State Space Model (SSM) to achieve efficient and accurate joint classification of HSI and LiDAR data. The objective is to maintain linear computational complexity while effectively fusing multi-source features and capturing global contextual relationships, thereby supporting resource-constrained applications without accuracy degradation. Methods The proposed LightMamba framework consists of three core components. First, a MultiSource Alignment Module (MSAM) is designed to address heterogeneity between HSI and LiDAR data. A dual-branch network with shared weights projects both modalities into a unified feature space, which ensures consistent spatial-spectral representation. This shared-weight strategy reduces the parameter count and strengthens inter-modal correlation through the learning of common foundational features. Second, the Multi-Source Lightweight Mamba Module (MSLMM) forms the core of the framework. Aligned HSI and LiDAR feature sequences are processed using a parameter-efficient Mamba architecture. A hybrid parameter-sharing strategy is adopted by combining shared matrices with modality-specific parameters, which preserves discriminative capability while reducing redundancy. LiDAR elevation information is used as a positional guide to enhance spatial awareness during feature fusion. The selective scanning mechanism of the SSM enables efficient modeling of long-range dependencies with linear complexity, thereby avoiding the quadratic cost associated with Transformers. Spectral bands are processed sequentially to preserve joint spectral spatial characteristics. Finally, a MultiLayer Perceptron (MLP)-based classifier maps fused high-level features to class probabilities with low computational overhead. The model is trained end to end using cross-entropy loss. Evaluations are conducted on two public benchmarks, namely the Houston and Augsburg datasets. Comparisons are performed against representative methods, including CoupledCNN, GAMF, HCT, MFT, Cross-HL, and S2CrossMamba, using Overall Accuracy (OA), Average Accuracy (AA), and the Kappa coefficient. Ablation experiments analyze the contribution of each module, and parameter count and FLoating-Point Operations (FLOPs) are reported. Results and Discussions Experimental results demonstrate that LightMamba achieves superior performance and efficiency. On the Houston dataset, an OA of 94.30%, an AA of 95.25%, and a Kappa coefficient of 93.83% are obtained, which exceed those of all comparison methods. Perfect classification accuracy is achieved for several classes, including Soil and Water. Classification maps exhibit improved spatial continuity and internal consistency, with reduced speckle noise, particularly in heterogeneous regions such as commercial areas. On the Augsburg dataset, LightMamba achieves the highest OA of 87.41% and a Kappa coefficient of 82.30%, which confirms strong generalization across different scenes. Although the AA is slightly lower than that of S2CrossMamba, the higher OA and Kappa values indicate better overall performance. Complexity analysis shows that LightMamba attains high accuracy with a lightweight structure containing only 69.93 k parameters, which is substantially fewer than GAMF and comparable to S2CrossMamba, while maintaining moderate FLOPs. Experiments on input patch size indicate adaptability to scene characteristics, with optimal performance observed at 17×17 for the Houston dataset and 9×9 for the Augsburg dataset. Conclusions A lightweight network architecture, LightMamba, is presented for joint HSI and LiDAR classification. By combining a shared-weight MSAM with a lightweight Mamba module that adopts hybrid parameterization and elevation-guided fusion, modal heterogeneity is effectively addressed and long-range contextual dependencies are captured with linear computational complexity. Experimental results on public benchmarks demonstrate state-of-the-art classification accuracy with a reduced parameter count and computational cost compared with existing methods. These findings confirm the potential of Mamba-based architectures for efficient multi-source remote sensing data fusion. Future research will explore optimized two-dimensional scanning mechanisms and adaptive scanning strategies to further improve feature capture efficiency and classification performance. The LightMamba code is available at https://www.scidb.cn/detail?dataSetId=064dc4ac5350418e87a8b82dd324737b&version=V1&code=j00173.
- Hyperspectral image classification /
- Multi-source data fusion /
- State space model /
- Lightweight network /
- Mamba

HTML全文

图 1 LightMamba网络结构

下载: 全尺寸图片幻灯片

图 2 基于多源权重共享的多源对齐模块结构

下载: 全尺寸图片幻灯片

图 3 详细流程图($ \varDelta $和$ \varDelta ' $分别表示HSI和LiDAR数据的离散步长)

下载: 全尺寸图片幻灯片

图 4 Houston 2013数据集上不同方法的分类图

下载: 全尺寸图片幻灯片

图 5 Augsburg数据集上不同方法的分类图

下载: 全尺寸图片幻灯片

图 6 不同方法准确率、模型复杂度和计算效率的比较

下载: 全尺寸图片幻灯片

图 7 不同方法输入空间大小对性能的影响

下载: 全尺寸图片幻灯片

表 1 两种数据集详细信息

类别	Houston			Augsburg
类别	名称	训练	测试	名称	训练	测试
C1	健康草地	20	1231	林地	20	13487
C2	受胁迫草地	20	1234	住宅区	20	30309
C3	人造草地	20	677	工业区	20	3831
C4	树木	20	1224	低矮植被	20	26837
C5	土壤	20	1222	社区园地	20	555
C6	水体	20	305	商业区	20	1625
C7	住宅区	20	1248	水体	20	1510
C8	商业区	20	1224
C9	普通道路	20	1232
C10	高速公路	20	1207
C11	铁路	20	1215
C12	停车场1	20	1213
C13	停车场2	20	449
C14	网球场	20	408
C15	塑胶跑道	20	640
-	总计	300	14729	总计	140	78154

下载: 导出CSV

表 2 LightMamba方法中不同模块对分类精度OA值的影响

情况	组成成分			Houston	Augsburg
情况	MSAM	1个MSLMM	2个MSLMM	OA(%)	OA(%)
1	√			81.28	75.96
2	√	√		93.86	85.50
3	√		√	94.30	87.41

下载: 导出CSV

表 3 对Houston数据集采用不同方法获得的分类结果进行比较(最佳结果以粗体显示)(%)

类别	HSI+LiDAR						本文方法
类别	CoupledCNN	GAMF	HCT	MFT	Cross-HL	S²CrossMamba	LightMamba
C1	93.01	99.59	92.93	96.91	97.97	90.75	94.55
C2	96.68	97.33	98.62	94.97	89.87	98.68	93.67
C3	99.85	99.85	99.85	100	98.82	99.69	99.11
C4	88.23	93.87	99.34	91.33	99.75	96.01	94.52
C5	95.49	99.67	99.67	97.70	100	99.75	100
C6	97.70	92.13	98.68	100	94.09	100	100
C7	81.33	81.41	98.71	76.84	91.51	39.49	91.34
C8	83.00	85.94	76.87	71.32	77.04	96.84	86.68
C9	70.86	79.38	76.13	82.22	85.95	28.05	90.58
C10	80.19	74.15	68.68	94.03	97.10	74.22	94.11
C11	89.62	87.48	94.89	89.95	81.64	88.20	97.36
C12	79.22	87.88	46.66	92.82	92.91	44.42	90.51
C13	82.20	93.76	97.55	98.44	95.32	100	96.21
C14	100	99.26	100	99.26	100	99.74	100
C15	98.90	100	99.37	91.09	100	80.32	100
OA	87.83	90.21	87.67	90.22	92.50	78.74	94.30
AA	89.76	91.45	89.87	91.80	93.47	82.41	95.25
K×100	86.86	89.41	86.69	89.43	91.90	77.23	93.83

下载: 导出CSV

表 4 对Augsburg数据集采用不同方法获得的分类结果进行比较(最佳结果以粗体显示)(%)

类别	HSI+LiDAR						本文方法
类别	CoupledCNN	GAMF	HCT	MFT	Cross-HL	S²CrossMamba	LightMamba
C1	96.57	94.52	95.69	92.17	96.36	96.04	92.17
C2	78.62	64.50	83.65	73.80	85.78	92.43	93.07
C3	66.56	41.00	72.95	40.17	71.86	85.66	62.85
C4	90.06	86.30	85.82	59.16	80.81	78.68	85.21
C5	89.36	98.01	96.93	64.86	69.57	90.99	87.74
C6	40.86	78.15	30.46	64.86	38.21	52.12	48.12
C7	75.76	65.43	80.66	63.97	57.08	58.54	74.43
OA	84.30	76.56	84.88	69.86	83.33	86.50	87.41
AA	76.83	75.42	78.03	65.57	66.67	79.21	77.66
K×100	78.58	69.09	79.36	60.33	76.57	81.31	82.30

下载: 导出CSV

参考文献(25)

[1]	FAUVEL M, TARABALKA Y, BENEDIKTSSON J A, et al. Advances in spectral-spatial classification of hyperspectral images[J]. Proceedings of the IEEE, 2013, 101(3): 652–675. doi: 10.1109/jproc.2012.2197589.
[2]	ZHANG Xia, SUN Yanli, SHANG Kun, et al. Crop classification based on feature band set construction and object-oriented approach using hyperspectral images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2016, 9(9): 4117–4128. doi: 10.1109/jstars.2016.2577339.
[3]	GHAMISI P, PLAZA J, CHEN Yushi, et al. Advanced spectral classifiers for hyperspectral images: A review[J]. IEEE Geoscience and Remote Sensing Magazine, 2017, 5(1): 8–32. doi: 10.1109/mgrs.2016.2616418.
[4]	PRICOPE N G, HALLS J N, DALTON E G, et al. Precision mapping of coastal wetlands: An integrated remote sensing approach using unoccupied aerial systems light detection and ranging and multispectral data[J]. Journal of Remote Sensing, 2024, 4: 0169. doi: 10.34133/remotesensing.0169.
[5]	SHI Cuiping, LIAO Diling, ZHANG Tianyu, et al. Hyperspectral image classification based on expansion convolution network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5528316. doi: 10.1109/tgrs.2022.3174015.
[6]	LIU Wenping, ZHANG Yuxiang, and DONG Yanni. Multifeature collaborative attention dynamic hypergraph convolutional network for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 5522115. doi: 10.1109/tgrs.2025.3598375.
[7]	LIU Quanyong, PENG Jiangtao, ZHANG Genwei, et al. Deep contrastive learning network for small-sample hyperspectral image classification[J]. Journal of Remote Sensing, 2023, 3: 0025. doi: 10.34133/remotesensing.0025.
[8]	ZHAO Xudong, TAO Ran, LI Wei, et al. Joint classification of hyperspectral and LiDAR data using hierarchical random walk and deep CNN architecture[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(10): 7355–7370. doi: 10.1109/tgrs.2020.2982064.
[9]	LI Zhi, ZHENG Ke, GAO Lianru, et al. Feature reconstruction guided fusion network for hyperspectral and LiDAR classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 4408914. doi: 10.1109/tgrs.2025.3562246.
[10]	GAO Hongmin, YANG Yao, LI Chenming, et al. Multiscale residual network with mixed depthwise convolution for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(4): 3396–3408. doi: 10.1109/tgrs.2020.3008286.
[11]	HONG Danfeng, HAN Zhu, YAO Jing, et al. SpectralFormer: Rethinking hyperspectral image classification with transformers[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5518615. doi: 10.1109/tgrs.2021.3130716.
[12]	WANG Minhui, SUN Yaxiu, XIANG Jianhong, et al. CITNet: Convolution interaction transformer network for hyperspectral and LiDAR image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5535918. doi: 10.1109/tgrs.2024.3477965.
[13]	PAN Yukai, WU Nan, and JIN Wei. Multimodal feature disentangle-fusion network for hyperspectral and LiDAR data classification[J]. IEEE Geoscience and Remote Sensing Letters, 2024, 21: 5510905. doi: 10.1109/lgrs.2024.3492252.
[14]	ZHU Fei, SHI Cuiping, SHI Kaijie, et al. Joint classification of hyperspectral and LiDAR data using hierarchical multimodal feature aggregation-based multihead axial attention transformer[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 5503817. doi: 10.1109/tgrs.2025.3533475.
[15]	GU A, GOEL K, and RÉ C. Efficiently modeling long sequences with structured state spaces[C]. Proceedings of the 10th International Conference on Learning Representations, 2022.
[16]	HE Yan, TU Bingtu, LIU Bo, et al. HSI-MFormer: Integrating mamba and transformer experts for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 5621916. doi: 10.1109/tgrs.2025.3564167.
[17]	ZHUANG Peixian, ZHANG Xiaochen, WANG Hao, et al. FAHM: Frequency-aware hierarchical mamba for hyperspectral image classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2025, 18: 6299–6313. doi: 10.1109/jstars.2025.3539791.
[18]	LIAO Diling, WANG Qingsong, LAI Tao, et al. Joint classification of hyperspectral and LiDAR data based on mamba[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5530915. doi: 10.1109/tgrs.2024.3459709.
[19]	HE Yan, TU Bing, JIANG Puzhao, et al. Classification of multisource remote sensing data using slice mamba[J]. IEEE Transactions on Geoscience and Remote Sensing, 2025, 63: 5505414. doi: 10.1109/tgrs.2025.3538553.
[20]	ZHANG Guanglian, ZHANG Zhanxu, DENG Jiangwei, et al. S²CrossMamba: Spatial–spectral cross-mamba for multimodal remote sensing image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2024, 21: 5510705. doi: 10.1109/lgrs.2024.3488036.
[21]	HANG Renlong, LI Zhu, GHAMISI P, et al. Classification of hyperspectral and LiDAR data using coupled CNNs[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(7): 4939–4950. doi: 10.1109/tgrs.2020.2969024.
[22]	CAI Jianghui, ZHANG Min, YANG Haifeng, et al. A novel graph-attention based multimodal fusion network for joint classification of hyperspectral image and LiDAR data[J]. Expert Systems with Applications, 2024, 249: 123587. doi: 10.1016/j.eswa.2024.123587.
[23]	ZHAO Guangrui, YE Qiaolin, SUN Len, et al. Joint classification of hyperspectral and LiDAR data using a hierarchical CNN and transformer[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 5500716. doi: 10.1109/tgrs.2022.3232498.
[24]	HOFFMANN D S, CLASEN K N, and DEMIR B. Transformer-based multi-modal learning for multi-label remote sensing image classification[C]. IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium, Pasadena, USA, 2023: 4891–4894. doi: 10.1109/igarss52108.2023.10281927.
[25]	ROY S K, SUKUL A, JAMALI A, et al. Cross hyperspectral and LiDAR attention transformer: An extended self-attention for land use and land cover classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 5512815. doi: 10.1109/tgrs.2024.3374324.