2024, 46(9): 3693-3702.
doi: 10.11999/JEIT240178
Abstract:
The multimodal fusion method can effectively improve the ground object classification accuracy by using the complementary characteristics of different modalities, which has become a research hotspot in various fields in recent years. The existing multimodal fusion methods have been successfully applied to multi-source remote sensing classification tasks oriented to HyperSpectral Image (HSI) and Light Detection And Ranging (LiDAR). However, existing research still faces many challenges, including difficulty in capturing spatial dependencies among irregular ground objects and obtaining discriminative information in multimodal data. To address the above challenges, a Scale Adaptive Fusion Network (SAFN)is proposed in this paper, by integrating the fusion of multimodal, multiscale, and multiview features into a unified framework. First, a dynamic multiscale graph module is proposed to capture the complex spatial dependencies of ground object, enhancing the model’s adaptability to irregular and scale-dissimilar ground object. Second, the complementary properties of LiDAR and HSI are utilized to constrain ground object within the same spatial neighborhood to have similar feature representations, thereby acquiring discriminative remote sensing features. Then, a multimodal spatial-spectral graph fusion module is proposed to establish feature interactions among multimodal, multiscale, and multiview features, providing discriminative fusion features for classification tasks by capturing class-recognition information that can be shared among features. Finally, the fusion features are fed into a classifier to obtain class probability scores for predicting the ground object class. To verify the effectiveness of SAFN, experiments are conducted on three datasets (i.e., Houston, Trento, and MUUFL). The experimental results show that, SAFN achieved state-of-the-art performance in multi-source remote sensing data classification tasks when compared with existing mainstream methods.