A Modal Fusion Deep Clustering Method for Multi-Sensor Fault Diagnosis of Rotating Machinery

WU Zhangjun; XU Renli; FANG Gang; SHAO Haidong

doi:10.11999/JEIT240648

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 >

WU Zhangjun, XU Renli, FANG Gang, SHAO Haidong. A Modal Fusion Deep Clustering Method for Multi-Sensor Fault Diagnosis of Rotating Machinery[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT240648

Citation:

WU Zhangjun, XU Renli, FANG Gang, SHAO Haidong. A Modal Fusion Deep Clustering Method for Multi-Sensor Fault Diagnosis of Rotating Machinery[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT240648

Citation:

PDF( 4209 KB)

A Modal Fusion Deep Clustering Method for Multi-Sensor Fault Diagnosis of Rotating Machinery

doi: 10.11999/JEIT240648

WU Zhangjun^{1, 2},
XU Renli^{1, 3},
FANG Gang^{1, 3},
SHAO Haidong^{4
,
,}

1.
School of Management, Hefei University of Technology, Hefei 230009, China
2.
Key Laboratory of Process Optimization and Intelligent Decision-making, Ministry of Education, Hefei 230009, China
3.
Ministry of Education Engineering Research Center for Intelligent Decision-Making & Information System Technologies, Hefei 230009, China
4.
College of Mechanical and Vehicle Engineering, Hunan University, Changsha 410082, China

Funds: The National Natural Science Foundation of China (52275104), Hunan Province Innovation Platform and Talent Plan Project (2023RC3097)

Received Date: 2024-07-25
Rev Recd Date: 2024-12-03

Available Online: 2024-12-06

Abstract

Abstract

Objective Rotating machinery is essential across various industrial sectors, including energy, aerospace, and manufacturing. However, these machines operate under complex and variable conditions, making timely and accurate fault detection a significant challenge. Traditional diagnostic methods, which use a single sensor and modality, often miss critical features, particularly subtle fault signatures. This can result in reduced reliability, increased downtime, and higher maintenance costs. To address these issues, this study proposes a novel modal fusion deep clustering approach for multi-sensor fault diagnosis in rotating machinery. The main objectives are to: (1) improve feature extraction through time-frequency transformations that reveal important temporal-spectral patterns, (2) implement an attention-based modality fusion strategy that integrates complementary information from various sensors, and (3) use a deep clustering framework to identify fault types without needing labeled training data. Methods The proposed approach utilizes a multi-stage pipeline for thorough feature extraction and analysis. First, raw multi-sensor signals, such as vibration data collected under different load and speed conditions, are preprocessed and transformed with the Short-Time Fourier Transform (STFT). This converts time-domain signals into time-frequency representations, highlighting distinct frequency components related to various fault conditions. Next, Gated Recurrent Units (GRUs) model temporal dependencies and capture long-range correlations, while Convolutional Autoencoders (CAEs) learn hierarchical spatial features from the transformed data. By combining GRUs and CAEs, the framework encodes both temporal and structural patterns, creating richer and more robust representations than traditional methods that rely solely on either technique or handcrafted features. A key innovation is the modality fusion attention mechanism. In multi-sensor environments, individual sensors typically capture complementary aspects of system behavior. Simply concatenating their outputs can lead to suboptimal results due to noise and irrelevant information. The proposed attention-based fusion calculates modality-specific affinity matrices to assess the relationship and importance of each sensor modality. With learnable attention weights, the framework prioritizes the most informative modalities while diminishing the impact of less relevant ones. This ensures the fused representation captures complementary information, resulting in improved discriminative power. Finally, an unsupervised clustering module is integrated into the deep learning pipeline. Rather than depending on labeled data, the model assigns samples to clusters by refining cluster assignments iteratively using a Kullback-Leibler (KL) divergence-based objective. Initially, a soft cluster distribution is created from the learned features. A target distribution is then computed to sharpen and define cluster boundaries. By continuously minimizing the KL divergence between these distributions, the model self-optimizes over time, producing well-separated clusters corresponding to distinct fault types without supervision. Results and Discussions The proposed approach’s effectiveness is illustrated using multi-sensor bearing and gearbox datasets. Compared to conventional unsupervised methods—like traditional clustering algorithms or single-domain feature extraction techniques—this framework significantly enhances clustering accuracy and fault recognition rates. Experimental results show recognition accuracies of approximately 99.16% on gearbox data and 98.63% on bearing data, representing a notable advancement over existing state-of-the-art techniques. These impressive results stem from the synergistic effects of advanced feature extraction, modality fusion, and iterative clustering refinement. By extracting time-frequency features through STFT, the method captures a richer representation than relying solely on raw time-domain signals. The use of GRUs incorporates temporal information, enabling the capture of dynamic signal changes that may indicate evolving fault patterns. Additionally, CAEs reveal meaningful spatial structures from time-frequency data, resulting in low-dimensional yet highly informative embeddings. The modality fusion attention mechanism further enhances these benefits by emphasizing relevant modalities, such as vibration data from various sensor placements or distinct physical principles, thus leveraging their complementary strengths. Through the iterative minimization of KL divergence, the clustering process becomes more discriminative. Initially broad and overlapping cluster boundaries are progressively refined, allowing the model to converge toward stable and well-defined fault groupings. This unsupervised approach is particularly valuable in practical scenarios, where obtaining labeled data is costly and time-consuming. The model’s ability to learn directly from unlabeled signals enables continuous monitoring and adaptation, facilitating timely interventions and reducing the risk of unexpected machine failures. The discussion emphasizes the adaptability of the proposed method. Industrial systems continuously evolve, and fault patterns can change over time due to aging, maintenance, or shifts in operational conditions. The unsupervised method can be periodically retrained or updated with new unlabeled data. This allows it to monitor changes in machinery health and quickly detect new fault conditions without the need for manual annotation. Additionally, the attention-based modality fusion is flexible enough to support the inclusion of new sensor types or measurement channels, potentially enhancing diagnostic performance as richer data sources become available. Conclusions This study presents a modal fusion deep clustering framework designed for the multi-sensor fault diagnosis of rotating machinery. By combining time-frequency transformations with GRU- and CAE-based deep feature encoders, attention-driven modality fusion, and KL divergence-based unsupervised clustering, this approach outperforms traditional methods in accuracy, robustness, and scalability. Key contributions include a comprehensive multi-domain feature extraction pipeline, an adaptive modality fusion strategy for heterogeneous sensor data integration, and a refined deep clustering mechanism that achieves high diagnostic accuracy without relying on labeled training samples. Looking ahead, there are several promising directions. Adding more modalities—like acoustic emissions, temperature signals, or electrical measurements—could lead to richer feature sets. Exploring semi-supervised or few-shot extensions may further enhance performance by utilizing minimal labeled guidance when available. Implementing the proposed model in an industrial setting, potentially for real-time use, would also validate its practical benefits for maintenance decision-making, helping to reduce operational costs and extend equipment life.
- Rotating machinery,
- Fault diagnosis,
- Multimodal fusion,
- Deep clustering

FullText(HTML)

References(28)

References

[1]	王玉静, 康守强, 张云, 等. 基于集合经验模态分解敏感固有模态函数选择算法的滚动轴承状态识别方法[J]. 电子与信息学报, 2014, 36(3): 595–600. doi: 10.3724/SP.J.1146.2013.00434. WANG Yujing, KANG Shouqiang, ZHANG Yun, et al. Condition recognition method of rolling bearing based on ensemble empirical mode decomposition sensitive intrinsic mode function selection algorithm[J]. Journal of Electronics & Information Technology, 2014, 36(3): 595–600. doi: 10.3724/SP.J.1146.2013.00434.
[2]	文成林, 吕菲亚. 基于深度学习的故障诊断方法综述[J]. 电子与信息学报, 2020, 42(1): 234–248. doi: 10.11999/JEIT190715. WEN Chenglin and LÜ Feiya. Review on deep learning based fault diagnosis[J]. Journal of Electronics & Information Technology, 2020, 42(1): 234–248. doi: 10.11999/JEIT190715.
[3]	邵海东, 肖一鸣, 邓乾旺, 等. 基于不确定性感知网络的可信机械故障诊断[J]. 机械工程学报, 2024, 60(12): 194–206. doi: 10.3901/JME.2024.12.194. SHAO Haidong, XIAO Yiming, DENG Qianwang, et al. Trustworthy mechanical fault diagnosis using uncertainty-aware network[J]. Journal of Mechanical Engineering, 2024, 60(12): 194–206. doi: 10.3901/JME.2024.12.194.
[4]	康守强, 杨佳轩, 王玉静, 等. 基于改进宽度模型迁移学习的不同负载下滚动轴承状态快速分类方法[J]. 电子与信息学报, 2023, 45(5): 1824–1832. doi: 10.11999/JEIT220401. KANG Shouqiang, YANG Jiaxuan, WANG Yujing, et al. A fast classification method of rolling bearing state under different loads based on improved broad model transfer learning[J]. Journal of Electronics & Information Technology, 2023, 45(5): 1824–1832. doi: 10.11999/JEIT220401.
[5]	邵海东, 颜深, 肖一鸣, 等. 时变转速下基于改进图注意力网络的轴承半监督故障诊断[J]. 电子与信息学报, 2023, 45(5): 1550–1558. doi: 10.11999/JEIT220303. SHAO Haidong, YAN Shen, XIAO Yiming, et al. Semi-supervised bearing fault diagnosis using improved graph attention network under time-varying speeds[J]. Journal of Electronics & Information Technology, 2023, 45(5): 1550–1558. doi: 10.11999/JEIT220303.
[6]	孙瑾铃, 张伟涛, 楼顺天. 基于等变化自适应源分离算法的滚动轴承故障信号自适应盲提取[J]. 电子与信息学报, 2020, 42(10): 2471–2477. doi: 10.11999/JEJT190722. SUN Jinling, ZHANG Weitao, and LOU Shuntian. Adaptive blind extraction of rolling bearing fault signal based on equivariant adaptive separation via independence[J]. Journal of Electronics & Information Technology, 2020, 42(10): 2471–2477. doi: 10.11999/JEJT190722.
[7]	邵海东, 林健, 闵志闪, 等. 分布外样本干扰下基于改进半监督原型网络的齿轮箱跨域故障诊断[J]. 机械工程学报, 2024, 60(4): 212–221. doi: 10.3901/JME.2024.04.212. SHAO Haidong, LIN Jian, MIN Zhishan, et al. Improved semi-supervised prototype network for cross-domain fault diagnosis of gearbox under out-of-distribution interference samples[J]. Journal of Mechanical Engineering, 2024, 60(4): 212–221. doi: 10.3901/JME.2024.04.212.
[8]	XIAO Yiming, SHAO Haidong, HAN Songyu, et al. Novel joint transfer network for unsupervised bearing fault diagnosis from simulation domain to experimental domain[J]. IEEE/ASME Transactions on Mechatronics, 2022, 27(6): 5254–5263. doi: 10.1109/TMECH.2022.3177174.
[9]	解骞, 徐浩岚, 王彤, 等. 基于自主认知深度时间聚类表示的隔离开关故障诊断方法[J]. 电气工程学报, 2024, 19(1): 281–289. doi: 10.11985/2024.01.030. XIE Qian, XU Haolan, WANG Tong, et al. Disconnector fault diagnosis method based on autonomous-cognition deep temporal clustering representation[J]. Journal of Electrical Engineering, 2024, 19(1): 281–289. doi: 10.11985/2024.01.030.
[10]	LI Xiang, LI Xu, and MA Hui. Deep representation clustering-based fault diagnosis method with unsupervised data applied to rotating machinery[J]. Mechanical Systems and Signal Processing, 2020, 143: 106825. doi: 10.1016/j.ymssp.2020.106825.
[11]	LIU Yongjie, DING Kun, ZHANG Jingwei, et al. Fault diagnosis approach for photovoltaic array based on the stacked auto-encoder and clustering with I-V curves[J]. Energy Conversion and Management, 2021, 245: 114603. doi: 10.1016/j.enconman.2021.114603.
[12]	YU Jianbo and YAN Xuefeng. Multiscale intelligent fault detection system based on agglomerative hierarchical clustering using stacked denoising autoencoder with temporal information[J]. Applied Soft Computing, 2020, 95: 106525. doi: 10.1016/j.asoc.2020.106525.
[13]	ZHAO Bo, ZHANG Xianmin, WU Qiqiang, et al. A novel unsupervised directed hierarchical graph network with clustering representation for intelligent fault diagnosis of machines[J]. Mechanical Systems and Signal Processing, 2023, 183: 109615. doi: 10.1016/j.ymssp.2022.109615.
[14]	REN Yazhou, PU Jingyu, YANG Zhimeng, et al. Deep clustering: A comprehensive survey[EB/OL]. https://arxiv.org/abs/2210.04142, 2022.
[15]	IKOTUN A M, EZUGWU A E, ABUALIGAH L, et al. K-means clustering algorithms: A comprehensive review, variants analysis, and advances in the era of big data[J]. Information Sciences, 2023, 622: 178–210. doi: 10.1016/j.ins.2022.11.139.
[16]	XIE Junyuan, GIRSHICK R, and FARHADI A. Unsupervised deep embedding for clustering analysis[C]. Proceedings of the 33rd International Conference on Machine Learning, New York City, USA, 2016: 478–487.
[17]	GUO Xifeng, GAO Long, LIU Xingwang, et al. Improved deep embedded clustering with local structure preservation[C]. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, Melbourne, Australia, 2017: 1753–1759. doi: 10.24963/ijcai.2017/243.
[18]	WU Zhangjun, FANG Gang, WANG Yifei, et al. An end-to-end deep clustering method with consistency and complementarity attention mechanism for multisensor fault diagnosis[J]. Applied Soft Computing, 2024, 158: 111594. doi: 10.1016/j.asoc.2024.111594.
[19]	WANG Kejun, WANG Wenqing, ZHAO Yabo, et al. Multisensor fault diagnosis via Markov chain and evidence theory[J]. Engineering Applications of Artificial Intelligence, 2023, 126: 106851. doi: 10.1016/j.engappai.2023.106851.
[20]	MAN Jie, DONG Honghui, JIA Limin, et al. AttGGCN model: A novel multi-sensor fault diagnosis method for high-speed train bogie[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(10): 19511–19522. doi: 10.1109/TITS.2022.3156281.
[21]	TONG Jinyu, LIU Cang, ZHENG Jinde, et al. Multi-sensor information fusion and coordinate attention-based fault diagnosis method and its interpretability research[J]. Engineering Applications of Artificial Intelligence, 2023, 124: 106614. doi: 10.1016/j.engappai.2023.106614.
[22]	CUI Jian, XIE Ping, WANG Xiao, et al. M2FN: An end-to-end multi-task and multi-sensor fusion network for intelligent fault diagnosis[J]. Measurement, 2022, 204: 112085. doi: 10.1016/j.measurement.2022.112085.
[23]	XU Yadong, FENG Ke, YAN Xiaoan, et al. CFCNN: A novel convolutional fusion framework for collaborative fault identification of rotating machinery[J]. Information Fusion, 2023, 95: 1–16. doi: 10.1016/j.inffus.2023.02.012.
[24]	YANG Chaoying, LIU Jie, ZHOU Kaibo, et al. Semisupervised machine fault diagnosis fusing unsupervised graph contrastive learning[J]. IEEE Transactions on Industrial Informatics, 2023, 19(8): 8644–8653. doi: 10.1109/TII.2022.3220847.
[25]	WANG Daichao, LI Yibin, JIA Lei, et al. Novel three-stage feature fusion method of multimodal data for bearing fault diagnosis[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 3514710. doi: 10.1109/TIM.2021.3071232.
[26]	MA Meng, SUN Chuang, and CHEN Xuefeng. Deep coupling autoencoder for fault diagnosis with multimodal sensory data[J]. IEEE Transactions on Industrial Informatics, 2018, 14(3): 1137–1145. doi: 10.1109/TII.2018.2793246.
[27]	CHE Changchang, WANG Huawei, NI Xiaomei, et al. Hybrid multimodal fusion with deep learning for rolling bearing fault diagnosis[J]. Measurement, 2021, 173: 108655. doi: 10.1016/j.measurement.2020.108655.
[28]	HU Zhanxuan, WANG Yichen, NING Hailong, et al. Mutual-taught deep clustering[J]. Knowledge-Based Systems, 2023, 282: 111100. doi: 10.1016/j.knosys.2023.111100.