Multi-constrained Non-negative Matrix Factorization Algorithm Based on Sinkhorn Distance Feature Scaling

LI Songtao; LI Weigang; GAN Pin; JIANG Lin

doi:10.11999/JEIT210946

Volume 44 Issue 12

Dec. 2022

Turn off MathJax

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2022 > 44(12): 4384-4394

LI Songtao, LI Weigang, GAN Pin, JIANG Lin. Multi-constrained Non-negative Matrix Factorization Algorithm Based on Sinkhorn Distance Feature Scaling[J]. Journal of Electronics & Information Technology, 2022, 44(12): 4384-4394. doi: 10.11999/JEIT210946

Citation:

LI Songtao, LI Weigang, GAN Pin, JIANG Lin. Multi-constrained Non-negative Matrix Factorization Algorithm Based on Sinkhorn Distance Feature Scaling[J]. Journal of Electronics & Information Technology, 2022, 44(12): 4384-4394. doi: 10.11999/JEIT210946

Citation:

PDF( 4989 KB)

Multi-constrained Non-negative Matrix Factorization Algorithm Based on Sinkhorn Distance Feature Scaling

doi: 10.11999/JEIT210946

1.
Engineering Research Center for Metallurgical Automation and Measurement Technology, Ministry of Education, Wuhan University of Science and Technology, Wuhan 430081, China
2.
Key Laboratory of Metallurgical Equipment and Control Technology, Ministry of Education, Wuhan University of Science and Technology, Wuhan 430081, China

Funds: The National Key R&D Program (2019YFB1310000), Hubei Province Science and Technology Projects (2020BED003), Hubei Province Key R&D Program (2020BAB098)

Received Date: 2021-09-06
Accepted Date: 2021-11-23
Rev Recd Date: 2021-11-18

Available Online: 2021-11-26

Publish Date: 2022-12-16

Abstract

Abstract

In order to reduce the co-adaptability interference of the original feature to the Non-negative Matrix Factorization (NMF) algorithm and improve the performance of non-negative matrix factorization subspace learning and clustering performance, a novel multi-constrained semi-supervised non-negative matrix factorization algorithm based on Sinkhorn distance feature scaling is proposed. First, the algorithm is feature-scaled by the Sinkhorn distance to the original input matrix to improve the correlation between features of the same type of data in the space, then, the dual graph manifold structure combined with the sample label information and the norm sparsity constraint are embedded in the model as a dual regular term, so that the decomposed base matrix has sparse characteristics and strong spatial expression ability. Finally, the objective function of the proposed algorithm is optimized by Karush-Kuhn-Tucker (KKT) conditions, and effective multiplication update rules are obtained. Through the comparative analysis of the results of multiple clustering experiments on multiple image data sets and translational noise data, the algorithm proposed in this paper has a strong subspace learning ability and is more robust to translational noise.
- Non-negative Matrix Factorization(NMF),
- Feature scaling,
- Subspace manifold regularization,
- Sparse constraint,
- Clustering

FullText(HTML)

References(28)

References

[1]	钱智明, 钟平, 王润生. 基于图正则化与非负组稀疏的自动图像标注[J]. 电子与信息学报, 2015, 37(4): 784–790. doi: 10.11999/JEIT141282 QIAN Zhiming, ZHONG Ping, and WANG Runsheng. Automatic image annotation via graph regularization and non-negative group sparsity[J]. Journal of Electronics &Information Technology, 2015, 37(4): 784–790. doi: 10.11999/JEIT141282
[2]	刘正, 张国印, 陈志远. 基于特征加权和非负矩阵分解的多视角聚类算法[J]. 电子学报, 2016, 44(3): 535–540. doi: 10.3969/j.issn.0372-2112.2016.03.006 LIU Zheng, ZHANG Guoyin, and CHEN Zhiyuan. A Multiview clustering algorithm based on feature weighting and non-negative matrix factorization[J]. Acta Electronica Sinica, 2016, 44(3): 535–540. doi: 10.3969/j.issn.0372-2112.2016.03.006
[3]	马源培, 杨卓璇, 李慧嘉. 结合Bass模型和LTV的创新产品扩散预测[J]. 聊城大学学报:自然科学版, 2020, 33(4): 26–32. MA Peiyuan, YANG Zhuoxuan, and LI Huijia. Innovative product diffusion forecasting combined bass model and LTV[J]. Journal of Liaocheng University:Natural Science Edition, 2020, 33(4): 26–32.
[4]	JIA Yuheng, KWONG S, HOU Junhui, et al. Semi-supervised non-negative matrix factorization with dissimilarity and similarity regularization[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(7): 2510–2521.
[5]	邱飞岳, 陈博文, 陈铁明, 等. 稀疏诱导流形正则化凸非负矩阵分解算法[J]. 通信学报, 2020, 41(5): 84–95. doi: 10.11959/j.issn.1000-436x.2020064 QIU Feiyue, CHEN Bowen, CHEN Tieming, et al. Sparsity induced convex nonnegative matrix factorization algorithm with manifold regularization[J]. Journal on Communications, 2020, 41(5): 84–95. doi: 10.11959/j.issn.1000-436x.2020064
[6]	陈善学, 刘荣华. 基于子空间结构正则化的L₂₁非负矩阵分解高光谱解混[J]. 电子与信息学报, 2022, 44(5): 1704–1713. doi: 10.11999/JEIT210232 CHEN Shanxue and LIU Ronghua. L₂₁ Nonnegative matrix factorization based on subspace structure regularization for hyperspectral unmixing[J]. Journal of Electronics &Information Technology, 2022, 44(5): 1704–1713. doi: 10.11999/JEIT210232
[7]	CHEN Wensheng, LIU Jingmin, PAN Binbin, et al. Face recognition using nonnegative matrix factorization with fractional power inner product kernel[J]. Neurocomputing, 2019, 348: 40–53. doi: 10.1016/j.neucom.2018.06.083
[8]	YANG Zhen, CHEN Weitong, and HUANG Jian. Enhancing recommendation on extremely sparse data with blocks-coupled non-negative matrix factorization[J]. Neurocomputing, 2018, 278: 126–133. doi: 10.1016/j.neucom.2017.04.080
[9]	SHIRDHONKAR S and JACOBS D W. Approximate earth mover’s distance in linear time[C]. 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, USA, 2008: 1–8.
[10]	CAI Deng, HE Xiaofei, HAN Jiawei, et al. Graph regularized nonnegative matrix factorization for data representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(8): 1548–1560. doi: 10.1109/TPAMI.2010.231
[11]	QIAN Wei, HONG Bin, CAI Deng, et al. Non-negative matrix factorization with Sinkhorn distance[C]. The 25th International Joint Conference on Artificial Intelligence, New York, USA, 2016: 1960–1966.
[12]	HOYER P O. Non-negative matrix factorization with sparseness constraints[J]. Journal of Machine Learning Research, 2004, 5(9): 1457–1469.
[13]	LIU Haifeng, WU Zhaohui, LI Xuelong, et al. Constrained nonnegative matrix factorization for image representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(7): 1299–1311. doi: 10.1109/TPAMI.2011.217
[14]	LI Dong, ZHANG Baoxian, YAO Zheng, et al. A feature scaling based k-nearest neighbor algorithm for indoor positioning system[C]. 2014 IEEE Global Communications Conference, Austin, USA, 2014: 436–441.
[15]	JIMÉNEZ-CORDERO A and MALDONADO S. Automatic feature scaling and selection for support vector machine classification with functional data[J]. Applied Intelligence, 2021, 51(1): 161–184.
[16]	RUBNER Y, TOMASI C, and GUIBAS L J. The earth mover’s distance as a metric for image retrieval[J]. International Journal of Computer Vision, 2000, 40(2): 99–121. doi: 10.1023/A:1026543900054
[17]	CUTURI M. Sinkhorn distances: Lightspeed computation of optimal transport[C]. The 26th International Conference on Neural Information Processing Systems, Lake Tahoe, Spain, 2013: 2292–2300.
[18]	FROGNER C, ZHANG Chiyuan, MOBAHI H, et al. Learning with a Wasserstein loss[C]. The 28th International Conference on Neural Information Processing Systems, Montreal, Canada, 2015: 2053–2061.
[19]	HUANG Shudong, WANG Hongjun, LI Tao, et al. Robust graph regularized nonnegative matrix factorization for clustering[J]. Data Mining and Knowledge Discovery, 2018, 32(2): 483–503. doi: 10.1007/s10618-017-0543-9
[20]	XU Zongben, CHANG Xiangyu, XU Fengmin, et al. L_1/2 regularization: A thresholding representation theory and a fast solver[J]. IEEE Transactions on Neural Networks and Learning Systems, 2012, 23(7): 1013–1027. doi: 10.1109/TNNLS.2012.2197412
[21]	LEE D D and SEUNG H S. Learning the parts of objects by non-negative matrix factorization[J]. Nature, 1999, 401(6755): 788–791. doi: 10.1038/44565
[22]	ZHANG Yunmeng, SHU Zhenqiu, ZHANG Jie, et al. Dual Graph regularized NMF with Sinkhorn Distance[C]. The 19th International Symposium on Distributed Computing and Applications for Business Engineering and Science, Xuzhou, China, 2020: 146–149.
[23]	MENG Yang, SHANG Ronghua, JIAO Licheng, et al. Dual-graph regularized non-negative matrix factorization with sparse and orthogonal constraints[J]. Engineering Applications of Artificial Intelligence, 2018, 69: 24–35. doi: 10.1016/j.engappai.2017.11.008
[24]	ZHANG Lin, LIU Zhonghua, PU Jiexin, et al. Adaptive graph regularized nonnegative matrix factorization for data representation[J]. Applied Intelligence, 2020, 50(2): 438–447. doi: 10.1007/s10489-019-01539-9
[25]	CHOI S. Algorithms for orthogonal nonnegative matrix factorization[C]. Proceedings of 2008 IEEE International Joint Conference on Neural Networks, Hong Kong, China, 2008: 1828–1832.
[26]	WU Wenhui, KWONG S, HOU Junhui, et al. Simultaneous dimensionality reduction and classification via dual embedding regularized nonnegative matrix factorization[J]. IEEE Transactions on Image Processing, 2019, 28(8): 3836–3847. doi: 10.1109/TIP.2019.2907054
[27]	LIANG Naiyao, YANG Zuyuan, LI Zhenni, et al. Multi-view clustering by non-negative matrix factorization with co-orthogonal constraints[J]. Knowledge-Based Systems, 2020, 194: 105582. doi: 10.1016/j.knosys.2020.105582
[28]	QIU Xiru, CHEN Zhikui, ZHAO Liang, et al. Unsupervised multi-view non-negative for law data feature learning with dual graph-regularization in smart Internet of Things[J]. Future Generation Computer Systems, 2019, 100: 523–530. doi: 10.1016/j.future.2019.05.055