全球尺度下的海洋鱼类图像智能分类研究进展

周鹏; 李昌永; 步雨馨; 周芷诺; 王春生; 沈红斌; 潘小勇

doi:10.11999/JEIT231365

全球尺度下的海洋鱼类图像智能分类研究进展

doi: 10.11999/JEIT231365

周鹏^{1, 2, 3},
李昌永³,
步雨馨^{1, 2},
周芷诺³,
王春生^{1, 2},
沈红斌³,
潘小勇^3, ,

1.
自然资源部第二海洋研究所杭州 310012
2.
自然资源部海洋生态系统动力学实验室杭州 310012
3.
上海交通大学图像处理与模式识别研究所上海 200240

基金项目: 国家重点研发计划(2023YFC2811502)，上海交通大学深蓝计划(SL2022ZD108,SL2021MS005)

详细信息

作者简介:
周鹏：男，高级工程师，研究方向为海洋生物学与生物信息学研究

李昌永：男，博士生，研究方向为模式识别与生物信息学

步雨馨：女，硕士生，研究方向为海洋生物学

周芷诺：女，本科生，研究方向为模式识别与生物信息学

王春生：男，研究员，研究方向为海洋生态学

沈红斌：男，教授，研究方向为识别与生物信息学

潘小勇：男，长聘教轨副教授，研究方向为图像处理与模式识别研究

通讯作者:
潘小勇　2008xypan@sjtu.edu.cn

中图分类号: TN081; TN911.7
计量
- 文章访问数: 369
- HTML全文浏览量: 282
- PDF下载量: 71
- 被引次数: 0
出版历程
- 收稿日期: 2023-12-11
- 修回日期: 2024-03-29
- 网络出版日期: 2024-04-12
- 刊出日期: 2024-05-30

A Review of the Artificial Intelligence-based Image Classification of Fishes in the Global Oceans

ZHOU Peng^{1, 2, 3},
LI Changyong³,
BU Yuxin^{1, 2},
ZHOU Zhinuo³,
WANG Chunsheng^{1, 2},
SHEN Hongbin³,
PAN Xiaoyong^{3
, ,}

1.
Second Institute of Oceanography, Ministry of Natural Resources, Hangzhou 310012, China
2.
Key Laboratory of Marine Ecosystem Dynamics, Ministry of Natural Resources, Hangzhou 310012, China
3.
Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, and Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai 200240, China

Funds: The National Key Research and Development Program of China (2023YFC2811502), The Oceanic Interdisciplinary Program of Shanghai Jiao Tong University (SL2022ZD108, SL2021MS005)

摘要

摘要: 在全球尺度上了解鱼类物种组成、丰度及时空分布等，将有助于其生物多样保护。水下图像采集是获取鱼类物种多样性数据的主要调查手段之一，但图像信息分析工作耗时耗力。2015年以来，海洋鱼类图像数据集更新和深度学习模型算法优化等方面取得了一系列进展，但细粒度分类表现仍显不足，研究成果的生产实践应用相对薄弱。因此，该文首先分析海洋相关行业对鱼类自动化图像分类的需求，然后综合介绍鱼类图像数据集和深度学习算法应用，并分析了所面临的小样本下的细粒度分析等主要挑战及相应解决方法。最后探讨了基于深度学习的海洋鱼类图像自动化分类对相关图像信息处理研究及应用平台对未来在生态环境监测等海洋相关产业领域的重要性及其前景。该文旨在为快速了解基于深度学习的海洋鱼类图像自动化分类的研究背景、进展和未来方向的工作者提供相关信息。
- 深度学习 /
- 海洋鱼类 /
- 生物多样性 /
- 小样本 /
- 图像分类
Abstract: Understanding the species composition, abundance and temporal and spatial distribution of fish on a global scale will help their biodiversity conservation. Underwater image acquisition is one of the main means to survey fish species diversity, but image data analysis is time-consuming and labor-intensive. Since 2015, a series of progress has been made in updating the datasets of marine fish images and optimizing the algorithm of deep learning models, but the performance of fine-grained classification is still insufficient, and the production practice application of research results is relatively weak. Therefore, the need for automated fish image classification in marine investigations is firstly studied. Then a comprehensive introduction to fish image datasets and deep learning algorithm applications is provided, and the main challenges and the corresponding solutions are analyzed. Finally, the importance of automated classification of marine fish images for related image information processing research is discussed, and its prospects in the field of marine monitoring are summarized.
- Deep learning /
- Marine fish /
- Biodiversity /
- Few shots /
- Image classification

HTML全文

图 1 鱼类检测算法发展时序图

下载: 全尺寸图片幻灯片

图 2 全球海洋鱼类图像分类实现及应用示意图

下载: 全尺寸图片幻灯片

表 1 海洋鱼类多样性调查的主要海洋产业需求及示例场景

海洋产业	行业需求	功能场景示例
海洋资源开发利用	重大工程生态环境影响评价/监测	核电对鱼类多样性影响
海洋资源开发利用	海洋油气、海底矿产等资源开发	采矿对鱼类多样性影响
海洋生态环境保护	海洋污染治理评估	微塑料等污染对鱼类影响
海洋生态环境保护	海洋保护区设立及划区依据	鱼类多样性及地理分布调查
海洋防灾减灾	赤潮、绿潮、海洋酸化	灾害对鱼类影响
海洋防灾减灾	全球气候变化	厄尔尼诺和拉尼娜现象对鱼类多样性影响；鱼类迁移变化
海洋安全保障	海洋环境安全保障	核污水对鱼类影响
海洋安全保障	港口航运	外来鱼类物种入侵监测
海洋科学认知	海底深部探测	鱼类新物种发现
海洋科学认知	公众普及	海洋鱼类识别
海洋渔业	海洋捕捞	鱼类资源调查与评估
海洋渔业	海水养殖	环境污染对鱼类影响
滨海旅游	海岸带可持续发展	污染影响评估；濒危鱼类识别
滨海旅游	健康与安全	危险物种识别，如有毒虾虎鱼

下载: 导出CSV

表 2 深度学习在鱼类图像自动化识别中的应用实例

年份	数据源	物种数	原始图像张数	准确率(%)	参考文献
2015	ImageCLEF	12	24 277	81.40	[6]
2015	Croatian fish dataset	12	794	66.78	[8]
2016	Fish4Knowledge	23	27 370	98.64	[13]
2017	ImageCLEF	12	24 277	89.95	[14]
2017	Fish4Knowledge	23	27 142	96.29	[15]
2018	Fish4Knowledge	23	27 000	99.45	[16]
2018	Croatian fish dataset	12	794	83.92	[17]
2018	自建图集	16	1 647	94.30	[18]
2018	WildFish	1 000	54 459	74.70	[19]
2020	DeepFish	20	39 766	99.00	[9]
2020	WildFish++	2 348	103 034	74.70 ^*	[11]
2021	自建图集	15 ^†	23 211	99.23	[20]
2021	QUT dataset	6	1 334	90.48	[21]
2023	OceanFish	136	63 622	97.12	[10]
2023	FishNet	17 357	94 532	61.38 ^△	[12]
注：^* 仅针对选取出的易混淆的22对鱼类，训练集含1 668张图片，测试集含1 320张图片。　　^† 包括鱼类11种、棘皮动物类3种和甲壳动物类1种。　　^△ 在科一级分类水平上，共463科。

下载: 导出CSV

表 3 主要问题、原因分析及解决方案

问题	原因	解决方案
识别准确率低	训练数据集小	扩大数据集
识别准确率低	算法需要优化	使用和优化小样本算法
使用流程复杂	需数学、编程等专业知识	利用网络平台提高易用性
应用范围不广	受操作系统、区域限制	利用网络平台提高易用性

下载: 导出CSV

参考文献(64)

[1]	KUITER R H and DEBELIUS H. World atlas of marine fishes[M]. 2nd ed. Frankfurt: IKAN-Unterwasserarchiv, 2006.
[2]	ALLEN G R. Field Guide to Marine Fishes of Tropical Australia and South-East Asia[M]. Western Australian Museum, 2009.
[3]	陈大刚, 张美昭. 中国海洋鱼类上[M]. 青岛: 中国海洋大学出版社, 2015. CHEN Dagang and ZHANG Meizhao. Marine Fishes of China[M]. Qingdao: China Ocean University Press, 2015.
[4]	陈大刚, 张美昭. 中国海洋鱼类下[M]. 青岛: 中国海洋大学出版社, 2015. CHEN Dagang and ZHANG Meizhao. Marine Fishes of China[M]. Qingdao: China Ocean University Press, 2015.
[5]	陈大刚, 张美昭. 中国海洋鱼类中[M]. 青岛: 中国海洋大学出版社, 2015. CHEN Dagang and ZHANG Meizhao. Marine Fishes of China[M]. Qingdao: China Ocean University Press, 2015.
[6]	LI Xiu, SHANG Min, QIN Hongwei, et al. Fast accurate fish detection and recognition of underwater images with Fast R-CNN[C]. MTS/IEEE Washington, Washington, USA, 2015: 1–5. doi: 10.23919/OCEANS.2015.7404464.
[7]	VILLON S, CHAUMONT M, SUBSOL G, et al. Coral reef fish detection and recognition in underwater videos by supervised machine learning: Comparison between Deep Learning and HOG+SVM methods[C]. The 17th International Conference on Advanced Concepts for Intelligent Vision Systems, Lecce, Italy, 2016: 160–171. doi: 10.1007/978-3-319-48680-2_15.
[8]	JÄGER J, SIMON M, DENZLER J, et al. Croatian fish dataset: Fine-grained classification of fish species in their natural habitat[C]. Machine Vision of Animals and their Behaviour Workshop 2015, 2015: 2. doi: 10.5244/C.29.MVAB.6.
[9]	SALEH A, LARADJI I H, KONOVALOV D A, et al. A realistic fish-habitat dataset to evaluate algorithms for underwater visual analysis[J]. Scientific Reports, 2020, 10(1): 14671. doi: 10.1038/s41598-020-71639-x.
[10]	LIN Yuan, CHU Zhaoqi, KORHONEN J, et al. Fast accurate fish recognition with deep learning based on a domain-specific large-scale fish dataset[C]. The 29th International Conference on Multimedia Modeling, Bergen, Norway, 2023: 515–526. doi: 10.1007/978-3-031-27077-2_40.
[11]	ZHUANG Peiqin, WANG Yali, and QIAO Yu. Wildfish++: A comprehensive fish benchmark for multimedia research[J]. IEEE Transactions on Multimedia, 2021, 23: 3603–3617. doi: 10.1109/TMM.2020.3028482.
[12]	KHAN F F, LI Xiang, TEMPLE A J, et al. FishNet: A large-scale dataset and benchmark for fish recognition, detection, and functional trait prediction[C]. 2023 IEEE/CVF International Conference on Computer Vision, Paris, France, 2023: 20439–20449. doi: 10.1109/ICCV51070.2023.01874.
[13]	QIN Hongwei, LI Xiu, LIANG Jian, et al. DeepFish: Accurate underwater live fish recognition with a deep architecture[J]. Neurocomputing, 2016, 187: 49–58. doi: 10.1016/j.neucom.2015.10.122.
[14]	LI Xiu, TANG Youhua, and GAO Tingwei. Deep but lightweight neural networks for fish detection[C]. OCEANS 2017 - Aberdeen, Aberdeen, UK, 2017: 1–5. doi: 10.1109/OCEANSE.2017.8084961.
[15]	RATHI D, JAIN S, and INDU S. Underwater fish species classification using convolutional neural network and deep learning[C]. The 2017 Ninth International Conference on Advances in Pattern Recognition, Bangalore, India, 2017: 1–6. doi: 10.1109/ICAPR.2017.8593044.
[16]	TAMOU A B, BENZINOU A, NASREDDINE K, et al. Underwater live fish recognition by deep learning[C]. The 8th International Conference on Image and Signal Processing, Cherbourg, France, 2018: 275–283. doi: 10.1007/978-3-319-94211-7_30.
[17]	QIU Chenchen, ZHANG Shaoyong, WANG Chao, et al. Improving transfer learning and squeeze- and-excitation networks for small-scale fine-grained fish image classification[J]. IEEE Access, 2018, 6: 78503–78512. doi: 10.1109/ACCESS.2018.2885055.
[18]	SIDDIQUI S A, SALMAN A, MALIK M I, et al. Automatic fish species classification in underwater videos: Exploiting pre-trained deep neural network models to compensate for limited labelled data[J]. ICES Journal of Marine Science, 2018, 75(1): 374–389. doi: 10.1093/icesjms/fsx109.
[19]	ZHUANG Peiqin, WANG Yali, and QIAO Yu. WildFish: A large benchmark for fish recognition in the wild[C]. The 26th ACM international conference on Multimedia. Seoul, Republic of Korea, 2018: 1301–1039. doi: 10.1145/3240508.3240616.
[20]	孙东洋, 刘辉, 张纪红, 等. 基于深度卷积神经网络的海洋牧场岩礁性生物图像分类[J]. 海洋与湖沼, 2021, 52(5): 1160–1169. doi: 10.11693/hyhz20210100005. SUN Dongyang, LIU Hui, ZHANG Jihong, et al. Classification of reef biological images of marine ranch based on deep convolution neural network[J]. Oceanologia et Limnologia Sinica, 2021, 52(5): 1160–1169. doi: 10.11693/hyhz20210100005.
[21]	IQBAL M A, WANG Zhijie, ALI Z A, et al. Automatic fish species classification using deep convolutional neural networks[J]. Wireless Personal Communications, 2021, 116(2): 1043–1053. doi: 10.1007/s11277-019-06634-1.
[22]	BOOM B J, HUANG P X, HE Jiyin, et al. Supporting ground-truth annotation of image datasets using clustering[C]. The 21st International Conference on Pattern Recognition, Tsukuba, Japan, 2012: 1542–1545.
[23]	KAY J and MERRIFIELD M. The fishnet open images database: A dataset for fish detection and fine-grained categorization in fisheries[J]. arXiv: 2106.09178, 2021.
[24]	TORISAWA S, KADOTA M, KOMEYAMA K, et al. A digital stereo-video camera system for three-dimensional monitoring of free-swimming Pacific bluefin tuna, Thunnus orientalis, cultured in a net cage[J]. Aquatic Living Resources, 2011, 24(2): 107–112. doi: 10.1051/alr/2011133.
[25]	MENG Lin, HIRAYAMA T, and OYANAGI S. Underwater-drone with panoramic camera for automatic fish recognition based on deep learning[J]. IEEE Access, 2018, 6: 17880–17886. doi: 10.1109/ACCESS.2018.2820326.
[26]	PLATT J C. Sequential minimal optimization: A fast algorithm for training support vector machines[R]. MSR-TR-98-14, 1998.
[27]	HUANG P X, BOOM B J, and FISHER R B. Hierarchical classification with reject option for live fish recognition[J]. Machine Vision and Applications, 2015, 26(1): 89–102. doi: 10.1007/s00138-014-0641-2.
[28]	CHUANG Mengche, HWANG J N, and WILLIAMS K. A feature learning and object recognition framework for underwater fish images[J]. IEEE Transactions on Image Processing, 2016, 25(4): 1862–1872. doi: 10.1109/TIP.2016.2535342.
[29]	ISLAM S M M, BANI S I, and GHOSH R. Content-based fish classification using combination of machine learning methods[J]. International Journal of Information Technology and Computer Science (IJITCS), 2021, 13(8): 62–68. doi: 10.5815/ijitcs.2021.01.05.
[30]	OU Liguo, LIU Bilin, CHEN Xinjun, et al. Automatic classification of the phenotype textures of three Thunnus species based on the machine learning SVM algorithm[J]. Canadian Journal of Fisheries and Aquatic Sciences, 2023, 80(8): 1221–1236. doi: 10.1139/cjfas-2022-0270.
[31]	LUAN Jing, ZHANG Chongliang, XU Binduo, et al. The predictive performances of random forest models with limited sample size and different species traits[J]. Fisheries Research, 2020, 227: 105534. doi: 10.1016/j.fishres.2020.105534.
[32]	MAMPITIYA L I, NALMI R, and RATHNAYAKE N. Performance comparison of sea fish species classification using hybrid and supervised machine learning algorithms[C]. 2022 Moratuwa Engineering Research Conference, Moratuwa, Sri Lanka, 2022: 1–6. doi: 10.1109/MERCon55799.2022.9906206.
[33]	LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278–2324. doi: 10.1109/5.726791.
[34]	SARIGÜL M and AVCI M. Comparison of different deep structures for fish classification[J]. International Journal of Computer Theory and Engineering, 2017, 9(5): 362–366. doi: 10.7763/IJCTE.2017.V9.1167.
[35]	DEEP B V and DASH R. Underwater fish species recognition using deep learning techniques[C]. The 2019 6th International Conference on Signal Processing and Integrated Networks, Noida, India, 2019: 665–669. doi: 10.1109/SPIN.2019.8711657.
[36]	KONOVALOV D A, SALEH A, BRADLEY M, et al. Underwater fish detection with weak multi-domain supervision[C]. 2019 International Joint Conference on Neural Networks, Budapest, Hungary, 2019: 1–8. doi: 10.1109/IJCNN.2019.8851907.
[37]	ZHAO Zhenxi, LIU Yang, SUN Xudong, et al. Composited fishnet: Fish detection and species recognition from low-quality underwater videos[J]. IEEE Transactions on Image Processing, 2021, 30: 4719–4734. doi: 10.1109/TIP.2021.3074738.
[38]	ZENG Lingcai, SUN Bing, and ZHU Daqi. Underwater target detection based on Faster R-CNN and adversarial occlusion network[J]. Engineering Applications of Artificial Intelligence, 2021, 100: 104190. doi: 10.1016/j.engappai.2021.104190.
[39]	HONG KHAI T, ABDULLAH S N H S, HASAN M K, et al. Underwater fish detection and counting using mask regional convolutional neural network[J]. Water, 2022, 14(2): 222. doi: 10.3390/w14020222.
[40]	SONG Pinhao, LI Pengteng, DAI Linhui, et al. Boosting R-CNN: Reweighting R-CNN samples by RPN’s error for underwater object detection[J]. Neurocomputing, 2023, 530: 150–164. doi: 10.1016/j.neucom.2023.01.088.
[41]	XU Wenwei and MATZNER S. Underwater fish detection using deep learning for water power applications[C]. 2018 International Conference on Computational Science and Computational Intelligence, Las Vegas, USA, 2018: 313–318. doi: 10.1109/CSCI46756.2018.00067.
[42]	HU Xuelong, LIU Yang, ZHAO Zhengxi, et al. Real-time detection of uneaten feed pellets in underwater images for aquaculture using an improved YOLO-V4 network[J]. Computers and Electronics in Agriculture, 2021, 185: 106135. doi: 10.1016/j.compag.2021.106135.
[43]	AL MUKSIT A, HASAN F, HASAN BHUIYAN EMON F, et al. YOLO-Fish: A robust fish detection model to detect fish in realistic underwater environment[J]. Ecological Informatics, 2022, 72: 101847. doi: 10.1016/j.ecoinf.2022.101847.
[44]	LI Jianyuan, LIU Chunna, LU Xiaochun, et al. CME-YOLOv5: An efficient object detection network for densely spaced fish and small targets[J]. Water, 2022, 14(15): 2412. doi: 10.3390/w14152412.
[45]	ALABA S Y, NABI M M, SHAH C, et al. Class-aware fish species recognition using deep learning for an imbalanced dataset[J]. Sensors, 2022, 22(21): 8268. doi: 10.3390/s22218268.
[46]	MNIH V, HEESS N, GRAVES A, et al. Recurrent models of visual attention[C]. The 27th International Conference on Neural Information Processing Systems, Cambridge, USA, 2014: 2204–2212. doi: 10.5555/2969033.2969073.
[47]	ZHANG Wenbo, WU Chaoyi, and BAO Zhenshan. DPANet: Dual pooling-aggregated attention network for fish segmentation[J]. IET Computer Vision, 2022, 16(1): 67–82. doi: 10.1049/cvi2.12065.
[48]	GUPTA S, MUKHERJEE P, CHAUDHURY S, et al. DFTNet: Deep fish tracker with attention mechanism in unconstrained marine environments[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 5016213. doi: 10.1109/TIM.2021.3109731.
[49]	LI Shanmin, PAN Bei, CHENG Yuanshun, et al. Underwater fish object detection based on attention mechanism improved ghost-YOLOv5[C]. The 2022 7th International Conference on Intelligent Computing and Signal Processing, Xi’an, China, 2022: 599–603. doi: 10.1109/ICSP54964.2022.9778582.
[50]	CHEN Lulu, ZANG Zhaoxiang, HUANG Tianxing, et al. Marine fish object detection based on YOLOv5 and attention mechanism[C]. 2022 IEEE Smartworld, Ubiquitous Intelligence & Computing, Scalable Computing & Communications, Digital Twin, Privacy Computing, Metaverse, Autonomous & Trusted Vehicles (SmartWorld/UIC/ScalCom/DigitalTwin/PriComp/Meta), Haikou, China, 2022: 1252–1258. doi: 10.1109/SmartWorld-UIC-ATC-ScalCom-DigitalTwin-PriComp-Metaverse56740.2022.00185.
[51]	HAN Xu, ZHANG Zhengyan, DING Ning, et al. Pre-trained models: Past, present and future[J]. AI Open, 2021, 2: 225–250. doi: 10.1016/j.aiopen.2021.08.002.
[52]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]. The 31st International Conference on Neural Information Processing Systems, Long Beach, USA, 2017: 6000–6010. doi: 10.5555/3295222.3295349.
[53]	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[C]. The 9th International Conference on Learning Representations, 2021.
[54]	SALEH A, SHEAVES M, JERRY D, et al. Transformer-based Self-Supervised Fish Segmentation in Underwater Videos[J]. arXiv: 2206.05390, 2022.
[55]	SALEH A, JONES D, JERRY D, et al. A lightweight Transformer-based model for fish landmark detection[J]. arXiv: 2209.05777, 2022.
[56]	LIU Yang, AN Dong, REN Yinjie, et al. DP-FishNet: Dual-path Pyramid Vision Transformer-based underwater fish detection network[J]. Expert Systems with Applications, 2024, 238: 122018. doi: 10.1016/j.eswa.2023.122018.
[57]	GONG Bo, DAI Kanyuan, SHAO Ji, et al. Fish-TViT: A novel fish species classification method in multi water areas based on transfer learning and vision transformer[J]. Heliyon, 2023, 9(6): e16761. doi: 10.1016/j.heliyon.2023.e16761.
[58]	WANG Yaqing, YAO Quanming, KWOK J T, et al. Generalizing from a few examples: A survey on few-shot learning[J]. ACM Computing Surveys, 2020, 53(3): 63. doi: 10.1145/3386252.
[59]	VILLON S, IOVAN C, MANGEAS M, et al. Automatic underwater fish species classification with limited data using few-shot learning[J]. Ecological Informatics, 2021, 63: 101320. doi: 10.1016/j.ecoinf.2021.101320.
[60]	GUO Zonghui, ZHANG Liqiang, JIANG Yufeng, et al. Few-shot fish image generation and classification[C]. Global Oceans 2020: Singapore – U. S. Gulf Coast, Biloxi, USA, 2020: 1–6. doi: 10.1109/IEEECONF38699.2020.9389005.
[61]	LIU Feng, DING Hao, LI Daihui, et al. Few-shot learning with data enhancement and transfer learning for underwater target recognition[C]. 2021 OES China Ocean Acoustics, Harbin, China, 2021: 992–994. doi: 10.1109/COA50123.2021.9519853.
[62]	GONG Longqin, HU Zhuhua, and ZHOU Xiaoyi. A few samples underwater fish tracking method based on semi-supervised and attention mechanism[C]. The 2022 6th International Conference on Robotics, Control and Automation, Xiamen, China, 2022: 18–22. doi: 10.1109/ICRCA55033.2022.9828911.
[63]	ZHAI Jiping, HAN Lu, XIAO Ying, et al. Few-shot fine-grained fish species classification via sandwich attention CovaMNet[J]. Frontiers in Marine Science, 2023, 10: 1149186. doi: 10.3389/fmars.2023.1149186.
[64]	WEI Xiushen, SONG Yizhe, MAC AODHA O, et al. Fine-grained image analysis with deep learning: A survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(12): 8927–8948. doi: 10.1109/TPAMI.2021.3126648.