基于改进Res-UNet网络的钢铁表面缺陷图像分割研究

李原; 李燕君; 刘进超; 范衠; 王庆林

doi:10.11999/JEIT211350

基于改进Res-UNet网络的钢铁表面缺陷图像分割研究

doi: 10.11999/JEIT211350

1.
北京理工大学自动化学院北京 100081
2.
南开大学人工智能学院天津 300071
3.
汕头大学工学院汕头 515063

详细信息

作者简介:
李原：男，1977年生，副教授，研究方向为智能机器人系统、计算机视觉、人工智能

李燕君：女，1997年生，硕士生，研究方向为计算机视觉、深度学习

刘进超：男，1981年生，副教授，研究方向为机器学习、机器视觉、智能检测

范衠：男，1974年生，教授，研究方向为人工智能与机器人、智能计算、图像处理

王庆林：男，1963年生，教授，研究方向为智能信息处理、非线性控制

通讯作者:
李原　liyuan@bit.edu.cn

中图分类号: TN911.73; TP391
计量
- 文章访问数: 1633
- HTML全文浏览量: 1208
- PDF下载量: 325
- 被引次数: 0
出版历程
- 收稿日期: 2021-11-29
- 修回日期: 2022-03-22
- 网络出版日期: 2022-03-30
- 刊出日期: 2022-05-25

Research on Segmentation of Steel Surface Defect Images Based on Improved Res-UNet Network

1.
School of Automation, Beijing Institute of Technology, Beijing 100081, China
2.
College of Artificial Intelligence, Nankai University, Tianjin 300071, China
3.
College of Engineering, Shantou University, Shantou 515063, China

摘要

摘要: 为了提高钢铁质量图像检测的效率和精度，提高生产自动化水平，该文提出一种改进的Res-UNet网络分割算法。使用ResNet50代替ResNet18作为编码模块，增强特征提取能力；修改编码模块，使残差块间稠密连接，增强浅层特征的深度延展，充分利用特征；使用加权Dice损失和加权交叉熵损失(BCEloss)结合的新损失函数缓解样本不均衡的情况；数据集增强策略保证网络学习更多的样本特征，增强细节分割精度。相比于经典的UNet算法，组合优化后的Res-UNet网络的Dice系数最多提高了12.64%，达到0.7930，网络训练时间更短，对各类缺陷的分割精准度更优，证明该文算法在钢铁表面缺陷分割领域具有应用价值。
- 缺陷分割 /
- Res-UNet /
- 稠密连接 /
- 加权损失 /
- 图像增强
Abstract: In order to improve the efficiency and accuracy of steel quality images detection and promote the automation level of industry, an improved Res-UNet segmentation algorithm is proposed. ResNet50 is used instead of ResNet18 as the encode module to enhance feature extraction capability. Structure like DenseNet is added to encode module, which helps to make full use of shallow feature maps. A new loss function combining weighted Dice loss and weighted Binary Cross Entropy loss (BCEloss) is used to alleviate data imbalance. Data set enhancement strategy ensures that the network learns more features and improves the segmentation accuracy. Compared with the classic UNet, the Dice coefficient of the improved Res-UNet increases by 12.64% and reaches 0.7930. In all, the improved Res-UNet achieves much better accuracy on various defects while requires much less training efforts. The algorithm proposed by this paper is of practical use in the field of steel surface defect segmentation.
- Defect segmentation /
- Res-UNet /
- Dense connection /
- Weighted loss /
- Image enhancement

HTML全文

图 1 Res-UNet网络结构

下载: 全尺寸图片幻灯片

图 2 残差模块和译码模块结构

下载: 全尺寸图片幻灯片

图 3 4类缺陷分割示例图

下载: 全尺寸图片幻灯片

图 4 缺陷真实标签和各网络预测结果示例图

下载: 全尺寸图片幻灯片

表 1 网络各层参数及特征图大小

类型	核大小/步长	输出
Conv1	$ 7 \times 7/2 $	$ 128 \times 800 \times 64 $
Maxpool	$ 3 \times 3/2 $	$ 64 \times 400 \times 64 $
Res Block	$ 3 \times 3/1 $	$ 64 \times 400 \times 64 $
Res Block	$ 3 \times 3/2 $	$ 32 \times 200 \times 128 $
Res Block	$ 3 \times 3/2 $	$ 16 \times 100 \times 256 $
Res Block	$ 3 \times 3/2 $	$ 8 \times 50 \times 512 $
Decode Block	$ 3 \times 3/1 $	$ 16 \times 100 \times 256 $
Decode Block	$ 3 \times 3/1 $	$ 32 \times 200 \times 128 $
Decode Block	$ 3 \times 3/1 $	$ 64 \times 400 \times 64 $
Decode Block	$ 3 \times 3/1 $	$ 128 \times 800 \times 32 $
Up Sample	插值规模因子=2	$ 256 \times 1600 \times 32 $
Con2d	$ 3 \times 3/1 $	$ 256 \times 1600 \times 16 $
Con2d	$ 3 \times 3/1 $	$ 256 \times 1600 \times 16 $

下载: 导出CSV

表 2 训练数据集缺陷分布情况(张)

有损图片	无损图片	仅含划痕	仅含埋渣	仅含皮麟	仅含氧化	含两种缺陷
4864	4311	559	138	3495	380	292

下载: 导出CSV

表 3 各网络训练超参数及实验结果

网络名称	训练周期(epoch)	训练时间(min)	Dice系数	识别准确率(%)
UNet (M1)	80	552	0.6666	69.02
Res18-UNet (M2)	40	674	0.7551	80.47
Res50-UNet (M3)	50	1191	0.7564	80.80
Res18-UNet+ReCon (M4)	40	560	0.7629	82.83
Res18-UNet+train-aug (M5)	45	554	0.7644	81.14
Res18-UNet+BDloss (M6)	35	568	0.7797	87.54
Res18-UNet+BDloss+ReCon (M7)	45	566	0.7921	88.21
Res18-UNet+BDloss+train-aug (M8)	40	531	0.7881	89.23
Res50-UNet+BDloss+train-aug (M9)	55	1172	0.7910	89.90
Res18-UNet+BDloss+train-aug +ReCon (M10)	29	405	0.7930	88.89
AlexNet (M11)	300	495	–	53.80
DenseNet (M12)	35	469	–	80.56
Xception (M13)	10	65	0.6238	52.86
Mask RCNN (M14)	35	299	0.7350	62.96

下载: 导出CSV

参考文献(22)

[1]	徐镪, 朱洪锦, 范洪辉, 等. 改进的YOLOv3网络在钢板表面缺陷检测研究[J]. 计算机工程与应用, 2020, 56(16): 265–272. doi: 10.3778/j.issn.1002-8331.2003-0232 XU Qiang, ZHU Hongjin, FAN Honghui, et al. Study on detection of steel plate surface defects by improved YOLOv3 network[J]. Computer Engineering and Applications, 2020, 56(16): 265–272. doi: 10.3778/j.issn.1002-8331.2003-0232
[2]	BORSELLI A, COLLA V, VANNUCCI M, et al. A fuzzy inference system applied to defect detection in flat steel production[C]. The International Conference on Fuzzy Systems, Barcelona, Spain, 2010: 1–6.
[3]	XU Ke, XU Yang, ZHOU Peng, et al. Application of RNAMlet to surface defect identification of steels[J]. Optics and Lasers in Engineering, 2018, 105: 110–117. doi: 10.1016/j.optlaseng.2018.01.010
[4]	汤勃, 孔建益, 王兴东, 等. 粗糙集理论的带钢表面缺陷图像的识别与分类[J]. 中国图象图形学报, 2011, 16(7): 1213–1218. doi: 10.11834/jig.20110718 TANG Bo, KONG Jianyi, WANG Xingdong, et al. Recognition and classification for steel strip surface defect images based on rough set theory[J]. Journal of Image and Graphics, 2011, 16(7): 1213–1218. doi: 10.11834/jig.20110718
[5]	LIU Kun, WANG Heying, CHEN Haiyong, et al. Steel surface defect detection using a new haar–weibull-variance model in unsupervised manner[J]. IEEE Transactions on Instrumentation and Measurement, 2017, 66(10): 2585–2596. doi: 10.1109/TIM.2017.2712838
[6]	WANG Heying, ZHANG Jiawei, TIAN Ying, et al. A simple guidance template-based defect detection method for strip steel surfaces[J]. IEEE Transactions on Industrial Informatics, 2019, 15(5): 2798–2809. doi: 10.1109/TII.2018.2887145
[7]	KWON B K, WON J S, and KANG D J. Fast defect detection for various types of surfaces using random forest with VOV features[J]. International Journal of Precision Engineering and Manufacturing, 2015, 16(5): 965–970. doi: 10.1007/s12541-015-0125-y
[8]	CHU Maoxiang, LIU Xiaoping, GONG Rongfen, et al. Multi-class classification method using twin support vector machines with multi-information for steel surface defects[J]. Chemometrics and Intelligent Laboratory Systems, 2018, 176: 108–118. doi: 10.1016/j.chemolab.2018.03.014
[9]	YANG Changhui, ZHANG Jinxun, JI Gang, et al. Recognition of defects in steel surface image based on neural networks and morphology[C]. The Second Workshop on Digital Media and its Application in Museum & Heritages (DMAMH 2007), Chongqing, China, 2007: 72–77.
[10]	REN Ruoxu, HUNG T, and TAN K C. A generic deep-learning-based approach for automated surface inspection[J]. IEEE Transactions on Cybernetics, 2018, 48(3): 929–940. doi: 10.1109/TCYB.2017.2668395
[11]	王宪保, 李洁, 姚明海, 等. 基于深度学习的太阳能电池片表面缺陷检测方法[J]. 模式识别与人工智能, 2014, 27(6): 517–523. doi: 10.3969/j.issn.1003-6059.2014.06.006 WANG Xianbao, LI Jie, YAO Minghai, et al. Solar cells surface defects detection based on deep learning[J]. Pattern Recognition and Artificial Intelligence, 2014, 27(6): 517–523. doi: 10.3969/j.issn.1003-6059.2014.06.006
[12]	JIA Hongbin, MURPHEY Y L, SHI Jianjun, et al. An intelligent real-time vision system for surface defect detection[C]. The 17th International Conference on Pattern Recognition, Cambridge, UK, 2004: 239–242.
[13]	KIM M S, PARK T, and PARK P. Classification of steel surface defect using convolutional neural network with few images[C]. The 12th Asian Control Conference, Kitakyushu, Japan, 2019: 1398–1401.
[14]	MASCI J, GIUSTI A, CIRESAN D, et al. A fast learning algorithm for image segmentation with max-pooling convolutional networks[C]. 2013 IEEE International Conference on Image Processing, Melbourne, Australia, 2013: 2713–2717.
[15]	郭兴宝. 基于机器学习的无缝钢管表面缺陷检测技术研究[D]. [硕士论文], 大连交通大学, 2019. GUO Baoxing. Research on surface defect detection technology of seamless steel tube based on machine learning[D]. [Master dissertation], Dalian Jiaotong University, 2019.
[16]	甘胜丰. 带钢表面缺陷图像检测与分类方法研究[D]. [博士论文], 中国地质大学, 2013. GAN Shengfeng. Method of strip surface defect image detection and classification[D]. [Ph. D. dissertation], China University of Geosciences, 2013.
[17]	STEPHAN M and SANTRA A. Radar-based human target detection using deep residual U-net for smart home applications[C]. The 18th IEEE International Conference On Machine Learning And Applications (ICMLA), Boca Raton, USA, 2019: 175–182.
[18]	储珺, 朱晓阳, 冷璐, 等. 引入通道注意力和残差学习的目标检测器[J]. 模式识别与人工智能, 2020, 33(10): 889–897. doi: 10.16451/j.cnki.issn1003-6059.202010003 CHU Jun, ZHU Xiaoyang, LENG Lu, et al. Target detector with channel attention and residual learning[J]. Pattern Recognition and Artificial Intelligence, 2020, 33(10): 889–897. doi: 10.16451/j.cnki.issn1003-6059.202010003
[19]	ZENG Zitao, XIE Weihao, ZHANG Yunzhe, et al. RIC-Unet: An improved neural network based on unet for nuclei segmentation in histology images[J]. IEEE Access, 2019, 7: 21420–21428. doi: 10.1109/ACCESS.2019.2896920
[20]	刘市祺, 孙晓波, 谢晓亮, 等. 基于区域建议网络和残差结构的导丝跟踪[J]. 模式识别与人工智能, 2019, 32(1): 36–42. doi: 10.16451/j.cnki.issn1003-6059.201901005 LIU Shiqi, SUN Xiaobo, XIE Xiaoliang, et al. Guidewire tracking based on regional proposal network and residual structure[J]. Pattern Recognition and Artificial Intelligence, 2019, 32(1): 36–42. doi: 10.16451/j.cnki.issn1003-6059.201901005
[21]	SUDRE C H, LI Wenqi, VERCAUTEREN T, et al. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations[C]. The 3rd International Workshop on Deep Learning in Medical Image Analysis, Québec City, Canada, 2017: 240–248.
[22]	张立恒, 王浩, 薛博维, 等. 基于改进D-LinkNet模型的高分遥感影像道路提取研究[J]. 计算机工程, 2021, 47(9): 288–296. doi: 10.19678/j.issn.1000-3428.0058977 ZHANG Liheng, WANG Hao, XUE Bowei, et al. Research of road extraction from high-resolution remote sensing images based on improved D-LinkNet model[J]. Computer Engineering, 2021, 47(9): 288–296. doi: 10.19678/j.issn.1000-3428.0058977