高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于改进Res-UNet网络的钢铁表面缺陷图像分割研究

李原 李燕君 刘进超 范衠 王庆林

李原, 李燕君, 刘进超, 范衠, 王庆林. 基于改进Res-UNet网络的钢铁表面缺陷图像分割研究[J]. 电子与信息学报, 2022, 44(5): 1513-1520. doi: 10.11999/JEIT211350
引用本文: 李原, 李燕君, 刘进超, 范衠, 王庆林. 基于改进Res-UNet网络的钢铁表面缺陷图像分割研究[J]. 电子与信息学报, 2022, 44(5): 1513-1520. doi: 10.11999/JEIT211350
LI Yuan, LI Yanjun, LIU Jinchao, FAN Zhun, WANG Qinglin. Research on Segmentation of Steel Surface Defect Images Based on Improved Res-UNet Network[J]. Journal of Electronics & Information Technology, 2022, 44(5): 1513-1520. doi: 10.11999/JEIT211350
Citation: LI Yuan, LI Yanjun, LIU Jinchao, FAN Zhun, WANG Qinglin. Research on Segmentation of Steel Surface Defect Images Based on Improved Res-UNet Network[J]. Journal of Electronics & Information Technology, 2022, 44(5): 1513-1520. doi: 10.11999/JEIT211350

基于改进Res-UNet网络的钢铁表面缺陷图像分割研究

doi: 10.11999/JEIT211350
详细信息
    作者简介:

    李原:男,1977年生,副教授,研究方向为智能机器人系统、计算机视觉、人工智能

    李燕君:女,1997年生,硕士生,研究方向为计算机视觉、深度学习

    刘进超:男,1981年生,副教授,研究方向为机器学习、机器视觉、智能检测

    范衠:男,1974年生,教授,研究方向为人工智能与机器人、智能计算、图像处理

    王庆林:男,1963年生,教授,研究方向为智能信息处理、非线性控制

    通讯作者:

    李原 liyuan@bit.edu.cn

  • 中图分类号: TN911.73; TP391

Research on Segmentation of Steel Surface Defect Images Based on Improved Res-UNet Network

  • 摘要: 为了提高钢铁质量图像检测的效率和精度,提高生产自动化水平,该文提出一种改进的Res-UNet网络分割算法。使用ResNet50代替ResNet18作为编码模块,增强特征提取能力;修改编码模块,使残差块间稠密连接,增强浅层特征的深度延展,充分利用特征;使用加权Dice损失和加权交叉熵损失(BCEloss)结合的新损失函数缓解样本不均衡的情况;数据集增强策略保证网络学习更多的样本特征,增强细节分割精度。相比于经典的UNet算法,组合优化后的Res-UNet网络的Dice系数最多提高了12.64%,达到0.7930,网络训练时间更短,对各类缺陷的分割精准度更优,证明该文算法在钢铁表面缺陷分割领域具有应用价值。
  • 图  1  Res-UNet网络结构

    图  2  残差模块和译码模块结构

    图  3  4类缺陷分割示例图

    图  4  缺陷真实标签和各网络预测结果示例图

    表  1  网络各层参数及特征图大小

    类型核大小/步长输出
    Conv1$ 7 \times 7/2 $$ 128 \times 800 \times 64 $
    Maxpool$ 3 \times 3/2 $$ 64 \times 400 \times 64 $
    Res Block$ 3 \times 3/1 $$ 64 \times 400 \times 64 $
    Res Block$ 3 \times 3/2 $$ 32 \times 200 \times 128 $
    Res Block$ 3 \times 3/2 $$ 16 \times 100 \times 256 $
    Res Block$ 3 \times 3/2 $$ 8 \times 50 \times 512 $
    Decode Block$ 3 \times 3/1 $$ 16 \times 100 \times 256 $
    Decode Block$ 3 \times 3/1 $$ 32 \times 200 \times 128 $
    Decode Block$ 3 \times 3/1 $$ 64 \times 400 \times 64 $
    Decode Block$ 3 \times 3/1 $$ 128 \times 800 \times 32 $
    Up Sample插值规模因子=2$ 256 \times 1600 \times 32 $
    Con2d$ 3 \times 3/1 $$ 256 \times 1600 \times 16 $
    Con2d$ 3 \times 3/1 $$ 256 \times 1600 \times 16 $
    下载: 导出CSV

    表  2  训练数据集缺陷分布情况(张)

    有损图片无损图片仅含划痕仅含埋渣仅含皮麟仅含氧化含两种缺陷
    486443115591383495380292
    下载: 导出CSV

    表  3  各网络训练超参数及实验结果

    网络名称训练周期(epoch)训练时间(min)Dice系数识别准确率(%)
    UNet (M1)805520.666669.02
    Res18-UNet (M2)406740.755180.47
    Res50-UNet (M3)5011910.756480.80
    Res18-UNet+ReCon (M4)405600.762982.83
    Res18-UNet+train-aug (M5)455540.764481.14
    Res18-UNet+BDloss (M6)355680.779787.54
    Res18-UNet+BDloss+ReCon (M7)455660.792188.21
    Res18-UNet+BDloss+train-aug (M8)405310.788189.23
    Res50-UNet+BDloss+train-aug (M9)5511720.791089.90
    Res18-UNet+BDloss+train-aug +ReCon (M10)294050.793088.89
    AlexNet (M11)30049553.80
    DenseNet (M12)3546980.56
    Xception (M13)10650.623852.86
    Mask RCNN (M14)352990.735062.96
    下载: 导出CSV
  • [1] 徐镪, 朱洪锦, 范洪辉, 等. 改进的YOLOv3网络在钢板表面缺陷检测研究[J]. 计算机工程与应用, 2020, 56(16): 265–272. doi: 10.3778/j.issn.1002-8331.2003-0232

    XU Qiang, ZHU Hongjin, FAN Honghui, et al. Study on detection of steel plate surface defects by improved YOLOv3 network[J]. Computer Engineering and Applications, 2020, 56(16): 265–272. doi: 10.3778/j.issn.1002-8331.2003-0232
    [2] BORSELLI A, COLLA V, VANNUCCI M, et al. A fuzzy inference system applied to defect detection in flat steel production[C]. The International Conference on Fuzzy Systems, Barcelona, Spain, 2010: 1–6.
    [3] XU Ke, XU Yang, ZHOU Peng, et al. Application of RNAMlet to surface defect identification of steels[J]. Optics and Lasers in Engineering, 2018, 105: 110–117. doi: 10.1016/j.optlaseng.2018.01.010
    [4] 汤勃, 孔建益, 王兴东, 等. 粗糙集理论的带钢表面缺陷图像的识别与分类[J]. 中国图象图形学报, 2011, 16(7): 1213–1218. doi: 10.11834/jig.20110718

    TANG Bo, KONG Jianyi, WANG Xingdong, et al. Recognition and classification for steel strip surface defect images based on rough set theory[J]. Journal of Image and Graphics, 2011, 16(7): 1213–1218. doi: 10.11834/jig.20110718
    [5] LIU Kun, WANG Heying, CHEN Haiyong, et al. Steel surface defect detection using a new haar–weibull-variance model in unsupervised manner[J]. IEEE Transactions on Instrumentation and Measurement, 2017, 66(10): 2585–2596. doi: 10.1109/TIM.2017.2712838
    [6] WANG Heying, ZHANG Jiawei, TIAN Ying, et al. A simple guidance template-based defect detection method for strip steel surfaces[J]. IEEE Transactions on Industrial Informatics, 2019, 15(5): 2798–2809. doi: 10.1109/TII.2018.2887145
    [7] KWON B K, WON J S, and KANG D J. Fast defect detection for various types of surfaces using random forest with VOV features[J]. International Journal of Precision Engineering and Manufacturing, 2015, 16(5): 965–970. doi: 10.1007/s12541-015-0125-y
    [8] CHU Maoxiang, LIU Xiaoping, GONG Rongfen, et al. Multi-class classification method using twin support vector machines with multi-information for steel surface defects[J]. Chemometrics and Intelligent Laboratory Systems, 2018, 176: 108–118. doi: 10.1016/j.chemolab.2018.03.014
    [9] YANG Changhui, ZHANG Jinxun, JI Gang, et al. Recognition of defects in steel surface image based on neural networks and morphology[C]. The Second Workshop on Digital Media and its Application in Museum & Heritages (DMAMH 2007), Chongqing, China, 2007: 72–77.
    [10] REN Ruoxu, HUNG T, and TAN K C. A generic deep-learning-based approach for automated surface inspection[J]. IEEE Transactions on Cybernetics, 2018, 48(3): 929–940. doi: 10.1109/TCYB.2017.2668395
    [11] 王宪保, 李洁, 姚明海, 等. 基于深度学习的太阳能电池片表面缺陷检测方法[J]. 模式识别与人工智能, 2014, 27(6): 517–523. doi: 10.3969/j.issn.1003-6059.2014.06.006

    WANG Xianbao, LI Jie, YAO Minghai, et al. Solar cells surface defects detection based on deep learning[J]. Pattern Recognition and Artificial Intelligence, 2014, 27(6): 517–523. doi: 10.3969/j.issn.1003-6059.2014.06.006
    [12] JIA Hongbin, MURPHEY Y L, SHI Jianjun, et al. An intelligent real-time vision system for surface defect detection[C]. The 17th International Conference on Pattern Recognition, Cambridge, UK, 2004: 239–242.
    [13] KIM M S, PARK T, and PARK P. Classification of steel surface defect using convolutional neural network with few images[C]. The 12th Asian Control Conference, Kitakyushu, Japan, 2019: 1398–1401.
    [14] MASCI J, GIUSTI A, CIRESAN D, et al. A fast learning algorithm for image segmentation with max-pooling convolutional networks[C]. 2013 IEEE International Conference on Image Processing, Melbourne, Australia, 2013: 2713–2717.
    [15] 郭兴宝. 基于机器学习的无缝钢管表面缺陷检测技术研究[D]. [硕士论文], 大连交通大学, 2019.

    GUO Baoxing. Research on surface defect detection technology of seamless steel tube based on machine learning[D]. [Master dissertation], Dalian Jiaotong University, 2019.
    [16] 甘胜丰. 带钢表面缺陷图像检测与分类方法研究[D]. [博士论文], 中国地质大学, 2013.

    GAN Shengfeng. Method of strip surface defect image detection and classification[D]. [Ph. D. dissertation], China University of Geosciences, 2013.
    [17] STEPHAN M and SANTRA A. Radar-based human target detection using deep residual U-net for smart home applications[C]. The 18th IEEE International Conference On Machine Learning And Applications (ICMLA), Boca Raton, USA, 2019: 175–182.
    [18] 储珺, 朱晓阳, 冷璐, 等. 引入通道注意力和残差学习的目标检测器[J]. 模式识别与人工智能, 2020, 33(10): 889–897. doi: 10.16451/j.cnki.issn1003-6059.202010003

    CHU Jun, ZHU Xiaoyang, LENG Lu, et al. Target detector with channel attention and residual learning[J]. Pattern Recognition and Artificial Intelligence, 2020, 33(10): 889–897. doi: 10.16451/j.cnki.issn1003-6059.202010003
    [19] ZENG Zitao, XIE Weihao, ZHANG Yunzhe, et al. RIC-Unet: An improved neural network based on unet for nuclei segmentation in histology images[J]. IEEE Access, 2019, 7: 21420–21428. doi: 10.1109/ACCESS.2019.2896920
    [20] 刘市祺, 孙晓波, 谢晓亮, 等. 基于区域建议网络和残差结构的导丝跟踪[J]. 模式识别与人工智能, 2019, 32(1): 36–42. doi: 10.16451/j.cnki.issn1003-6059.201901005

    LIU Shiqi, SUN Xiaobo, XIE Xiaoliang, et al. Guidewire tracking based on regional proposal network and residual structure[J]. Pattern Recognition and Artificial Intelligence, 2019, 32(1): 36–42. doi: 10.16451/j.cnki.issn1003-6059.201901005
    [21] SUDRE C H, LI Wenqi, VERCAUTEREN T, et al. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations[C]. The 3rd International Workshop on Deep Learning in Medical Image Analysis, Québec City, Canada, 2017: 240–248.
    [22] 张立恒, 王浩, 薛博维, 等. 基于改进D-LinkNet模型的高分遥感影像道路提取研究[J]. 计算机工程, 2021, 47(9): 288–296. doi: 10.19678/j.issn.1000-3428.0058977

    ZHANG Liheng, WANG Hao, XUE Bowei, et al. Research of road extraction from high-resolution remote sensing images based on improved D-LinkNet model[J]. Computer Engineering, 2021, 47(9): 288–296. doi: 10.19678/j.issn.1000-3428.0058977
  • 加载中
图(4) / 表(3)
计量
  • 文章访问数:  1206
  • HTML全文浏览量:  544
  • PDF下载量:  283
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-11-29
  • 修回日期:  2022-03-22
  • 网络出版日期:  2022-03-30
  • 刊出日期:  2022-05-25

目录

    /

    返回文章
    返回