基于小波时频图与轻量级卷积神经网络的螺栓连接损伤识别

卓德兵; 曹晖

doi:10.6052/j.issn.1000-4750.2021.02.0116

基于小波时频图与轻量级卷积神经网络的螺栓连接损伤识别

doi: 10.6052/j.issn.1000-4750.2021.02.0116

卓德兵^{1, 2, 3,},
曹晖^{1, 2, ,}

1.
重庆大学土木工程学院，重庆 400045
2.
重庆大学山地城镇建设与新技术教育部重点实验室，重庆 400045
3.
吉首大学土木工程与建筑学院，张家界 427000

基金项目: 湖南省教育厅科学研究一般项目(20C1512)

详细信息

作者简介:
卓德兵(1985−)，男，湖南人，讲师，博士，主要从事结构健康监测研究(E-mail: zhuodebing2004@163.com)

通讯作者:
曹　晖(1969−)，男，四川人，教授，博士，博导，主要从事结构健康监测研究(E-mail: caohui@cqu.edu.cn)

中图分类号: TU317+.1
计量
- 文章访问数: 333
- HTML全文浏览量: 132
- PDF下载量: 85
- 被引次数: 0
出版历程
- 收稿日期: 2021-02-02
- 修回日期: 2021-07-11
- 网络出版日期: 2021-07-22
- 刊出日期: 2021-09-13

DAMAGE IDENTIFICATION OF BOLT CONNECTIONS BASED ON WAVELET TIME-FREQUENCY DIAGRAMS AND LIGHTWEIGHT CONVOLUTIONAL NEURAL NETWORKS

ZHUO De-bing^{1, 2, 3
,},
CAO Hui^{1, 2
, ,}

1.
College of Civil Engineering, Chongqing University, Chongqing 400045, China
2.
MOE Key Laboratory of New Technology for Construction of Cities in Mountain Area, Chongqing University, Chongqing 400045, China
3.
School of civil engineering and architecture, Jishou University, Zhangjiajie 427000, China

摘要

摘要: 针对目前大型结构螺栓连接状态监测的困难，该文采用声音信号，提出了结合小波时频图与轻量级卷积神经网络MobileNetv2优势的螺栓松动识别方法。该方法通过对采集到的声音信号进行预处理和连续小波变换得到小波时频图，以小波时频图作为样本对轻量级卷积神经网络MobileNetv2进行训练，从而实现螺栓松动声音信号的识别。对一钢桁架模型的室外试验研究表明：该方法能实现对各种环境噪声信号，不同位置、数目和松动程度的螺栓松动声音信号的精准识别；该方法不仅识别准确率高、稳定性好，而且对计算和存储的要求低，便于应用于移动设备和嵌入式设备，为环境激励下大型复杂结构的损伤在线识别提供了新的思路。
- 螺栓连接 /
- 损伤识别 /
- 声音信号 /
- 小波时频图 /
- MobileNetv2
Abstract: In view of the difficulty of monitoring the states of bolt connections of large-scale structures, proposed a method for bolt looseness recognition by sound signals, which takes the advantages of the wavelet time-frequency analysis and the powerful image classification ability of the lightweight convolution neural network MobileNetv2. Continuous wavelet transform was carried out for the preprocessed sound signals to obtain the wavelet time-frequency diagrams. The lightweight convolutional neural network MobileNetv2 was trained using the wavelet time-frequency diagrams as samples. The trained model was used to identify the sound signals generated by loosen bolts. An outdoor test of a steel truss model showed that the proposed method could accurately recognize the sound signals of loosen bolts at different positions, with different numbers and different degrees of looseness, and with various environmental noise signals. This novel method has high identification accuracy and good stability, and requires low calculation cost and storage space. It can be carried out easily by mobile devices and embedded devices, providing a new idea for online damage recognition of large and complex structures under environmental excitation.
- bolt connection /
- damage identification /
- sound signal /
- wavelet time-frequency diagram /
- MobileNetv2

HTML全文

图 1 MobileNetv2卷积过程

Figure 1. Convolution process of MobileNetv2

下载: 全尺寸图片幻灯片

图 2 小波时频图- MobileNetv2螺栓松动损伤识别方法结构图

Figure 2. Structural diagram of bolt loosening damage identification method based on wavelet time frequency diagrams and MobileNetv2

下载: 全尺寸图片幻灯片

图 3 试验模型及测试系统

Figure 3. Test model and test system

下载: 全尺寸图片幻灯片

图 4 蝉鸣声信号图及短时能量最大帧的分析结果

Figure 4. Cicada sound signal diagram and analysis results of frame with the largest short-time energy

下载: 全尺寸图片幻灯片

图 5 下雨声信号图及短时能量最大帧的分析结果

Figure 5. Rain sound signal diagram and analysis results of frame with the largest short-time energy

下载: 全尺寸图片幻灯片

图 6 工况6信号图及短时能量最大帧的分析结果

Figure 6. Signal diagram of Case 6 and analysis results of frame with the largest short-time energy

下载: 全尺寸图片幻灯片

图 7 样本帧长对MobileNetv2模型识别结果的影响

Figure 7. Influence of sample frame length on recognition results of MobileNetv2 model

下载: 全尺寸图片幻灯片

图 8 MobileNetv2模型与不同CNN模型的训练结果对比

Figure 8. Comparison of training results between MobileNetv2 and other CNN models

下载: 全尺寸图片幻灯片

图 9 混淆矩阵

Figure 9. Confusion matrix

下载: 全尺寸图片幻灯片

表 1 网络结构模型

Table 1. Network structure model

输入	操作	扩展因子	输出特征矩阵深度	重复操作次数	第一层步长
224²×3	Conv2d	−	32	1	2
112²×32	bottleneck	1	16	1	1
112²×16	bottleneck	6	24	2	2
56²×24	bottleneck	6	32	3	2
28²×32	bottleneck	6	64	4	2
14²×64	bottleneck	6	96	3	1
14²×96	bottleneck	6	160	3	2
7²×160	bottleneck	6	320	1	1
7²×320	conv2d 1×1	−	1280	1	1
7²×1280	avgpool 7×7	−	−	1	−
1×1×1280	conv2d 1×1	−	k	−	−

下载: 导出CSV

表 2 损伤工况

Table 2. Damage cases

工况	松动位置	松动个数	松动程度/圈
工况1	J₁₅	1	0.5
工况2	J₁₅	1	1.0
工况3	J₁₅	1	1.5
工况4	J₁₅	2	0.5
工况5	J₁₅	3	0.5
工况6	J₂₄	1	0.5
工况7	J₂₄	2	0.5
工况8	J₃₆	1	0.5
工况9	J₃₆	2	0.5
工况10	J₄₂	1	0.5

下载: 导出CSV

表 3 计算机平台及环境配置

Table 3. Computer platform and environment configuration

软硬件平台	型号参数
操作系统	Windows 10 64位系统
CPU	Intel Xeon E5-2650 V4
GPU	NVIDA TESLA P100 16G
内存	16 G
编程环境	Anaconda 3
	CUDA 10.0
	Cudnn 7.4.1.5
	Python 3.7 64位
	Tensorflow-gpu 1.13.2
	Numpy 1.17.4
	Keras 2.1.5

下载: 导出CSV

表 4 各模型迭代100次的分类准确率和损失值

Table 4. Classification accuracy and loss of each model after 100 iterations

项目	准确率		损失值
项目	训练集/(%)	验证集/(%)	训练集	验证集
AlexNet	99.34	98.63	0.04	0.07
GoogleNet	99.70	99.38	0.02	0.03
VGG16	100.00	99.60	0.00	0.01
ResNet50	100.00	99.90	0.00	0.00
MobileNetv2	100.00	100.00	0.00	0.00

下载: 导出CSV

表 5 各模型100次迭代后的总参数量和权重文件大小

Table 5. Total parameters and weight file size of each model after 100 iterations

项目	总参数量	权重文件大小
AlexNet	1.5×10⁷	56 M
GoogleNet	7.0×10⁷	268 M
VGG16	1.0×10⁷	39 M
ResNet50	2.1×10⁷	90 M
MobileNetv2	0.2×10⁷	9 M

下载: 导出CSV

表 6 分类精确率、召回率和特异度

Table 6. Precision, recall and specificity

标签	精确率	召回率	特异度
A	1.000	0.98	1.000
B	1.000	1.00	1.000
C	0.873	0.96	0.991
D	1.000	1.00	1.000
E	1.000	0.94	1.000
F	1.000	1.00	1.000
G	1.000	1.00	1.000
H	1.000	1.00	1.000
I	0.907	0.98	0.993
J	1.000	0.84	1.000
K	1.000	1.00	1.000
L	0.943	1.00	0.996
M	1.000	1.00	1.000
N	0.980	0.98	0.999
O	1.000	1.00	1.000
P	1.000	1.00	1.000

下载: 导出CSV

参考文献(33)

[1]	Huda F, Kajiwara I, Hosoya N, et al. Bolt loosening analysis and diagnosis by non-contact laser excitation vibration tests [J]. Mechanical Systems & Signal Processing, 2013, 40(2): 589 − 604.
[2]	Razi P, Esmaeel R A, Taheri F. Improvement of a vibration-based damage detection approach for health monitoring of bolted flange joints in pipelines [J]. Structural Health Monitoring, 2013, 12(3): 207 − 224. doi: 10.1177/1475921713479641
[3]	侯博文, 李佳静, 高亮, 等. 基于脉冲激励下钢轨振动响应的扣件失效识别方法[J]. 工程力学, 2021, 38(2): 122 − 133. doi: 10.6052/j.issn.1000-4750.2020.04.0210 Hou Bowen, Li Jiajing, Gao Liang, et al. Failure identification method of fastening system based on rail response under pulse excitation [J]. Engineering Mechanics, 2021, 38(2): 122 − 133. (in Chinese) doi: 10.6052/j.issn.1000-4750.2020.04.0210
[4]	Todd M D, Nichols J M, Nichols C J, et al. An assessment of modal property effectiveness in detecting bolted joint degradation: theory and experiment [J]. Journal of Sound & Vibration, 2004, 275(3/4/5): 1113 − 1126.
[5]	徐超, 周帮友, 刘信恩, 等. 机械螺栓连接状态监测和辨识方法研究进展[J]. 强度与环境, 2009, 36(2): 28 − 36. doi: 10.3969/j.issn.1006-3919.2009.02.004 Xu Chao, Zhou Bangyou, Liu Xinen, et al. A review of vibration-based condition monitoring and indentification for mechanical bolted joints [J]. Structure & Environment Engineering, 2009, 36(2): 28 − 36. (in Chinese) doi: 10.3969/j.issn.1006-3919.2009.02.004
[6]	翁顺, 朱宏平. 基于有限元模型修正的土木结构损伤识别方法[J]. 工程力学, 2021, 38(3): 1 − 16. doi: 10.6052/j.issn.1000-4750.2020.06.ST02 Weng Shun, Zhu Hongping. Damage identification of civil structures based on finite element model updating [J]. Engineering Mechanics, 2021, 38(3): 1 − 16. (in Chinese) doi: 10.6052/j.issn.1000-4750.2020.06.ST02
[7]	Sun F P, Chaudhry Z, Liang C, et al. Truss structure integrity identification using PZT sensor-actuator [J]. Journal of Intelligent Material Systems and Structures, 1995, 6(1): 134 − 139. doi: 10.1177/1045389X9500600117
[8]	Baghalian A, Senyurek V Y, Tashakori S, et al. A novel nonlinear acoustic health monitoring approach for detecting loose bolts [J]. Journal of Nondestructive Evaluation, 2018, 37(24): 1 − 10.
[9]	Amerini F, Meo M. Structural health monitoring of bolted joints using linear and nonlinear acoustic/ultrasound methods [J]. Structural Health Monitoring, 2011, 10(6): 659 − 672. doi: 10.1177/1475921710395810
[10]	朱宏平, 王丹生, 张俊兵. 基于压电阻抗技术的结构损伤识别基本理论及其应用[J]. 工程力学, 2008, 25(增刊 2): 34 − 43. Zhu Hongping, Wang Dansheng, Zhang Junbing. Theory and application of structure damage detection based on piezoelectric impedance technique [J]. Engineering Mechanics, 2008, 25(Suppl 2): 34 − 43. (in Chinese)
[11]	王硕朋, 杨鹏, 孙昊. 基于声音位置指纹的室内声源定位方法[J]. 北京工业大学学报, 2017, 43(2): 224 − 229. Wang Shuopeng, Yang Peng, Sun Hao. Indoor sound-position fingerprint method based on scenario analysis [J]. Journal of Beijing University of Technology, 2017, 43(2): 224 − 229. (in Chinese)
[12]	赵书涛, 张佩, 申路, 等. 高压断路器振声联合故障诊断方法[J]. 电工技术学报, 2014, 29(7): 216 − 221. doi: 10.3969/j.issn.1000-6753.2014.07.028 Zhao Shutao, Zhang Pei, Shen Lu, et al. Vibration and acoustic joint mechanical fault diagnosis method of high voltage circuit breakers [J]. Transactions of China Electrotechnical Society, 2014, 29(7): 216 − 221. (in Chinese) doi: 10.3969/j.issn.1000-6753.2014.07.028
[13]	Garcia-Perez A, Romero-Troncoso R J, Cabal-Yepez E, et al. Application of high-resolution spectral analysis for identifying faults in induction motors by means of sound [J]. Journal of Vibration & Control, 2012, 18(11): 1585 − 1594.
[14]	Amarnath M, Sugumaran V, Kumar H. Exploiting sound signals for fault diagnosis of bearings using decision tree [J]. Measurement, 2013, 46(3): 1250 − 1256. doi: 10.1016/j.measurement.2012.11.011
[15]	Madhusudana C K, Kumar H, Narendranath S. Fault diagnosis of face milling tool using decision tree and sound signal [J]. Materials Today Proceedings, 2018, 5(5): 12035 − 12044. doi: 10.1016/j.matpr.2018.02.178
[16]	Lu S, Wang X, He Q, et al. Fault diagnosis of motor bearing with speed fluctuation via angular resampling of transient sound signals [J]. Journal of Sound & Vibration, 2016, 385(22): 16 − 32.
[17]	瞿金秀, 杨飞宇, 张周锁, 等. 基于声音信号的结构损伤识别方法[J]. 振动、测试与诊断, 2014, 34(4): 638 − 643. doi: 10.3969/j.issn.1004-6801.2014.04.007 Qu Jinxiu, Yang Feiyu, Zhang Zhousuo, et al. Structural damage identification method based on sound signal [J]. Journal of Vibration, Measurement & Diagnosis, 2014, 34(4): 638 − 643. (in Chinese) doi: 10.3969/j.issn.1004-6801.2014.04.007
[18]	Dat T H, Terence N W Z, Dennis J W, et al. Generalized gaussian distribution kullback-leibler kernel for robust sound event recognition [C]. Florence: ICASSP 2014 − 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2014: 5949 − 5953.
[19]	Rawat W, Wang Z. Deep convolutional neural networks for image classification: a comprehensive review [J]. Neural Computation, 2017, 29(9): 2352 − 2449. doi: 10.1162/neco_a_00990
[20]	Krizhevsky A, Sutskever I, Hinton G. Imagenet classification with deep convolutional neural networks [J]. Communications of the Acm, 2017, 60(6): 84 − 90. doi: 10.1145/3065386
[21]	Szegedy C, Liu W, Jia Y Q, et al . Going deeper with convolutions [C]. Boston, MA: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015: 1 − 9.
[22]	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition [C]. Las Vegas, NV, USA: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 770 − 778.
[23]	鲍跃全, 李惠. 人工智能时代的土木工程[J]. 土木工程学报, 2019, 52(5): 5 − 15. Bao Yuequan, Li Hui. Artificial Intelligence for civil engineering [J]. Journal of Civil Engineering, 2019, 52(5): 5 − 15. (in Chinese)
[24]	何定桥, 王鹏军, 杨军. 深度神经网络在EMD虚假分量识别中的应用[J]. 工程力学, 2021, 38(增刊 1): 195 − 201. doi: 10.6052/j.issn.1000-4750.2020.04.S036 He Dingqiao, Wang Pengjun, Yang Jun. Application of deep neural networks in EMD false component identification [J]. Engineering Mechanics, 2021, 38(Suppl 1): 195 − 201. (in Chinese) doi: 10.6052/j.issn.1000-4750.2020.04.S036
[25]	杨娜, 张翀, 李天昊. 基于无人机与计算机视觉的中国古建筑木结构裂缝监测系统设计[J]. 工程力学, 2021, 38(3): 27 − 39. doi: 10.6052/j.issn.1000-4750.2020.04.0263 Yang Na, Zhang Chong, Li Tianhao. Design of crack monitoring system for Chinese ancient wooden buildings based on UAV and CV [J]. Engineering Mechanics, 2021, 38(3): 27 − 39. (in Chinese) doi: 10.6052/j.issn.1000-4750.2020.04.0263
[26]	鄢仁武, 林穿, 高硕勋, 等. 基于小波时频图和卷积神经网络的断路器故障诊断分析[J]. 振动与冲击, 2020, 39(10): 198 − 205. Yan Renwu, Lin Chuan, Gao Shuoxun, et al. Fault diagnosis and analysis of circuit breaker based on wavelet time-frequency representations and convolution neural network [J]. Journal of Vibration and Shock, 2020, 39(10): 198 − 205. (in Chinese)
[27]	Kim S, Choi J H. Convolutional neural network for gear fault diagnosis based on signal segmentation approach [J]. Structural Health Monitoring, 2019, 18(5/6): 1401 − 1415.
[28]	Saksamudre S K, Shrishrimal P P, Deshmukh R R. A review on different approaches for speech recognition system [J]. International Journal of Computer Applications, 2015, 115(22): 23 − 28. doi: 10.5120/20284-2839
[29]	Abeer J. A review of deep learning based methods for acoustic scene classification [J]. Applied Sciences, 2020, 10(6): 1 − 16.
[30]	张重远, 罗世豪, 岳浩天, 等. 基于Mel时频谱-卷积神经网络的变压器铁芯声纹模式识别方法[J]. 高电压技术, 2020, 46(2): 413 − 423. Zhang Zhongyuan, Luo Shihao, Yue Haotian, et al. Pattern recognition of acoustic signals of transformer core based on mel-spectrum and CNN [J]. High Voltage Engineering, 2020, 46(2): 413 − 423. (in Chinese)
[31]	Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks [J]. Science, 2006, 313(5786): 504 − 507. doi: 10.1126/science.1127647
[32]	Howard A G, Zhu M, Chen B, et al. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications [J]. arXiv preprint arXiv:1704.04861, 2017.
[33]	Sandler M, Howard A, Zhu M, et al. MobileNetv2: Inverted residuals and linear bottlenecks [C]. Salt Lake City, UT, USA: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018: 4510 − 4520.