摘要
Industrial Internet of Things(IoT)connecting society and industrial systems represents a tremendous and promising paradigm shift.With IoT,multimodal and heterogeneous data from industrial devices can be easily collected,and further analyzed to discover device maintenance and health related potential knowledge behind.IoT data-based fault diagnosis for industrial devices is very helpful to the sustainability and applicability of an IoT ecosystem.But how to efficiently use and fuse this multimodal heterogeneous data to realize intelligent fault diagnosis is still a challenge.In this paper,a novel Deep Multimodal Learning and Fusion(DMLF)based fault diagnosis method is proposed for addressing heterogeneous data from IoT environments where industrial devices coexist.First,a DMLF model is designed by combining a Convolution Neural Network(CNN)and Stacked Denoising Autoencoder(SDAE)together to capture more comprehensive fault knowledge and extract features from different modal data.Second,these multimodal features are seamlessly integrated at a fusion layer and the resulting fused features are further used to train a classifier for recognizing potential faults.Third,a two-stage training algorithm is proposed by combining supervised pre-training and fine-tuning to simplify the training process for deep structure models.A series of experiments are conducted over multimodal heterogeneous data from a gear device to verify our proposed fault diagnosis method.The experimental results show that our method outperforms the benchmarking ones in fault diagnosis accuracy.
基金
supported in part by the National Key Research and Development Program of China(No.2018YFB1003700)
in part by the National Natural Science Foundation of China(No.61836001)。
作者简介
Corresponding author:Huifang Li,Email:huifang@bit.edu.cn,received her B.S.degree in Automatic Control from Nanjing University of Aeronautical and Astronautics,Nanjing,China,in 1988,and M.S.and Ph.D.degrees in Systems Engineering from Xi’an Jiaotong University,Xi’an,China in 1994 and 2000 respectively.She was a postdoctoral researcher in Department of Automation,Tsinghua University,Beijing,China in 2001-2003.She worked as a researcher in Computer Integrated Manufacturing Research Unit,National University of Ireland,Galway in 2007-2008.She was a visiting scholar in University of California,Berkeley,California,United States from May to Aug.2012,and Research Center of Enterprise Integration and Urban Systems Engineering,University of Toronto,Ontario,Canada from Jun.to Dec.2016.She is currently an associate professor in School of Automation,Beijing Institute of Technology,Beijing,China.Her research interests are machine learning,industrial fault diagnosis,cloud computing and task scheduling;Jianghang Huang,received the B.S.degree from China Agricultural University,Beijing,China,in 2019.He is currently working toward a Master’s degree in Beijing Institute of Technology,Beijing,China.His current research interests include workflow scheduling and reinforcement learning;Jingwei Huang,received his B.S.degree in Electrical Engineering and Automation from Beijing Institute of Technology,Beijing,China,in 2020.He is currently working toward a Master’s degree in Electronic Information at Beijing Institute of Technology,Beijing,China.His current research interest is intelligent optimization;Senchun Chai,received the B.S.Eng and M.S.Eng degrees in School of Automation from the Beijing Institute of Technology,Beijing,China,in 2001 and 2004,respectively,and the Ph.D.degree and Postdoc fellowship from the School of Electronics,University of Glamorgan,Pontypridd,U.K.,in 2007 and 2009,respectively.He has been a researcher with Cranfield University,Cranfield,U.K.,from 2009 to 2010,and a visiting scholar with the University of Illinois at Urbana-Champaign,Urbana,IL,USA,from January 2010 to May 2010.He is now a professor with the Beijing Institute of Technology.His research interests include design of Industrial Internet,network security,multi-agent systems,networked control systems,wireless sensor network and aircraft control;Leilei Zhao,received her B.S.degree from North China Electric Power University,Baoding,China,in 2017.She received her Master’s degree from Beijing Institute of Technology,Beijing,China,in 2019.Her current research interests include deep learning and ad recommendation systems;Yuanqing Xia,received the M.S.degree in fundamental mathematics from Anhui University,Hefei,China,in 1998,and the Ph.D.degree in control theory and control engineering from the Beijing University of Aeronautics and Astronautics,Beijing,China,in 2001.He was a Post-Doctoral Research Associate with the Institute of Systems Science,Academy of Mathematics and Systems Science,Chinese Academy of Sciences,Beijing,from 2002 to 2003.From 2004 to 2006,he was with the University of Glamorgan,Pontypridd,U.K.,as a Research Fellow.From 2007 to 2008,he was a Guest Professor with Innsbruck Medical University,Innsbruck,Austria.He is currently a Professor with the School of Automation,Beijing Institute of Technology,Beijing.His research interests are cloud control systems,networked control systems,robust control and signal processing,active disturbance rejection control,unmanned system control,and flight control.
