Abstract：To build a classifier model of high dimensional data, the traditional deep brief networks (DBN) modeling method suffers from large network load and high algorithm complexity. In this work, the data dimension is reduced based on the nonlinear PCA (NPCA), then a new DBN classifier with nonlinear feature extraction pre-processing mechanism is proposed where the nonlinear feature is extracted as the network input to the DBN. With the entropy theory, the advantage of the improved DBN is proved in terms of network structure and algorithm complexity. A PM2.5 concentration prediction and diagnosis problem is employed to exemplify applications of the proposed methods. Compared with the traditional classifier, it shows the advantage of the proposed method in modeling accuracy and convergence speed.
高月, 宿翀, 李宏光. 一类基于非线性PCA和深度置信网络的混合分类器及其在PM2.5浓度预测和影响因素诊断中的应用. 自动化学报, 2018, 44(2): 318-329.
GAO Yue, SU Chong, LI Hong-Guang. A Kind of Deep Belief Networks Based on Nonlinear Features Extraction with Application to PM2.5 Concentration Prediction and Diagnosis. Acta Automatica Sinica, 2018, 44(2): 318-329.
1 Saki F, Kehtarnavaz N. Online frame-based clustering with unknown number of clusters. Pattern Recognition, 2016, 57:70-83 2 Li H, Chung F L, Wang S T. A SVM based classification method for homogeneous data. Applied Soft Computing, 2015, 36:228-235 3 Embrechts M J, Rossi F, Schleif F M, Lee J A. Advances in artificial neural networks, machine learning, and computational intelligence (ESANN 2013). Neurocomputing, 2014, 141:1-2 4 Zhang Y P, Li X, Zhang Z F, Wu F, Zhao L M. Deep learning driven blockwise moving object detection with binary scene modeling. Neurocomputing, 2015, 168:454-463 5 Shang C, Yang F, Huang D X, Lyu W X. Data-driven soft sensor development based on deep learning technique. Journal of Process Control, 2014, 24 (3):223-233 6 Gao Ying-Ying, Zhu Wei-Bin. Deep neural networks with visible intermediate layers. Acta Automatica Sinica, 2015, 41 (9):1627-1637(高莹莹, 朱维彬. 深层神经网络中间层可见化建模. 自动化学报, 2015, 41 (9):1627-1637) 7 Ding Ke, Tan Ying. A review on general purpose computing on GPUs and its applications in computational intelligence. CAAI Transactions on Intelligent Systems, 2015, 10 (1):1-11(丁科, 谭营. GPU通用计算及其在计算智能领域的应用. 智能系统学报, 2015, 10 (1):1-11) 8 Shen F R, Chao J, Zhao J X. Forecasting exchange rate using deep belief networks and conjugate gradient method. Neurocomputing, 2015, 167:243-253 9 Huang S, Yang D, Ge Y X, Zhang X H. Combined supervised information with PCA via discriminative component selection. Information Processing Letters, 2015, 115 (11):812-816 10 Gan Jun-Ying, Li Chun-Zhi. Face Recognition based on wavelet transform, two-dimensional principal component analysis and independent component analysis. Pattern Recognition and Artificial Intelligence, 2007, 20 (3):377-381(甘俊英, 李春芝. 基于小波变换、二维主元分析与独立元分析的人脸识别方法. 模式识别与人工智能, 2007, 20 (3):377-381) 11 Tang Jian, Chai Tian-You, Yu Wen, Zhao Li-Jie. On-line KPLS algorithm with application to ensemble modeling parameters of mill load. Acta Automatica Sinica, 2013, 39 (5):471-486(汤健, 柴天佑, 余文, 赵立杰. 在线KPLS建模方法及在磨机负荷参数集成建模中的应用. 自动化学报, 2013, 39 (5):471-486) 12 Cobourn W G. An enhanced PM2.5 air quality forecast model based on nonlinear regression and back-trajectory concentrations. Atmospheric Environment, 2010, 44 (25):3015-3023 13 Voukantsis D, Karatzas K, Kukkonen J, Räsänen T, Karppinen A, Kolehmainen M. Intercomparison of air quality data using principal component analysis, and forecasting of PM10 and PM2.5 concentrations using artificial neural networks, in Thessaloniki and Helsinki. Science of the Total Environment, 2011, 409 (7):1266-1276 14 Xia D H, Jiang B F, Xie Y L. Modeling and analysis of PM2.5 generation for key factors identification in China. Atmospheric Environment, 2016, 134:208-216 15 De Ridder K, Kumar U, Lauwaet D, Blyth L, Lefebvre W. Kalman filter-based air quality forecast adjustment. Atmospheric Environment, 2012, 50:381-384 16 de Gennaro G, Trizio L, Gilio A D, Pey J, Pérez N, Cusack M, Alastuey A, Querol X. Neural network model for the prediction of PM10 daily concentrations in two sites in the Western Mediterranean. Science of The Total Environment, 2013, 463-464:875-883 17 Feng X, Li Q, Zhu Y J, Hou J X, Jin L Y, Wang J J. Artificial neural networks forecasting of PM2.5 pollution using air mass trajectory based geographic model and wavelet transformation. Atmospheric Environment, 2015, 107:118-128 18 Tan S F, Mayrovouniotis M L. Reducing data dimensionality through optimizing neural network inputs. AIChE Journal, 1995, 41 (6):1471-1480 19 Hinton G E, Osindero S, The Y W. A fast learning algorithm for deep belief nets. Neural Computation, 2006, 18 (7):1527-1554 20 Qiao Jun-Fei, Pan Guang-Yuan, Han Hong-Gui. Design and application of continuous deep belief network. Acta Automatica Sinica, 2015, 41 (12):2138-2146(乔俊飞, 潘广源, 韩红桂. 一种连续型深度信念网的设计与应用. 自动化学报, 2015, 41 (12):2138-2146) 21 Li Er-Guo, Yu Jin-Shou. An input-training neural network based nonlinear principal component analysis approach for fault diagnosis. Control and Decision, 2003, 18 (2):229-232(李尔国, 俞金寿. 一种基于输入训练神经网络的非线性PCA故障诊断方法. 控制与决策, 2003, 18 (2):229-232) 22 He R, Hu B G, Yuan X T, Zheng W S. Principal component analysis based on non-parametric maximum entropy. Neurocomputing, 2010, 73 (10-12):1840-1852 23 Wu Yan, Zhang Li-Ming. A survey of research work on neural network generalization and structure optimization algorithms. Application Research of Computers, 2002, 19 (6):21-25, 84(武妍, 张立明. 神经网络的泛化能力与结构优化算法研究. 计算机应用研究, 2002, 19 (6):21-25, 84) 24 Deco G, Finnoff W, Zimmermann H G. Unsupervised mutual information criterion for elimination of overtraining in supervised multilayer networks. Neural Computation, 1995, 7 (1):86-107 25 Wu Xin-Gen, Lv Wei-Xue. A neural network rule expression based on information entropy. Computer Engineering, 1996, 22 (5):46-51(吴新根, 吕维雪. 一种基于信息熵的神经网络规则表示. 计算机工程, 1996, 22 (5):46-51) 26 Sánchez D, Melin P, Castillo O. Optimization of modular granular neural networks using a hierarchical genetic algorithm based on the database complexity applied to human recognition. Information Sciences, 2015, 309:73-101 27 Wang L T, Wei Z, Yang J, Zhang Y, Zhang F F, Su J, et al. The 2013 severe haze over southern Hebei, China:model evaluation, source apportionment, and policy implications. Atmospheric Chemistry and Physics Discussions, 2013, 13 (11):28395-28451 28 Peng K X, Zhang K, Li G. Online contribution rate based fault diagnosis for nonlinear industrial processes. Acta Automatica Sinica, 2014, 40 (3):423-430 29 Yao L, Yang L X, Yuan Q, Yan C, Dong C, Meng C P, et al. Sources apportionment of PM2.5 in a background site in the North China Plain. Science of The Total Environment, 2016, 541:590-598
2018, Vol. 44 
