一种基于几何关系的多分类器差异性度量及其在多分类器系统构造中的应用

梁绍一; 韩德强; 韩崇昭

doi:10.3724/SP.J.1004.2014.00449

一种基于几何关系的多分类器差异性度量及其在多分类器系统构造中的应用

doi: 10.3724/SP.J.1004.2014.00449

1.
西安交通大学电子与信息工程学院综合自动化研究所智能网络与网络安全教育部重点实验室西安 710049

基金项目:

国家重点基础研究发展计划（2013CB329405），国家自然科学基金（61104214，61203222），国家自然科学基金创新群体（61221063），中国博士后科学基金（201104670），中央高校基本科研业务费专项资金（xjj2012104）资助

详细信息

作者简介:
梁绍一西安交通大学电信学院硕士研究生. 主要研究方向为目标识别与信息融合.E-mail：shaoyi.liang2@stu.xjtu.edu.cn

通讯作者:
韩德强

计量
- 文章访问数: 1585
- HTML全文浏览量: 63
- PDF下载量: 914
- 被引次数: 0
出版历程
- 收稿日期: 2012-10-08
- 修回日期: 2013-06-14
- 刊出日期: 2014-03-20

A Novel Diversity Measure Based on Geometric Relationship and Its Application to Design of Multiple Classifier Systems

1.
Ministry of Education Key Lab For Intelligent Networks and Network Security (MOE KLINNS Lab), Institute of Integrated Automation, School of Electronics and Information Engineering, Xi'an Jiaotong University, Xi'an 710049

Funds:

Supported by National Basic Research Program of China (973 Program) (2013CB329405), National Natural Science Foundation of China (61104214, 61203222), Foundation for Innovative Research Groups of the National Natural Science Foundation of China (61221063), China Postdoctoral Science FoundationSpecial Fund (201104670), and the Fundamental Research Funds for the Central Universities (xjj2012104)

摘要

摘要: 多分类器系统是应对复杂模式识别问题的有效手段之一. 当子分类器之间存在差异性或互补性时，多分类器系统往往能够获得比单分类器更高的分类正确率. 因而差异性度量在多分类器系统设计中至关重要. 目前已有的差异性度量方法虽能够在一定程度上刻画分类器之间的差异，但在应用中可能出现诸如差异性淹没等问题. 本文提出了一种基于几何关系的多分类器差异性度量，并在此基础上提出了一种多分类器系统构造方法，同时通过实验对比了使用新差异性度量方法和传统方法对多分类器系统融合分类正确率的影响. 结果表明，本文所提出的差异性度量能够很好地刻画分类器之间的差异，能从很大程度上抑制差异性淹没问题，并能有效应用于多分类器系统构造.
- 多分类器系统 /
- 差异性度量 /
- 差异性淹没 /
- 几何中心
Abstract: The multiple classifier system is one of the effective means to resolve pattern recognition under complicated environments. When the member classifiers are diverse or complementary, multiple classifier systems can usually obtain higher classification accuracy compared with a single classifier. Thus, diversity measures are crucial to multiple classifier systems design. Though the existing diversity measures can, to some degree, describe the difference among classifiers, they may lead to problems like diversity submergence in some cases. In this paper, a novel multiple classifier diversity measure based on geometric relationship and a multiple classifier system constructing method based on the new diversity measure are proposed. It is experimentally shown that the proposed diversity measure can well describe the diversity among classifiers and effectively suppress the problem of diversity submergence. It can also be effectively used in designing multiple classifier systems.
- Multiple classifier system /
- diversity measure /
- diversity submergence /
- geometric center

HTML全文

参考文献(34)

[1]	Didaci L, Fumera G, Roli F. Diversity in classifier ensembles: fertile concept or dead end? In: Proceedings of the 11th International Workshop on Multiple Classifier Systems. Nanjing, China: Sprinter-Verlag, 2013. 37-48
[2]	[2] Bar A, Rokach L, Shani G, Shapira B, Schclar A. Improving simple collaborative filtering models using ensemble methods. In: Proceedings of the 11th International Workshop on Multiple Classifier Systems. Nanjing, China: Sprinter-Verlag, 2013. 1-12
[3]	[3] Sciarrone F. An extension of the Q diversity metric from single-label to multi-label and multi-ranking multiple classifier systems for pattern classification. In: Proceedings of the 2012 International Conference on Machine Learning and Cybernetics. Xi'an, China: IEEE, 2012. 6-10
[4]	[4] Radtke P, Granger E, Sabourin R, Gorodnichy D. Adaptive ensemble selection for face re-identification under class imbalance. In: Proceedings of the 11th International Workshop on Multiple Classifier Systems. Nanjing, China: Sprinter-Verlag, 2013. 95-108
[5]	Sun Liang, Han Chong-Zhao, Shen Jian-Jing, Dai Ning. Generalized rough set method for ensemble feature selection and multiple classifier fusion. Acta Automatica Sinica, 2008, 34(3): 298-304 (孙亮, 韩崇昭, 沈建京, 戴宁. 集成特征选择的广义粗集方法与多分类器融合. 自动化学报, 2008, 34(3): 298-304)
[6]	Zhang Cai-Po. Fuzzy Integral and Fusion of Multiple Classifiers Applying in Medical Diagnosis [Master dissertation], Tianjin University of Technology, China, 2010 (张彩坡. 模糊积分及多分类器融合在医疗诊断中的应用 [硕士学位论文], 天津理工大学, 中国, 2010)
[7]	Zhang Xue-Feng, Wang Peng-Hui, Feng Bo, Du Lan, Liu Hong-Wei. A new method to improve radar HRRP recognition and outlier rejection performance based on classifier combination. Acta Automatica Sinica, 2013, 39(1): 1-9 (张学峰, 王鹏辉, 冯博, 杜兰, 刘宏伟. 基于多分类器融合的雷达高分辨距离像目标识别与拒判新方法. 自动化学报, 2013, 39(1): 1-9)
[8]	[8] Xu L, Krzyzak A, Suen C Y. Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Transactions on Systems, Man, and Cybernetics, 1992, 22(3): 418-435
[9]	[9] Lam L, Suen C Y. Optimal combinations of pattern classifiers. Pattern Recognition Letters, 1995, 16(9): 945-954
[10]	Huang Y S, Suen C Y. A method of combining multiple experts for the recognition of unconstrained handwritten numerals. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1995, 17(1): 90-94
[11]	Verlinde P, Ghollet G. Comparing decision fusion paradigms using k-NN based classifiers, decision trees and logistic regression in a multi-modal identity verification application. In: Proceedings of the 2nd International Conference on Audio and Video Based Biometric Person Authentication. Washington D.C., USA: Springer-Verlag, 1999. 188-193
[12]	Lv Yue, Shi Peng-Fei, Zhao Yu-Ming. Voting principle for combination of multiple classifiers. Journal of Shanghai Jiaotong University, 2000, 34(5): 680-683 (吕岳, 施鹏飞, 赵宇明. 多分类器组合的投票表决规则. 上海交通大学学报, 2000, 34(5): 680-683)
[13]	Yang Yi, Han De-Qiang, Han Chong-Zhao. A novel diversity measure of multiple classifier systems based on distance of evidence. Acta Aeronautica et Astronautica Sinica, 2012, 33(6): 1093-1099(杨艺, 韩德强, 韩崇昭. 一种基于证据距离的多分类器差异性度量. 航空学报, 2012, 33(6): 1093-1099)
[14]	Kittler J, Hatef M, Duin R P W, Matas J. On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(3): 226-239
[15]	Ali K M, Pazzam M J. On the Link Between Error Correlation and Error Reduction in Decision Tree Ensembles, Technical Report 95-38, Department of Information and Computer Science, University of California Irvine, 1995
[16]	Tumer K, Ghosh J. Theoretical Foundations of Linear and Order Statistics Combiners for Neural Pattern Classifiers, Technical Report TR-95-02-98, Computer and Vision Research Center, University of Texas, Austin, 1995
[17]	Dietterich T G. An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Machine Learning, 2000, 40(2): 139-157
[18]	Kuncheva L I, Whitaker C J. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning, 2003, 51(2): 181-207
[19]	Shipp C A, Kuncheva L I. Relationship between combination methods and measures of diversity in combining classifiers. Information Fusion, 2002, 3(2): 135-148
[20]	Windeatt T. Diversity measures for multiple classifier system analysis and design. Information Fusion, 2005, 6(1): 21-36
[21]	Brown G, Wyatt J, Harris R, Yao X. Diversity creation methods: a survey and categorization. Information Fusion, 2005, 6(1): 5-20
[22]	Gal-Or M, May J H, Spangler W E. Assessing the predictive accuracy of diversity measures with domain-dependent, asymmetric misclassification costs. Information Fusion, 2005, 6(1): 37-48
[23]	Fan T G, Zhu Y, Chen J M. A new measure of classifier diversity in multiple classifier system. In: Proceedings of the 7th International Conference on Machine Learning and Cybernetics. Kunming, China, 2008. 18-21
[24]	Trawinski K, Quirin A, Cordn A. On the combination of accuracy and diversity measures for genetic selection of bagging fuzzy rule-based multiclassification systems. In: Proceedings of the 9th International Conference on Intelligent Systems Design and Applications. Pisa, Italy: IEEE, 2009. 121-127
[25]	Nascimento D S C, Canuto A M P, Silva L M M, Coelho A L V. Combining different ways to generate diversity in bagging models: an evolutionary approach. In: Proceedings of the 2011 International Joint Conference on Neural Networks. San Jose, USA: IEEE, 2011. 2235-2242
[26]	Kamel M S, Wanas N M. Data dependence in combining classifiers. In: Proceedings of the 4th International Workshop on Multiple Classifier Systems. Guilford, UK: Springer-Verlag, 2003. 1-14
[27]	Ho T K, Hull J J, Srihari S N. Decision combination in multiple classifier systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1994, 16(1): 66-75
[28]	Efron B, Tibshirani R. An Introduction to the Bootstrap. New York: Chapman Hall, 1993
[29]	Freund Y, Schapire R E. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 1997, 55(1): 119-139
[30]	Kuncheva L I, Whitaker C J. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning, 2003, 51(2): 181-207
[31]	Ma Cai-Hong, Dai Qin, Liu Shi-Bin. A hybrid PSO-ISODATA algorithm for remote sensing image segmentation. In: Proceedings of the 2012 International Conference on Industrial Control and Electronics Engineering (ICICEE). Xi'an, China: IEEE, 2012. 1371-1375
[32]	Zhang Xue-Gong. Pattern Recognition (3rd edition). Beijing: Tsinghua University Press, 2010(张学工. 模式识别 (第三版). 北京: 清华大学出版社, 2010)
[33]	Ho T K. The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(8): 832-844
[34]	Cheplygina V, Tax D M J. Pruned random subspace method for one-class classifiers. In: Proceedings of the 10th International Conference on Multiple Classifier Systems. Naples, Italy: Springer-Verlag, 2011. 96-105