基于相似度衡量的决策树自适应迁移

王雪松; 潘杰程; 玉虎; 曹戈

doi:10.3724/SP.J.1004.2013.02186

基于相似度衡量的决策树自适应迁移

doi: 10.3724/SP.J.1004.2013.02186

1.
中国矿业大学信息与电气工程学院徐州 221116

基金项目:

国家自然科学基金（61072094，61273143），教育部博士点基金（20110095110016，20120095110025），江苏省研究生科研创新计划（CXZZ12_0932）资助

详细信息

作者简介:
王雪松中国矿业大学教授. 主要研究方向为机器学习，生物信息学. 本文通信作者. E-mail：wangxuesongcumt@163.com

计量
- 文章访问数: 1553
- HTML全文浏览量: 110
- PDF下载量: 2256
- 被引次数: 0
出版历程
- 收稿日期: 2012-03-26
- 修回日期: 2012-09-18
- 刊出日期: 2013-12-20

Self-adaptive Transfer for Decision Trees Based on Similarity Metric

1.
School of Information and Electrical Engineering, China University of Mining and Technology, Xuzhou 221116

Funds:

Supported by National Natural Science Foundation of China (61072094, 61273143), Special Grade of the Financial Supportfrom China Postdoctoral Science Foundation (20110095110016,20120095110025), and College Graduate Research and Innovation Projects of Jiangsu Province (CXZZ12 0932)

摘要

摘要: 如何解决迁移学习中的负迁移问题并合理把握迁移的时机与方法，是影响迁移学习广泛应用的关键点. 针对这个问题，提出一种基于相似度衡量机制的决策树自适应迁移方法（Self-adaptive transfer for decision trees based on a similarity metric，STDT）. 首先，根据源任务数据集是否允许访问，自适应地采用成分预测概率或路径预测概率对决策树间的相似性进行判定，其亲和系数作为量化衡量关联任务相似程度的依据. 然后，根据多源判定条件确定是否采用多源集成迁移，并将相似度归一化后依次分配给待迁移源决策树作为迁移权值. 最后，对源决策树进行集成迁移以辅助目标任务实现决策. 基于UCI 机器学习库的仿真结果说明，与多源迁移加权求和算法（Weighted sum rule，WSR）和MS-TrAdaBoost 相比，STDT 能够在保证决策精度的前提下实现更为快速的迁移.
- 迁移学习 /
- 决策树 /
- 相似度 /
- 亲和系数
Abstract: Negative transfer, transfer opportunity and transfer method are the most key problems affecting the learning performance of transfer learning. In order to solve these problems, a self-adaptive transfer for decision trees based on a similarity metric (STDT) is proposed. At first, according to whether the source task datasets to be allowed to access, a prediction probability based on constituents or paths is adaptively used to calculate the affinity coefficient between decision trees, which can quantify the similarity degree of related tasks. Secondly, a judgment condition of multi-sources is used to determine whether the multi-source integrated transfer is adopted. If do, the similarity degrees are normalized, which can be viewed as transfer weights assigned to source decision trees to be transferred. At last, the source decision trees are transferred to assist the target task in making decisions. Simulation results on UCI and text classification datasets illustrate that, compared with multi-source transfer algorithms, i.e., weighted sum rule (WSR) and MS-TrAdaBoost, the proposed STDT has a faster transfer speed with the assurance of high decision accuracy.
- Transfer learning /
- decision tree /
- similarity metric /
- affinity coefficient

HTML全文

参考文献(17)

[1]	Pan S J, Yang Q. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359
[2]	Ceci M, Appice A, Barile N, Malerba D. Transductive learning from relational data. In: Proceedings of the 5th International Conference on Machine Learning and Data Mining in Pattern Recognition. Leipzig, Germany: Springer-Verlag, 2007. 324-338
[3]	Dai W Y, Yang Q, Xue G R, Yu Y. Boosting for transfer learning. In: Proceedings of the 24th International Conference on Machine Learning. Corvalis, USA: ACM, 2007. 193-200
[4]	Yao Y, Doretto G. Boosting for transfer learning with multiple sources. In: Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, USA: IEEE, 2010. 1855-1862
[5]	Hong Jia-Ming, Yin Jian, Huang Yun, Liu Yu-Bao, Wang Jia-Hai. TrSVM: a transfer learning algorithm using domain similarity. Journal of Computer Research and Development, 2011, 48(10): 1823-1830(洪佳明, 印鉴, 黄云, 刘玉葆, 王甲海. TrSVM: 一种基于领域相似性的迁移学习算法. 计算机研究与发展, 2011, 48(10): 1823-1830)
[6]	Zadrozny B. Learning and evaluating classifiers under sample selection bias. In: Proceedings of the 21st International Conference on Machine Learning. Banff, Canada: ACM, 2004. 903-910
[7]	Torrey L, Shavlik J, Walker T, Malin R. Relational macros for transfer in reinforcement learning. In: Proceedings of the 17th International Conference on Inductive Logic Programming. Corvallis, USA: Springer-Verlag, 2008. 254-268
[8]	Arnold A, Nallapati R, Cohen W W. Exploiting feature hierarchy for transfer learning in named entity recognition. In: Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. Columbus, USA: ACL, 2008. 245-253
[9]	Wang H Y, Yang Q. Transfer learning by structural analogy. In: Proceedings of the 25th AAAI Conference on Artificial Intelligence and the 23rd Innovative Applications of Artificial Intelligence Conference. San Francisco, USA: AAAI Press, 2011. 513-518
[10]	Koçcćer B, Arslan A. Genetic transfer learning. Expert Systems with Applications, 2010, 37(10): 6997-7002
[11]	Mihalkova L, Huynh T, Mooney R J. Mapping and revising Markov logic networks for transfer learning. In: Proceedings of the 22nd AAAI Conference on Artificial Intelligence and the 19th Innovative Applications of Artificial Intelligence Conference. Vancouver, Canada: AAAI, 2007. 608-614
[12]	Yu K, Chu W. Gaussian process models for link analysis and transfer learning. In: Proceedings of the 2007 Annual Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates, 2007. 1-8
[13]	Lee J W, Giraud C C. Transfer learning in decision trees. In: Proceedings of the 2007 International Joint Conference on Neural Networks. Orlando, USA: IEEE, 2007. 726-731
[14]	Ntoutsi I, Kalousis A, Theodoridis Y. A general framework for estimating similarity of datasets and decision trees: exploring semantic similarity of decision trees. In: Proceedings of the 8th SIAM International Conference on Data Mining. Atlanta, USA: Society for Industrial and Applied Mathematics Publications, 2008. 810-821
[15]	Fumera G, Roli F. A theoretical and experimental analysis of linear combiners for multiple classifier systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(6): 942-956
[16]	Fawcett T. An introduction to ROC analysis. Pattern Recognition Letters, 2006, 27(8): 861-874
[17]	Li S S, Huang C R, Zong C Q. Multi-domain sentiment classification with classifier combination. Journal of Computer Science and Technology, 2011, 26(1): 25-33