误差准则的信息论解释

陈霸东; 胡金春; 朱煜; 孙增圻

doi:10.3724/SP.J.1004.2009.01302

误差准则的信息论解释

doi: 10.3724/SP.J.1004.2009.01302

1.
Institute of Manufacturing Engineering, Department of Precision Instruments and Mechanology, Tsinghua University, Beijing 100084, P.R. China
2.
State Key Laboratory of Intelligent Technology and Systems, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, P.R. China

详细信息

通讯作者:
陈霸东

计量
- 文章访问数: 1697
- HTML全文浏览量: 43
- PDF下载量: 1429
- 被引次数: 0
出版历程
- 收稿日期: 2008-04-07
- 修回日期: 2008-12-20
- 刊出日期: 2009-10-20

Information Theoretic Interpretation of Error Criteria

1.
Institute of Manufacturing Engineering, Department of Precision Instruments and Mechanology, Tsinghua University, Beijing 100084, P.R. China;
2.
State Key Laboratory of Intelligent Technology and Systems, Department of Computer Science and Technology, Tsinghua University, Beijing 100084, P.R. China

More Information

Corresponding author: CHEN Ba-Dong

摘要

摘要: Error criteria (or error cost functions) play significant roles in statistical estimation problems. In this paper, we study error criteria from the viewpoint of information theory. The relationships between error criteria and error's entropy criterion are investigated. It is shown that an error criterion is equivalent to the error's entropy criterion plus a Kullback-Leibler information divergence (KL-divergence). Based on this result, two important properties of the error criteria are proved. Particularly, the optimum error criterion can be interpreted via the meanings of entropy and KL-divergence. Furthermore, a novel approach is proposed for the choice of p-power error criteria, in which a KL-divergence based cost is minimized. The proposed method is verified by Monte Carlo simulation experiments.
- Estimation /
- error criteria /
- entropy /
- Kullback-Leibler information divergence (KL-divergence) /
- adaptive filtering
Abstract: Error criteria (or error cost functions) play significant roles in statistical estimation problems. In this paper, we study error criteria from the viewpoint of information theory. The relationships between error criteria and error's entropy criterion are investigated. It is shown that an error criterion is equivalent to the error's entropy criterion plus a Kullback-Leibler information divergence (KL-divergence). Based on this result, two important properties of the error criteria are proved. Particularly, the optimum error criterion can be interpreted via the meanings of entropy and KL-divergence. Furthermore, a novel approach is proposed for the choice of p-power error criteria, in which a KL-divergence based cost is minimized. The proposed method is verified by Monte Carlo simulation experiments.
- Estimation /
- error criteria /
- entropy /
- Kullback-Leibler information divergence (KL-divergence) /
- adaptive filtering