引入视觉注意机制的目标跟踪方法综述

黎万义; 王鹏; 乔红

doi:10.3724/SP.J.1004.2014.00561

引入视觉注意机制的目标跟踪方法综述

doi: 10.3724/SP.J.1004.2014.00561

1.
中国科学院自动化研究所精密感知与控制研究中心北京 100190;
2.
中国科学院自动化研究所复杂系统管理与控制国家重点实验室北京 100190

基金项目:

国家自然科学基金（61210009，61100098，61379097）资助

详细信息

作者简介:
王鹏中国科学院自动化研究所副研究员.2010 年获得中国科学院自动化研究所博士学位.主要研究方向为视觉检测与跟踪，视觉注意力模型.E-mail：peng wang@ia.ac.cn

计量
- 文章访问数: 3405
- HTML全文浏览量: 152
- PDF下载量: 3127
- 被引次数: 0
出版历程
- 收稿日期: 2013-04-26
- 修回日期: 2013-09-22
- 刊出日期: 2014-04-20

A Survey of Visual Attention Based Methods for Object Tracking

1.
Research Center of Precision Sensing and Control, Institute of Automation, Chinese Academy of Sciences, Beijing 100190;
2.
State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190

Funds:

Supported by National Natural Science Foundation of China (61210009, 61100098, 61379097)

摘要

摘要: 视觉跟踪在无人飞行器、移动机器人、智能监控等领域有着广泛的应用，但由于目标外观和环境的变化，以及背景干扰等因素的存在，使得复杂场景下的鲁棒实时的目标跟踪成为一项极具挑战性的任务. 视觉注意是人类视觉信息处理过程中的一项重要的心理调节机制，在视觉注意的引导下，人类能够从众多的视觉信息中快速地选择那些最重要、最有用、与当前行为最相关的感兴趣的视觉信息，特别地，人类能够快速指向感兴趣的目标，从而可以轻松地实现对目标的稳定跟踪.因此，将视觉注意机制引入到复杂场景下的目标跟踪中，有利于实现更为稳定和接近于人类认知机制的视觉跟踪算法.本文旨在对引入了视觉注意机制的目标跟踪方法进行综述. 首先，介绍了视觉注意的基本概念及其代表性的计算模型；其次，对视觉注意与跟踪的内在关系进行了阐述；然后，对引入视觉注意机制的目标跟踪方法进行归纳、总结和分类，对代表性的方法进行介绍和分析；最后，对该类方法的特点和优势进行了讨论，并对未来的研究趋势进行了展望.
- 目标跟踪 /
- 视觉注意 /
- 显著性 /
- 选择性注意 /
- 视觉认知
Abstract: Visual tracking has been widely used in numerous applications, such as unmanned aerial vehicles, mobile robots and intelligent visual surveillance. Robust and real-time object tracking in complex scenes is a challenge task. Difficulties in tracking objects can arise due to changing appearance patterns of both the object and the environment, as well as factors such as background interference. Visual attention is one of the key mechanisms of visual perception which directs the processing resources to the visual data of the potentially most relevant, specially directs our gaze rapidly towards objects of interest in our visual environment and as a result humans can easily achieve stable object tracking. Therefore introducing the visual attention mechanism to the object tracking in complex scenes, will facilitate the realization of stable and humanoid tracking algorithms. This paper aims to review the state-of-the-art of visual attention based methods for tracking. Firstly, we introduce the basic concepts of visual attention and its representative computational models. Secondly, the relationship between visual attention and tracking is described. Thirdly, the attention-based visual tracking algorithms are classified into five categories and detailed descriptions of representative methods in each category are provided, and their pros and cons are examined. Finally, we highlight the advantages of attention-based tracking methods and provide insights for future.
- Object tracking /
- visual attention /
- saliency /
- selective attention /
- visual cognition

HTML全文

参考文献(85)

[1]	Zhang S P, Yao H X, Sun X, Lu X S. Sparse coding based visual tracking: review and experimental comparison. Pattern Recognition, 2013, 46(7): 1772-1788
[2]	Yilmaz A, Javed O, Shah M. Object tracking: a survey. ACM Computing Surveys, 2006, 38(4): 13
[3]	Hou Zhi-Qiang, Han Chong-Zhao. A survey of visual tracking. Acta Automatica Sinica, 2006, 32(4): 603-617(侯志强, 韩崇昭. 视觉跟踪技术综述. 自动化学报, 2006, 32(4): 603-617)
[4]	Maggio E, Cavallaro A. Video Tracking: Theory and Practice. West Sussex: Wiley, 2011
[5]	Zou Hai-Rong, Gong Zhen-Bang, Luo Jun. The status quo and prospect of tracking system of ground moving object by UAV. Journal of Astronautics, 2006, 27(z1): 233-236(邹海荣, 龚振邦, 罗均. 无人飞行器地面移动目标跟踪系统研究现状与展望. 宇航学报, 2006, 27(z1): 233-236)
[6]	Yoo S, Kim W, Kim C. Saliency combined particle filtering for aircraft tracking. Journal of Signal Processing Systems, 2013, doi: 10.1007/s11265-013-0803-x
[7]	Liu Wei-Feng, Chai Zhong, Wen Cheng-Lin. Multi-measurement target tracking by using random sampling approach. Acta Automatica Sinica, 2013, 39(2): 164-174(刘伟峰, 柴中, 文成林. 基于随机采样的多量测目标跟踪算法. 自动化学报, 2013, 39(2): 164-174)
[8]	Itti L, Koch C. Computational modelling of visual attention. Nature Reviews Neuroscience, 2001, 2(3): 194-203
[9]	Frintrop S, Rome E, Christensen H I. Computational visual attention systems and their cognitive foundations: a survey. ACM Transactions on Applied Perception, 2010, 7(1): 1-39
[10]	Frintrop S. Computational visual attention. Computer Analysis of Human Behavior. London: Springer, 2011. 69101
[11]	Borji A, Itti L. State-of-the-art in visual attention modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1): 185-207
[12]	Guo Ying-Chun, Yuan Hao-Jie, Wu Peng. Image saliency detection based on local and regional features. Acta Automatica Sinica, 2013, 39(8): 1214-1224(郭迎春, 袁浩杰, 吴鹏. 基于Local特征和Regional特征的图像显著性检测. 自动化学报, 2013, 39(8): 1214-1224)
[13]	Ma Ru-Ning, Tu Xiao-Po, Ding Jun-Di, Yang Jing-Yu. To evaluate salience map towards popping out visual objects. Acta Automatica Sinica, 2012, 38(5): 870-875(马儒宁, 涂小坡, 丁军娣, 杨静宇. 视觉显著性凸显目标的评价. 自动化学报, 2012, 38(5): 870-875)
[14]	Navalpakkam V, Itti L. An integrated model of top-down and bottom-up attention for optimizing detection speed. In: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2006. 2049-2056
[15]	Hu Zheng-Ping, Meng Peng-Quan. Graph presentation random walk salient object detection algorithm based on global isolation and local homogeneity. Acta Automatica Sinica, 2011, 37(10): 1279-1284(胡正平, 孟鹏权. 全局孤立性和局部同质性图表示的随机游走显著目标检测算法. 自动化学报, 2011, 37(10): 1279-1284)
[16]	Gao D S, Han S Y, Vasconcelos N. Discriminant saliency, the detection of suspicious coincidences, and applications to visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(6): 989-1005
[17]	Feng Xin, Yang Dan, Zhang Ling. Saliency variation based quality assessment for packet-loss-impaired videos. Acta Automatica Sinica, 2011, 37(11): 1322-1331(冯欣, 杨丹, 张凌. 基于视觉注意力变化的网络丢包视频质量评估. 自动化学报, 2011, 37(11): 1322-1331)
[18]	Anderson J R. Cognitive Psychology and Its Implications. New York: Worth Publishers, 2004
[19]	Treisman A M, Gelade G. A feature-integration theory of attention. Cognitive Psychology, 1980, 12(1): 97-136
[20]	Wolfe J M. Guided Search 2.0. A revised model of visual search. Psychonomic Bulletin and Review, 1994, 1(2): 202238
[21]	Wolfe J M. Guided search 4.0. Integrated Models of Cognitive Systems, 2006, 44(3): 99-120
[22]	Wu Y, Lim J, Yang M H. Online object tracking: A benchmark. In: Proceedings of the 2003 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Portland, OR, USA: IEEE, 2013. 2411-2418
[23]	Itti L, Koch C, Niebur E. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(11): 1254-1259
[24]	Walther D, Koch C. Modeling attention to salient proto-objects. Neural Networks, 2006, 19(9): 1395-1407
[25]	Frintrop S. VOCUS: A Visual Attention System for Object Detection and Goal-Directed Search. Berlin: Springer, 2006
[26]	Harel J, Koch C, Perona P. Graph-based visual saliency. In: Proceedings of the 20th Annual Conference on Neural Information Processing Systems, NIPS 2006. New York: Neural Information Processing System Foundation, 2007. 545-552
[27]	Itti L, Dhavale N, Pighin F. Realistic avatar eye and head animation using a neurobiological model of visual attention. In: Proceedings of the SPIE's 48th Annual Meeting Optical Science and Technology. New York: International Society for Optics and Photonics, 2004. 64-78
[28]	Gao D, Vasconcelos N. Decision-theoretic saliency: computational principles, biological plausibility, and implications for neurophysiology and psychophysics. Neural Computation, 2009, 21(1): 239-271
[29]	Hou X, Zhang L. Saliency detection: a spectral residual approach. In: Proceedings of the 2007 Computer Vision and Pattern Recognition. Minneapolis, MN: IEEE, 2007. 1-8
[30]	Guo C L, Ma Q, Zhang L M. Spatio-temporal Saliency detection using phase spectrum of quaternion Fourier transform. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, AK: IEEE, 2008. 1-8
[31]	Achanta R, Hemami S, Estrada F, Susstrunk S. Frequency-tuned salient region detection. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, FL: IEEE, 2009. 1597-1604
[32]	Li J, Levine M D, An X, Xu X, He H. Visual saliency based on scale-space analysis in the frequency domain. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(4): 996-1010
[33]	Li W Y, Wang P, Qiao H. Top-down spatiotemporal saliency detection using spectral Filtering. In: Proceedings of the 5th International Conference on Digital Image Processing (ICDIP 2013). Beijing, China: SPIE, 2013. 88782G88782G-5
[34]	Liu T, Sun J, Zheng N N, Tang X O, Shum H Y. Learning to detect a salient object. In: Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, MN: IEEE, 2007. 1-8
[35]	Mahadevan V, Vasconcelos N. Saliency-based discriminant tracking. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, FL: IEEE, 2009. 1007-1013
[36]	Yantis S. Multielement visual tracking: attention and perceptual organization. Cognitive Psychology, 1992, 24(3): 295-340
[37]	Allen R, Mcgeorge P, Pearson D, Milne A B. Attention and expertise in multiple target tracking. Applied Cognitive Psychology, 2004, 18(3): 337-347
[38]	McKeever P, Pylyshyn Z. Nontarget Numerosity and Identity Maintenance with FINSTs: A Two Component Account of Multiple Target Tracking, Technical Report, Centre for Cognitive Science, University of Western Ontario, Canada, 1993
[39]	Sears C R, Pylyshyn Z W. Multiple object tracking and attentional processing. Canadian Journal of Experimental Psychology/Revue Canadienne de Psychologie Experimentale, 2000, 54(1): 1-14
[40]	Makovski T, Jiang Y V. Feature binding in attentive tracking of distinct objects. Visual Cognition, 2009, 17(1-2): 180 -194
[41]	Doran M M, Hoffman J E. The role of visual attention in multiple object tracking: evidence from ERPs. Attention, Perception, and Psychophysics, 2010, 72(1): 33-52
[42]	Mahadevan V, Vasconcelos N. Automatic initialization and tracking using attentional mechanisms. In: Proceedings of the 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2011). Colorado Springs, CO: IEEE, 2011. 15-20
[43]	Mahadevan V, Vasconcelos N. On the connections between saliency and tracking. In: Proceedings of the 26th Annual Conference on Neural Information Processing Systems. Lake Tahoe, Nevada, USA: Curran Associates, Inc., 2012. 1673-1681
[44]	Collins R T, Liu Y, Leordeanu M. Online selection of discriminative tracking features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(10): 16311643
[45]	Wang P, Qiao H. Online appearance model learning and generation for adaptive visual tracking. IEEE Transactions on Circuits and Systems for Video Technology, 2011, 21(2): 156-169
[46]	Ouerhani N, Hügli H. A model of dynamic visual attention for object tracking in natural image sequences. Computational Methods in Neural Modeling, 2003, 2686: 702-709.
[47]	Walther D, Edgington D R, Koch C. Detection and tracking of objects in underwater video. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC, USA: IEEE, 2004. I-544-I-549
[48]	Veyret M, Maisel E. Attention-based target tracking for an augmented reality application. In: Proceedings of the 14th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2006. Vaclav Skala, Union Agency, 2006. 101-108
[49]	Li S, Lee M C. Fast visual tracking using motion saliency in video. In: Proceedings of the 2007 IEEE International Conference on Acoustics, Speech and Signal Processing. Honolulu, HI: IEEE, 2007. I1073-I1076
[50]	Liu H, Shi Y. A robust visual tracking based on selective attention shift. In: Proceedings of the 2009 IEEE Control Applications, (CCA) and Intelligent Control. New York, USA: IEEE, 2009. 1176-1179
[51]	Guo W, Xu C S, Ma S D, Xu M. Visual attention based motion object detection and trajectory tracking. In: Proceedings of the 11th Pacific Rim Conference on Multimedia. Berlin: Springer, 2010. 462-470
[52]	Lee S, Kim G J, Choi S. Real-time tracking of visually attended objects in virtual environments and its application to LOD. IEEE Transactions Visualization and Computer Graphics, 2009, 15(1): 6-19
[53]	Yang G, Liu H. Visual attention and multi-cue fusion based human motion tracking method. In: Proceedings of the 6th International Conference on Natural Computation (ICNC). Yantai, China: IEEE, 2010. 2044-2054
[54]	Ma L L, Cheng J, Liu J, Wang J Q, Lu H Q. Visual attention model based object tracking. In: Proceedings of the 2010 Advances in Multimedia Information Processing, and the 11th Pacific Rim Conference on Multimedia. Berlin: Springer, 2010. 483-493
[55]	Yu Y L, Mann G K I, Gosine R G. A single-object tracking method for robots using object-based visual attention. International Journal of Humanoid Robotics, 2012, 9(4): 50030
[56]	Zhu Ming-Qing, Wang Zhi-Ling, Chen Zong-Hai. Human visual intelligence and particle filter based robust object tracking algorithm. Control and Decision, 2012, 27(11): 17201724(朱明清, 王智灵, 陈宗海. 基于人类视觉智能和粒子滤波的鲁棒目标跟踪算法. 控制与决策, 2012, 27(11): 1720-1724)
[57]	Santana P, Correia L, Mendon\c{ca R, Alves N, Barata J. Tracking natural trails with swarm-based visual saliency. Journal of Field Robotics, 2013, 30(1): 64-86
[58]	Frintrop S, Kessel M. Most salient region tracking. In: Proceedings of the 2009 IEEE International Conference on Robotics and Automation. Kobe: IEEE, 2009. 1869-1874
[59]	Frintrop S, Künigs A, Hoeller F, Schulz D. Visual person tracking using a cognitive observation model. In: Proceedings of the 2009 ICRA Workshop on People Detection and Tracking. Kobe, Japan: IEEE, 2009
[60]	Frintrop S. General object tracking with a component-based target descriptor. In: Proceedings of the 2010 IEEE International Conference on Robotics and Automation (ICRA). Anchorage, AK: IEEE, 2010. 4531-4536
[61]	Frintrop S, Künigs A, Hoeller F, Schulz D. A component-based approach to visual person tracking from a mobile platform. International Journal of Social Robotics, 2010, 2(1): 53-62
[62]	López M T, Fernóndez-Caballero A, Fernóndez M A, Mira J, Delgado A E. Visual surveillance by dynamic visual attention method. Pattern Recognition, 2006, 39(11): 2194-2211
[63]	Zhang S P, Yao H X, Liu S H. Robust visual tracking using feature-based visual attention. In: Proceedings of the 2010 IEEE International Conference on Acoustics, Speech, and Signal Processing. New York: IEEE, 2010. 1150-1153
[64]	Chu J K, Li R H, Li Q Y, Wang H Q. A visual attention model for robot object tracking. International Journal of Automation and Computing, 2010, 7(1): 39-46
[65]	Mahadevan V, Vasconcelos N. Biologically inspired object tracking using center-surround saliency mechanisms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(3): 541-554
[66]	Huang J W, Li Z N. Automatic detection of object of interest and tracking in active video. Journal of Signal Processing Systems, 2011, 65(1): 49-62
[67]	Sidibé D, Fofi D, Mériaudeau F. Using visual saliency for object tracking with particle filters. In: Proceedings of the 18th European Signal Processing Conference (EUSIPCO 2010), 2010. 1776-1780
[68]	Li Z D, Wang W H, Wang Y, Chen F, Wang Y. Visual tracking by proto-objects. Pattern Recognition, 2013, 46(8): 2187-2201
[69]	Yan J, Chen X, Zhu Q. Robust online tracking via adaptive samples selection with saliency detection. EURASIP Journal of Advances in Signal Processing, 2013, 2013(1): 1-11
[70]	Zhang G, Yuan Z J, Zheng N N, Sheng X D, Liu T. Visual saliency based object tracking. In: Proceedings of the 9th Asian Conference on Computer Vision (ACCV 2009). Berlin: Springer, 2009. 193-203
[71]	Zhang G, Yuan Z, Zheng N. Key object discovery and tracking based on context-aware saliency. International Journal of Advanced Robotic Systems, 2013, 10(5): 1-12
[72]	Li Y, Ma Y F, Zhang H J. Salient region detection and tracking in video. In: Proceedings of the 2003 International Conference on Multimedia and Expo. Baltimore, MD: IEEE, 2003. 269-272
[73]	Zhang Heng, Li Li-Chun, Li You, Yu Qi-Feng. Tracking method based on the significance weighted least square image matching. Opto-Electronic Engineering, 2008, 35(4): 23 -27(张恒, 李立春, 李由, 于起峰. 显著性加权最小二乘图像匹配跟踪方法. 光电工程, 2008, 35(4): 23-27)
[74]	Zhang Heng, Li You, Li Li-Chun, Yu Qi-Feng. A new significance weighted mean shift tracking method. Optical Technique, 2008, 191(3): 404-407(张恒, 李由, 李立春, 于起峰. 基于显著性加权的Mean Shift跟踪方法. 光学技术, 2008, 191(3): 404-407)
[75]	Yang M, Yuan J S, Wu Y. Spatial selection for attentional visual tracking. In: Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, MN: IEEE, 2007. 1-8
[76]	Fan J L, Wu Y, Dai S Y. Discriminative spatial attention for robust tracking. Computer Vision-ECCV 2010, Pt I. Berlin: Springer-Verlag, 2010. 480-493
[77]	Wang Q, Chen F, Xu W L. Saliency selection for robust visual tracking. In: Proceedings of the 17th IEEE International Conference on Image Processing (ICIP). Hong Kong, China: IEEE, 2010. 2785-2788
[78]	Zeng J H, Sun Y R. Real-time pedestrian tracking by visual attention and human knowledge learning. In: Proceedings of the 2010 IEEE International Conference on Progress in Informatics and Computing (PIC). Shanghai, China: IEEE, 2010. 345-348
[79]	Fan J L. Contextual saliency with an application to visual tracking. In: Proceedings of the 4th International Congress on Image and Signal Processing (CISP 2011). Shanghai, China: IEEE, 2011. 1416-1419
[80]	Li H L, Ngan K N. Saliency model-based face segmentation and tracking in head-and-shoulder video sequences. Journal of Visual Communication and Image Representation, 2008, 19(5): 320-333
[81]	Liu H, He H J. A salient feature and scene semantics based attention model for human tracking on mobile robots. In: Proceedings of the 2010 IEEE International Conference on Robotics and Automation (ICRA 2010). Anchorage, AK: IEEE, 2010. 4545-4552
[82]	Borji A, Frintrop S. Learning context-based feature descriptors for object tracking. In: Proceedings of the 5th ACM/ IEEE International Conference on Human-Robot Interaction. Osaka: IEEE, 2010. 79-80
[83]	Borji A, Frintrop S, Sihite D N, Itti L. Adaptive object tracking by learning background context. In: Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Providence, RI: IEEE, 2012. 23-30
[84]	Liu T, Yuan Z J, Sun J, Wang J D, Zheng N, Tang X. Learning to detect a salient object. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(2): 353-367
[85]	Comaniciu D, Ramesh V, Meer P. Real-time tracking of non-rigid objects using mean shift. In: Proceedings of the 2000 IEEE Conference on Computer Vision and Pattern Recognition. Hilton Head Island: IEEE, 2000. 142-149