2.793

2018影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于异步相关判别性学习的孪生网络目标跟踪算法

许龙 魏颖 商圣行 张皓云 边杰 徐楚翘

许龙, 魏颖, 商圣行, 张皓云, 边杰, 徐楚翘. 基于异步相关判别性学习的孪生网络目标跟踪算法. 自动化学报, 2020, 46(x): 1−19 doi: 10.16383/j.aas.c200237
引用本文: 许龙, 魏颖, 商圣行, 张皓云, 边杰, 徐楚翘. 基于异步相关判别性学习的孪生网络目标跟踪算法. 自动化学报, 2020, 46(x): 1−19 doi: 10.16383/j.aas.c200237
Xu Long, Wei Ying, Shang Sheng-Xing, Zhang Hao-Yun, Bian Jie, Xu Chu-Qiao. Design of asynchronous correlation discriminant single object tracker based on siamese network. Acta Automatica Sinica, 2020, 46(x): 1−19 doi: 10.16383/j.aas.c200237
Citation: Xu Long, Wei Ying, Shang Sheng-Xing, Zhang Hao-Yun, Bian Jie, Xu Chu-Qiao. Design of asynchronous correlation discriminant single object tracker based on siamese network. Acta Automatica Sinica, 2020, 46(x): 1−19 doi: 10.16383/j.aas.c200237

基于异步相关判别性学习的孪生网络目标跟踪算法

doi: 10.16383/j.aas.c200237
基金项目: 国家自然科学基金(61871106)资助
详细信息
    作者简介:

    许龙:东北大学模式识别专业博士研究生. 2016 年获得内蒙古大学学士学位. 主要研究方向为机器学习与目标跟踪. E-mail: wahaha4ever@163.com

    魏颖:东北大学博士生导师. 1990 年获得哈尔滨工业大学学士学位, 1997 年和~2001 年分别获得东北大学硕士学位和博士学位. 主要研究方向包括图像处理与模式识别, 医学图像计算和分析, 计算机辅助诊断等. 本文通信作者. E-mail: weiying@ise.neu.edu.cn

    商圣行:东北大学控制工程专业研究生. 主要研究方向为模式识别, 计算机视觉和深度学习. E-mail: ssh3108@163.com

    张皓云:东北大学控制工程专业硕士. 2019 年获得东北大学学士学位. 主要研究方向为目标跟踪与目标检测. E-mail: nicolascloud@163.com

    边杰:东北大学信息科学与工程学院硕士研究生. 2017 年获得东北大学学士学位. 主要研究方向为视觉目标跟踪. E-mail: qbzxbj@163.com

    徐楚翘:东北大学控制工程专业研究生. 主要研究方向为计算机视觉领域下的目标跟踪. E-mail: xuchuqiao@mail.neu.edu.cn

Design of Asynchronous Correlation Discriminant Single Object Tracker Based on Siamese Network

Funds: Supported by National Natural Science Foundation of China (61871106)
  • 摘要: 现有基于孪生网络的单目标跟踪算法能够实现很高的跟踪精度, 但是这些跟踪器不具备在线更新的能力, 而且其在跟踪时很依赖目标的语义信息, 这导致基于孪生网络的单目标跟踪算法在面对具有相似语义信息的干扰物时会跟踪失败. 为了解决这个问题, 本文提出了一种异步相关响应的计算模型, 并提出一种高效利用不同帧间目标语义信息的方法. 在此基础上, 提出了一种新的具有判别性的跟踪算法. 同时为了解决判别模型使用一阶优化算法收敛慢的问题, 本文使用近似二阶优化的方法更新判别模型. 为验证所提算法的有效性, 本文分别在Got-10k, TC128, OTB 和VOT2018 上做了对比实验, 实验结果表明, 本文提出的方法可以明显地改进基准算法的性能.
  • 图  1  (b)和(c)分别表示滤波器 ${\rm k}_0$ 与滤波器 ${\rm k}_{\rm{t}} = \phi({\rm{z}}_{\rm{t}})$ 计算得到的响应得分图

    Fig.  1  (b), and (c) denote the response which is calculated by ${\rm k}_0$ and ${\rm k}_t = \phi({\rm{z}}_{\rm{t}})$ respectively

    图  2  本文算法与其他先进跟踪器在Got-10k上的对比情况

    Fig.  2  Comparison between the proposed method with other advanced trackers on Got-10k

    图  3  Got-10k上跟踪结果对比实验. 其中虚线框表示本文算法的跟踪结果, 实线框表示基准算法的跟踪结果

    Fig.  3  Comparison of tracking results on Got-10k. The dotted line box indicates the tracking results of the proposed algorithm, and the solid line box indicates the baseline results

    图  4  本文所提出的算法在TC128 上的精度-成功率对比实验结果

    Fig.  4  The accuracy-success rate comparison experiment results of the proposed algorithm on TC128

    图  5  本文所提出的算法在OTB2015上的精度-成功率对比实验结果

    Fig.  5  The accuracy-success rate comparison experiment results of the proposed algorithm on OTB2015

    图  6  在OTB50的6 个序列上的实验结果. 其中Init Sampler 表示第一帧目标计算得到的 ${\rm k}_0$ , Current Sampler 表示当前帧目标计算得到的 ${\rm k}_t$ , Optim Sampler 表示对当前 ${\rm k}_t$ 进行优化后得到的 ${\rm k}_{\rm{t}} = \dfrac{1}{{\rm{m}}}\sum_{{\rm{i}}}^{{\rm{m}}}\Phi_{\rm{i}}({\rm{k}}_{\rm{t}})$

    Fig.  6  The response visualization on OTB50. Init Sampler denotes ${\rm k}_0$ , which is obtained in the first frame. Current Sampler denotes ${\rm k}_{\rm{t}}$ , which is calculated in the current frame. Optim Sampler denotes the ${\rm k}_{\rm{t}} = \dfrac{1}{{\rm{m}}}\sum_{{\rm{i}}}^{{\rm{m}}}\Phi_{\rm{i}}({\rm{k}}_{\rm{t}})$ , which is obtained after optimized discriminate model

    图  7  精度鲁棒性-跟踪失败情况对比图

    Fig.  7  Comparison of accuracy robustness and tracking faliure

    图  8  在VOT2018序列的不同情景下精度鲁棒性对比情况

    Fig.  8  Comparison of accuracy robustness performance under different attributes on VOT2018

    图  9  在VOT2018的baseline下的EOA对比曲线

    Fig.  9  Comparison of expected overlap performance on VOT2018

    图  10  在VOT2018的unsupervised下的EOA对比曲线

    Fig.  10  EOA comparison curve of unsupervisized training on VOT2018

    图  11  在VOT2018的realtime下的EOA对比曲线

    Fig.  11  EOA comparison curve in realtime on VOT2018

    图  12  在VOT2018的实时性能对比下不同跟踪器的期望重叠率性能排名情况对比

    Fig.  12  Ranking of different trackers' expected overlap ratio in real time on VOT2018

    表  1  本文所提方法与基准算法的消融实验

    Table  1  The ablation expirement of the proposed algorithm and the benchmark algorithm

    AO $ {\rm SR}_{0.5} $ $ {\rm SR}_{0.75} $ FPS
    baseline 0.445 0.539 0.208 21.95
    baseline+AC 0.445 0.539 0.211 20.03
    baseline+AC+S 0.447 0.542 0.211 19.63
    baseline+AC+S+ $ {\rm D}_{{\rm{KL}}} $ m = 3 0.442 0.537 0.209 18.72
    baseline+AC+S+ $ {\rm D}_{{\rm{KL}}} $ m = 6 0.457 0.553 0.215 18.60
    baseline+AC+S+ $ {\rm D}_{{\rm{KL}}} $ m = 9 0.440 0.532 0.211 18.49
    下载: 导出CSV

    表  2  OTB2013 的 BC、DEF 等情景下的跟踪精度对比结果

    Table  2  Comparison of tracking accuracy under 11 attributes on OTB2013

    BC BC DEF DEF FM FM IPR IPR
    S P S P S P S P
    ECO-HC 0.700 0.559 0.567 0.719 0.570 0.697 0.517 0.648
    ECO 0.776 0.619 0.613 0.772 0.655 0.783 0.630 0.764
    ATOM 0.733 0.598 0.623 0.771 0.595 0.709 0.579 0.714
    DIMP 0.749 0.607 0.602 0.740 0.618 0.739 0.561 0.685
    MDNet 0.777 0.621 0.620 0.780 0.652 0.796 0.658 0.822
    SiamFC 0.605 0.494 0.487 0.608 0.509 0.618 0.483 0.583
    DaSiamRPN 0.728 0.592 0.609 0.761 0.565 0.702 0.625 0.780
    SiamRPN(baseline) 0.605 0.745 0.591 0.724 0.589 0.724 0.627 0.770
    baseline+AC 0.605 0.745 0.591 0.724 0.589 0.724 0.627 0.770
    baseline+AC+ $ {\rm D}_{{\rm{KL}}}^{{\rm{m}} = 3} $ 0.599 0.741 0.603 0.749 0.645 0.797 0.651 0.808
    baseline+AC+ $ {\rm D}_{{\rm{KL}}}^{{\rm{m}} = 6} $ 0.592 0.733 0.597 0.742 0.636 0.787 0.650 0.807
    baseline+AC+ $ {\rm D}_{{\rm{KL}}}^{{\rm{m}} = 9} $ 0.598 0.736 0.586 0.725 0.587 0.723 0.654 0.809
    下载: 导出CSV

    表  3  OTB2013的IV、LR等情景下的跟踪精度对比结果

    Table  3  Comparison of tracking accuracy under 11 attributes on OTB2013

    IV IV LR LR MB MB OCC OCC
    S P S P S P S P
    ECO-HC 0.556 0.690 0.536 0.619 0.566 0.685 0.586 0.749
    ECO 0.616 0.766 0.569 0.677 0.659 0.786 0.636 0.800
    ATOM 0.604 0.749 0.554 0.654 0.529 0.665 0.617 0.762
    DIMP 0.606 0.749 0.485 0.571 0.564 0.695 0.610 0.750
    MDNet 0.619 0.780 0.644 0.804 0.662 0.813 0.623 0.777
    SiamFC 0.479 0.593 0.499 0.600 0.485 0.617 0.512 0.635
    DaSiamRPN 0.589 0.736 0.490 0.618 0.533 0.688 0.583 0.726
    SiamRPN(baseline) 0.585 0.723 0.519 0.653 0.532 0.684 0.586 0.726
    baseline+AC 0.585 0.723 0.519 0.653 0.532 0.684 0.586 0.726
    baseline+AC+ $ {\rm D}_{{\rm{KL}}}^{{\rm{m}} = 3} $ 0.600 0.749 0.554 0.697 0.610 0.785 0.593 0.740
    baseline+AC+ $ {\rm D}_{{\rm{KL}}}^{{\rm{m}} = 6} $ 0.592 0.741 0.546 0.688 0.596 0.770 0.586 0.732
    baseline+AC+ $ {\rm D}_{{\rm{KL}}}^{{\rm{m}} = 9} $ 0.581 0.724 0.549 0.689 0.533 0.687 0.576 0.716
    下载: 导出CSV

    表  4  OTB2013的OPR、OV等情景下的跟踪精度对比结果

    Table  4  Comparison of tracking accuracy under 11 attributes on OTB2013

    OPR OPR OV OV SV SV
    S P S P S P
    ECO-HC 0.563 0.718 0.549 0.763 0.587 0.740
    ECO 0.628 0.787 0.733 0.827 0.651 0.793
    ATOM 0.607 0.751 0.522 0.563 0.654 0.792
    DIMP 0.596 0.737 0.549 0.593 0.636 0.767
    MDNet 0.628 0.787 0.698 0.769 0.675 0.842
    SiamFC 0.500 0.620 0.574 0.642 0.542 0.665
    DaSiamRPN 0.599 0.750 0.570 0.633 0.587 0.740
    SiamRPN(baseline) 0.598 0.736 0.658 0.725 0.608 0.751
    baseline+AC 0.598 0.736 0.658 0.725 0.608 0.751
    baseline+AC+ $ {\rm D}_{{\rm{KL}}}^{{\rm{m}} = 3} $ 0.611 0.760 0.702 0.778 0.656 0.819
    baseline+AC+ $ {\rm D}_{{\rm{KL}}}^{{\rm{m}} = 6} $ 0.604 0.752 0.659 0.733 0.631 0.791
    baseline+AC+ $ {\rm D}_{{\rm{KL}}}^{{\rm{m}} = 9} $ 0.597 0.740 0.660 0.735 0.603 0.755
    下载: 导出CSV

    表  5  VOT2018 上的实验结果

    Table  5  Experimental results on VOT2018

    baseline unsupervised realtime
    A-R rank Failures EAO FPS AO FPS EAO
    KCF 0.4441 50.0994 0.1349 60.0053 0.2667 63.9847 0.1336
    SRDCF 0.4801 64.1136 0.1189 2.4624 0.2465 2.7379 0.0583
    ECO 0.4757 17.6628 0.2804 3.7056 0.402 4.5321 0.0775
    ATOM 0.5853 12.3591 0.4011 5.2061 0 NaN 0
    SiamFC 0.5002 34.0259 0.188 31.889 0.3445 35.2402 0.182
    DaSiamRPN 0.5779 17.6608 0.3826 58.854 0.4722 64.4143 0.3826
    SiamRPN(baseline) 0.5746 23.5694 0.2941 14.3760 0.4355 14.4187 0.0559
    baseline+AC 0.5825 27.0794 0.2710 13.7907 0.4431 13.8772 0.0539
    baseline+AC+ $ {\rm D}_{{\rm{KL}}}^{{\rm{m}} = 3} $ 0.5789 14.8312 0.2865 13.6035 0.4537 13.4039 0.0536
    baseline+AC+ $ {\rm D}_{{\rm{KL}}}^{{\rm{m}} = 6} $ 0.5722 22.6765 0.2992 13.5359 0.4430 12.4383 0.0531
    baseline+AC+ $ {\rm D}_{{\rm{KL}}}^{{\rm{m}} = 9} $ 0.5699 22.9148 0.2927 13.5046 0.4539 12.1159 0.0519
    下载: 导出CSV
  • [1] 刘巧元, 王玉茹, 张金玲, 殷明浩. 基于相关滤波器的视频跟踪方法研究进展. 自动化学报, 2019, 45(2): 265−275

    LIU Qiao-Yuan, WANG Yu-Ru, ZHANG Jin-Ling, YIN Ming-Hao. Research Progress of Visual Tracking Methods Based on Correlation Filter. ACTA AUTOMATICA SINICA, 2019, 45(2): 265−275
    [2] 刘畅, 赵巍, 刘鹏, 唐降龙. 目标跟踪中辅助目标的选择、跟踪与更新. 自动化学报, 2018, 44(7): 1195−1211

    LIU Chang, ZHAO Wei, LIU Peng, TANG Xiang-Long. Auxiliary Objects Selecting, Tracking and Updating in Target Tracking. ACTA AUTOMATICA SINICA, 2018, 44(7): 1195−1211
    [3] 蔺海峰, 宇峰, 宋涛. 基于SIFT特征目标跟踪算法研究. 自动化学报, 2010, 36(8): 1204−1208

    LIN Hai-Feng, MA Yu-Feng, SONG Tao. Research on Object Tracking Algorithm Based on SIFT. ACTA AUTOMATICA SINICA, 2010, 36(8): 1204−1208
    [4] Bolme D S, Beveridge J R, Draper B A, Lui Y M. Visual object tracking using adaptive correlation fllters. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, 2010. 2544−2550
    [5] Henriques J F, Caseiro R, Martins P, Batista J. High-Speed Tracking with Kernelized Correlation Filters. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 3, 1 March 2015. 583−596
    [6] Danelljan M, Hger G, Khan F S, Felsberg M. Learning Spatially Regularized Correlation Filters for Visual Tracking. In: 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, 2015. 4310−4318
    [7] Danelljan M, Bhat G, Khan F S, Felsberg M. ECO: E–cient Convolution Operators for Tracking. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, 2017. 6931−6939
    [8] Nam H, Han B. Learning Multi-domain Convolutional Neural Networks for Visual Tracking. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 2016. 4293−4302
    [9] Ma C, Huang J, Yang X, Yang M. Hierarchical Convolutional Features for Visual Tracking. In: 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, 2015. 3074−3082
    [10] Wang Nai-Yan, Dit-Yan Yeung. Learning a deep compact image representation for visual tracking. In: 2013 Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 1 (NIPS’ 13). Curran Associates Inc., Red Hook, NY, USA. 809−817
    [11] Held D, Thrun S, Savarese S. Learning to Track at 100 FPS with Deep Regression Networks. In: Leibe B., Matas J., Sebe N., Welling M. (eds) Computer Vision – European Conference on Computer Vision (ECCV) 2016. ECCV 2016. Lecture Notes in Computer Science, vol 9905. Springer, Cham. 749−765
    [12] Bertinetto L, Valmadre J, Henriques J F, Vedaldi A, Torr P H S. Fully-Convolutional Siamese Networks for Object Tracking. In: Hua Gang, JÉgou HervÉ(eds) Computer Vision – ECCV 2016 Workshops. ECCV 2016. Lecture Notes in Computer Science, vol 9914. Springer, Cham. 850−865
    [13] Li B, Yan J, Wu W, Zhu Z, Hu X. High Performance Visual Tracking with Siamese Region Proposal Network. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018. 8971−8980
    [14] Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, 2017. 1137−1149
    [15] Li B, Wu W, Wang Q, Zhang F, Xing J, Yan J. SiamRPN++: Evolution of Siamese Visual Tracking With Very Deep Networks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019. 4277−4286
    [16] Wang Q, Zhang L, Bertinetto L, Hu W, Torr P H S. Fast Online Object Tracking and Segmentation: A Unifying Approach. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019. 1328−1338
    [17] Zhang J, Ma S, Sclarofi S. MEEM: Robust Tracking via Multiple Experts Using Entropy Minimization. In: Fleet D., Pajdla T., Schiele B., Tuytelaars T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8694. Springer, Cham. 188−203
    [18] Hare Sam, Safiari Amir, Torr Philip H S. Struck: Structured Output Tracking with Kernels. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 10, 2016. 2096−2109
    [19] Grabner H, Leistner C, Bischof H. Semi-supervised OnLine Boosting for Robust Tracking. In: Forsyth D., Torr P., Zisserman A. (eds) Computer Vision – ECCV 2008. ECCV 2008. Lecture Notes in Computer Science, vol 5302. Springer, Berlin, Heidelberg. 234−247
    [20] Jia X, Lu H, Yang M. Visual tracking via adaptive structural local sparse appearance model. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, 2012. 1822−1829
    [21] Adam A, Rivlin E, Shimshoni I. Robust Fragments-based Tracking using the Integral Histogram. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), New York, NY, USA, 2006. 798−805
    [22] Danelljan M, Bhat G, Khan F S, Felsberg M. ATOM: Accurate Tracking by Overlap Maximization. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019. 4655−4664
    [23] Danelljan M, Bhat G, Khan F S, M Felsberg. ECO: E–cient Convolution Operators for Tracking. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, 2017. 6931−6939
    [24] Jiang B, Luo R, Mao J, Xiao T, Jiang Y. Acquisition of Localization Confldence for Accurate Object Detection. In: Ferrari V., Hebert M., Sminchisescu C., Weiss Y. (eds) Computer Vision – ECCV 2018. ECCV 2018. Lecture Notes in Computer Science, vol 11218. Springer, Cham. 816−832
    [25] Huang L, Zhao X, Huang K. GOT-10k: A Large HighDiversity Benchmark for Generic Object Tracking in the Wild. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019. 1−1
    [26] Liang P, Blasch E, Ling H. Encoding Color Information for Visual Tracking: Algorithms and Benchmark. In: IEEE Transactions on Image Processing, 2015. 5630−5644
    [27] Wu Y, Lim J, Yang M. Object Tracking Benchmark. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015. 1834−1848
    [28] Zhu Z, Wang Q, Li B, Wu W, Yan J, Hu W. DistractorAware Siamese Networks for Visual Object Tracking. In: Ferrari V., Hebert M., Sminchisescu C., Weiss Y. (eds) Computer Vision – ECCV 2018. ECCV 2018. Lecture Notes in Computer Science, vol 11213. Springer, Cham. 103−119
    [29] Bhat Goutam, Danelljan Martin, Gool Luc Van, Timofte Radu. Learning Discriminative Model Prediction for Tracking. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 2019. 6181−6190
    [30] Kristan M, et al. A Novel Performance Evaluation Methodology for Single-Target Trackers. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 11, 2016. 2137−2155
    [31] Danelljan M, Robinson A, Shahbaz Khan F, Felsberg M. Beyond Correlation Filters: Learning Continuous Convolution Operators for Visual Tracking. In: Leibe B., Matas J., Sebe N., Welling M. (eds) Computer Vision – ECCV 2016. ECCV 2016. Lecture Notes in Computer Science, vol 9909. Springer, Cham. 472−488
    [32] Ramasubramanian K, Singh A. (2017) Machine Learning Theory and Practices. In: Machine Learning Using R. Apress, Berkeley, CA.
    [33] Pearlmutter B A. Fast Exact Multiplication by the Hessian. In Neural Computation, vol. 6, no. 1, 1994. 147−160
  • 加载中
计量
  • 文章访问数:  88
  • HTML全文浏览量:  12
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-04-21
  • 录用日期:  2020-09-07

目录

    /

    返回文章
    返回