2.624

2020影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于最大化子模和RRWM的视频协同分割

苏亮亮 唐俊 梁栋 王年

苏亮亮, 唐俊, 梁栋, 王年. 基于最大化子模和RRWM的视频协同分割. 自动化学报, 2016, 42(10): 1532-1541. doi: 10.16383/j.aas.2016.c150459
引用本文: 苏亮亮, 唐俊, 梁栋, 王年. 基于最大化子模和RRWM的视频协同分割. 自动化学报, 2016, 42(10): 1532-1541. doi: 10.16383/j.aas.2016.c150459
SU Liang-Liang, TANG Jun, LIANG Dong, WANG Nian. A Video Co-segmentation Algorithm by Means of Maximizing Submodular Function and RRWM. ACTA AUTOMATICA SINICA, 2016, 42(10): 1532-1541. doi: 10.16383/j.aas.2016.c150459
Citation: SU Liang-Liang, TANG Jun, LIANG Dong, WANG Nian. A Video Co-segmentation Algorithm by Means of Maximizing Submodular Function and RRWM. ACTA AUTOMATICA SINICA, 2016, 42(10): 1532-1541. doi: 10.16383/j.aas.2016.c150459

基于最大化子模和RRWM的视频协同分割

doi: 10.16383/j.aas.2016.c150459
基金项目: 

安徽省自然科学基金 1508085MF120

国家自然科学基金 61172127

国家自然科学基金 61401001

高等学校博士学科点专项科研基金 20113401110006

详细信息
    作者简介:

    苏亮亮  安徽大学电子信息工程学院博士研究生.2012年获安徽大学硕士学位.主要研究方向为图像与视频处理, 模式识别.E-mail:sulanqing_sd@163.com

    唐俊  安徽大学电子信息工程学院教授.主要研究方向为计算机图像与视频处理, 模式识别.E-mail:tangjunahu@163.com

    王年  安徽大学电子信息工程学院教授.主要研究方向为计算视觉, 模式识别, 生物信号处理.E-mail:wn_xlb@ahu.edu.cn

    通讯作者:

    梁栋  安徽大学电子信息工程学院教授.主要研究方向为图像处理, 计算信号处理, 模式识别.本文通信作者.E-mail:dliang@ahu.edu.cn

A Video Co-segmentation Algorithm by Means of Maximizing Submodular Function and RRWM

Funds: 

Anhui Provincial Natural Science Foundation 1508085MF120

National Natural Science Foundation of China 61172127

National Natural Science Foundation of China 61401001

Specialized Research Fund for the Doctoral Program of Higher Education of China 20113401110006

More Information
    Author Bio:

     Ph. D. candidate at the School of Electronic and Information Engineering, Anhui University. He received his master degree from Anhui University in 2012. His research interest covers image and video processing, and pattern recognition.E-mail:

     Professor at the School of Electronic and Information Engineering, Anhui University. His research interest covers computer image and video processing, and pattern recognition.E-mail:

     Professor at the School of Electronic and Information Engineering, Anhui University. His research interest covers computer vision, pattern recognition, and bioinformatics.E-mail:

    Corresponding author: LIANG Dong  Professor at the School of Electronic and Information Engineering, Anhui University. His research interest covers image processing, computing signal processing, and pattern recognition. Corresponding author of this paper.E-mail:dliang@ahu.edu.cn
  • 摘要: 成对视频共同运动模式的协同分割指的是同时检测出两个相关视频中共有的行为模式,是计算机视觉研究的一个热点.本文提出了一种新的成对视频协同分割方法.首先,利用稠密轨迹方法对视频运动部分进行检测,并对运动轨迹进行特征表示;然后,引入子模优化方法对单视频内的运动轨迹进行聚类分析;接着采用基于重加权随机游走的图匹配方法对成对视频运动轨迹进行匹配,该方法对出格点、变形和噪声都具有很强的鲁棒性;同时根据图匹配结果实现运动轨迹的共显著性度量;最后,将所有轨迹分类成共同运动轨迹和异常运动轨迹的问题转化为基于图割的马尔科夫随机场的二值化标签问题.通过典型运动视频数据集的比较实验,其结果验证了本文方法的有效性.
  • 图  1  轨迹特征MBH表示

    Fig.  1  Feature representation of trajectory with MBH

    图  2  算法流程图

    Fig.  2  The working flow of this proposed algorithm

    图  3  模拟数据聚类结果

    Fig.  3  The clustering results of the toy data

    图  4  Bench press类中视频轨迹聚类与匹配结果

    Fig.  4  The clustering and matching results of video trajectories

    图  5  Bench press类中视频轨迹聚类与匹配结果

    Fig.  5  The clustering and matching results of video trajectories

    图  6  Bench press类中视频轨迹聚类与匹配结果

    Fig.  6  The clustering and matching results of video trajectories

    表  1  运动标签与其对应的视频对的数量

    Table  1  Action tags and the corresponding number of pairs of sequences

    运动类型数量
    Basketball shooting6
    Bench press8
    Clean and jerk5
    Jumping rope8
    Lunges6
    Rope climbing5
    Pommel horse4
    Golf6
    Pullup4
    Pushup3
    Ride bike7
    下载: 导出CSV

    表  2  本文方法与SC + GM方法的实验结果对比

    Table  2  The experimental results of the proposed method and SC + GM

    类名AODE (%, mean)AODE (%, median)LOC (%, mean)LOC (%, median)COV (%, mean)COV (%, median)
    Basketball36.7634.3896.3510086.0587.81
    47.6647.8464.425073.9081.03
    Bench press24.1027.4710010086.5788.49
    27.3426.5684.0910075.5281.86
    Clean and jerk31.0530.9410010073.8182.22
    30.7933.3494.5010070.4076.31
    Jumping rope5.4538.6010010084.2383.58
    20.1823.6965.8910077.3086.15
    Lunges28.2831.9189.6210052.7857.58
    29.3128.4837.655065.0583.76
    Kayaking51.6253.6841.505064.9669.39
    42.5143.6742.135065.4862.60
    Rope climbing20.6022.0074.1710063.0882.16
    33.8119.4164.7572.1456.4371.33
    下载: 导出CSV

    表  3  本文方法与其他方法的实验结果对比

    Table  3  The experimental results of the proposed method and other methods

    类名方法MR (%, mean)COV (%, mean)
    本文方法51.3760.12
    GolfSC + GM50.6458.90
    VCS48.8254.17
    本文方法82.2080.35
    PullupSC + GM79.5878.78
    VCS70.2272.41
    本文方法47.5452.81
    PushupSC + GM45.6152.97
    VCS55.3460.15
    本文方法35.3035.78
    Ride bikeSC + GM34.6236.35
    VCS38.4534.56
    下载: 导出CSV
  • [1] Rother C, Minka T, Blake A, Kolmogorov V. Cosegmentation of image pairs by histogram matching-incorporating a global constraint into MRFs. In:Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York, USA:IEEE, 2006. 993-1000
    [2] Vicente S, Rother C, Kolmogorov V. Object cosegmentation. In:Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI:IEEE, 2011. 2217-2224
    [3] Cao X C, Tao Z Q, Zhang B, Fu H Z, Feng W. Self-adaptively weighted co-saliency detection via rank constraint. IEEE Transactions on Image Processing, 2014, 23(9):4175-4186 https://www.ncbi.nlm.nih.gov/pubmed/24968170
    [4] Zhang D W, Han J W, Li C, Wang J D. Co-saliency detection via looking deep and wide. In:Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, Massachusetts, USA:IEEE, 2015. 2994-3002
    [5] Galasso F, Iwasaki M, Nobori K, Cipolla R. Spatio-temporal clustering of probabilistic region trajectories. In:Proceedings of the 2011 International Conference on Computer Vision. Barcelona, Spain:IEEE, 2011. 1738-1745
    [6] 王蒙, 戴亚平, 王庆林.单目视觉下目标三维行为的时间尺度不变建模及识别.自动化学报, 2014, 40(8):1644-1653 http://www.aas.net.cn/CN/article/downloadArticleFile.do?attachType=PDF&id=18432

    Wang Meng, Dai Ya-Ping, Wang Qing-Lin. Time-scale invariant modeling and classifying for object behaviors in 3D space based on monocular vision. Acta Automatica Sinica, 2014, 40(8):1644-1653 http://www.aas.net.cn/CN/article/downloadArticleFile.do?attachType=PDF&id=18432
    [7] 褚一平, 张引, 叶修梓, 张三元.基于隐条件随机场的自适应视频分割算法.自动化学报, 2007, 33(12):1252-1258 http://www.aas.net.cn/CN/article/searchArticle.do#

    Chu Yi-Ping, Zhang Yin, Ye Xiu-Zi, Zhang San-Yuan. Adaptive video segmentation algorithm using hidden conditional random fields. Acta Automatica Sinica, 2007, 33(12):1252-1258 http://www.aas.net.cn/CN/article/searchArticle.do#
    [8] Joulin A, Tang K, Li F F. Efficient image and video co-localization with frank-wolfe algorithm. In:Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland:Springer, 2014. 253-268 doi: 10.1007%2F978-3-319-10599-4_17
    [9] Prest A, Leistner C, Civera J, Schmid C, Ferrari V. Learning object class detectors from weakly annotated video. In:Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, Rhode Island, USA:IEEE, 2012. 3282-3289
    [10] Chen D J, Chen H T, Chang L W. Video object cosegmentation. In:Proceedings of the 20th ACM International Conference on Multimedia. Nara, Japan:ACM, 2012. 805-808
    [11] Rubio J C, Serrat J, López A. Video co-segmentation. In:Proceedings of the 11th Asian Conference on Computer Vision. Daejeon, Korea:Springer, 2013. 13-24 doi: 10.1007%2F978-3-642-37444-9_2
    [12] Chiu W C, Fritz M. Multi-class video co-segmentation with a generative multi-video model. In:Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, OR:IEEE, 2013. 321-328
    [13] Zhang D, Javed O, Shah M. Video object co-segmentation by regulated maximum weight cliques. In:Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland:Springer, 2014. 551-566 doi: 10.1007%2F978-3-319-10584-0_36
    [14] Fu H Z, Xu D, Zhang B, Lin S. Object-based multiple foreground video co-segmentation. In:Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, Ohio, USA:IEEE, 2014. 3166-3173
    [15] Wang W G, Shen J B, Li X L, Porikli F. Robust video object cosegmentation. IEEE Transactions on Image Processing, 2015, 24(10):3137-3148 doi: 10.1109/TIP.2015.2438550
    [16] Guo J M, Li Z W, Cheong L F, Zhou S Z. Video co-segmentation for meaningful action extraction. In:Proceedings of the 2013 IEEE International Conference on Computer Vision. Sydney, Australia:IEEE, 2013. 2232-2239
    [17] Wang H, Kläser A, Schmid C, Liu C L. Action recognition by dense trajectories. In:Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI:IEEE, 2011. 3169-3176
    [18] Tang J, Shao L, Li X L. Efficient dictionary learning for visual categorization. Computer Vision and Image Understanding, 2014, 124:91-98 doi: 10.1016/j.cviu.2014.02.007
    [19] Yang F, Jiang Z L, Davis L S. Submodular reranking with multiple feature modalities for image retrieval. In:Proceedings of the 12th Asian Conference on Computer Vision. Singapore:Springer, 2015. 19-34 http://www.umiacs.umd.edu/~fyang/papers/accv14.pdf
    [20] Xu J, Mukherjee L, Li Y, Warner J, Rehg J M, Singht V. Gaze-enabled egocentric video summarization via constrained submodular maximization. In:Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, Massachusetts, USA:IEEE, 2015. 2235-2244
    [21] Gygli M, Grabner H, van Gool L. Video summarization by learning submodular mixtures of objectives. In:Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, Massachusetts, USA:IEEE, 2015. 3090-3098
    [22] Leordeanu M, Hebert M. A spectral technique for correspondence problems using pairwise constraints. In:Proceedings of the 10th IEEE International Conference on Computer Vision. Beijing, China:IEEE, 2005. 1482-1489
    [23] Cho M, Lee J, Lee K M. Reweighted random walks for graph matching. In:Proceedings of the 11th European Conference on Computer Vision. Crete, Greece:Springer, 2010. 492-505 doi: 10.1007%2F978-3-642-15555-0_36
    [24] Dalal N, Triggs B, Schmid C. Human detection using oriented histograms of flow and appearance. In:Proceedings of the 9th European Conference on Computer Vision. Graz, Austria:Springer, 2006. 428-441 http://www.oalib.com/references/17185810
    [25] Lovász L. Submodular functions and convexity. Mathematical Programming the State of the Art. Berlin Heidelberg:Springer, 1983. 235-257 doi: 10.1007%2F978-3-642-68874-4_10
    [26] Jiang Z L, Davis L S. Submodular salient region detection. In:Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, OR:IEEE, 2013. 2043-2050
    [27] Boykov Y, Veksler O, Zabih R. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(11):1222-1239 doi: 10.1109/34.969114
  • 加载中
图(6) / 表(3)
计量
  • 文章访问数:  2032
  • HTML全文浏览量:  314
  • PDF下载量:  512
  • 被引次数: 0
出版历程
  • 收稿日期:  2015-07-20
  • 录用日期:  2016-04-18
  • 刊出日期:  2016-10-20

目录

    /

    返回文章
    返回