2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于SSVEP直接脑控机器人方向和速度研究

伏云发 郭衍龙 李松 熊馨 李勃 余正涛

伏云发, 郭衍龙, 李松, 熊馨, 李勃, 余正涛. 基于SSVEP直接脑控机器人方向和速度研究. 自动化学报, 2016, 42(11): 1630-1640. doi: 10.16383/j.aas.2016.c150880
引用本文: 伏云发, 郭衍龙, 李松, 熊馨, 李勃, 余正涛. 基于SSVEP直接脑控机器人方向和速度研究. 自动化学报, 2016, 42(11): 1630-1640. doi: 10.16383/j.aas.2016.c150880
FU Yun-Fa, GUO Yan-Long, LI Song, XIONG Xin, LI Bo, YU Zheng-Tao. Direct-brain-controlled Robot Direction and Speed Based on SSVEP Brain Computer Interaction. ACTA AUTOMATICA SINICA, 2016, 42(11): 1630-1640. doi: 10.16383/j.aas.2016.c150880
Citation: FU Yun-Fa, GUO Yan-Long, LI Song, XIONG Xin, LI Bo, YU Zheng-Tao. Direct-brain-controlled Robot Direction and Speed Based on SSVEP Brain Computer Interaction. ACTA AUTOMATICA SINICA, 2016, 42(11): 1630-1640. doi: 10.16383/j.aas.2016.c150880

基于SSVEP直接脑控机器人方向和速度研究

doi: 10.16383/j.aas.2016.c150880
基金项目: 

云南省级人培项目 KKSY201303048

云南省教育厅重点项目 2013Z130

云南省应用基础研究计划 2013FB026

国家自然科学基金 61363043, 61463024, 81470084

详细信息
    作者简介:

    伏云发 昆明理工大学信息工程与自动化学院副教授.主要研究方向为模式识别与智能控制, 脑信息处理与脑控机器人接口, 脑-机交互控制理论和方法, 脑网络连通性.E-mail:fyf@ynu.edu.cn

    李松 昆明理工大学信息工程与自动化学院硕士研究生.主要研究方向为脑信息处理与脑机交互控制, 模式识别与智能控制.E-mail:lksong1234@sina.com

    熊馨 昆明理工大学信息工程与自动化学院讲师.主要研究方向为医学图像处理与模式识别, 脑网络连通性, 脑信息处理与脑机交互.E-mail:xiongxin840826@163.com

    李勃 昆明理工大学信息工程与自动化学院教授.主要研究方向为智能信息处理, 图像处理与模式识别.E-mail:lbly9177@163.com

    余正涛 昆明理工大学信息工程与自动化学院教授.主要研究方向为智能信息处理.E-mail:ztyu@hotmail.com

    通讯作者:

    郭衍龙 昆明理工大学信息工程与自动化学院硕士研究生.主要研究方向为脑信息处理与脑机交互控制, 模式识别与智能控制. E-mail:hrbeu_gyl@foxmail.com.

Direct-brain-controlled Robot Direction and Speed Based on SSVEP Brain Computer Interaction

Funds: 

Cultiva- tion Program of Talents of Yunnan Province KKSY201303048

Focal Program for Education Office of Yunnan Province 2013Z130

Research Project for Appli- cation Foundation of Yunnan Province 2013FB026

Supported by National Natural Science Foundation of China 61363043, 61463024, 81470084

More Information
    Author Bio:

    Associate professor at the Faculty of Information Engineering and Automation, Kunming University of Science and Technology. His research interest covers pattern recognition and intelligent control, brain information processing and brain-controlled robot interface, theories and methods for brain-machine interaction control, and brain network connectivity.

    Lecturer at the Faculty of Information Engineering and Automation, Kunming University of Science and Technology. Her research interest covers medical image processing and pattern recognition, brain network connectivity, brain information processing, and brain-computer interaction.

    Professor at the Faculty of Information Engineering and Automation, Kunming University of Science and Technolog. His research interest covers intelligent information processing, image processing, and pattern recognition.

    Professor at the Faculty of Information Engineering and Automation, Kunming University of Science and Technology. His main research interest is intelligent information processing.

    Corresponding author: GUO Yan-LongMaster student at the Faculty of Information Engineering and Automation, Kunming University of Science and Technology. His research interest covers brain information processing, brain-computer interaction control, pattern recognition, and intelligent control. Corresponding author of this paper.
  • 摘要: 直接用思维意图来控制机器人而没有大脑外周神经和肌肉的参与是人类的一个梦想,目前这一研究已成为国际前沿热点和突破点.传统的脑控机器人(Brain-controlled robot,BCR)主要控制其方向,而本文旨在探讨能够同时脑控机器人方向和速度的有效方法.采用可分类目标数多、单次识别率高且训练时间短的稳态视觉诱发电位(Steady state visual evoked potentials,SSVEP)脑机交互(Brain-computer/machine interaction,BCI/BMI)方法,为脑控机器人运动规划了向左、向右、前进和后退4个方向,设计了低速、中速和高速3级运动速度并组合了9个脑控指令;进而比较并优化了SSVEP刺激目标布局间距以及刺激目标闪烁时间,采用典型相关分析(Canonical correlation analysis,CCA)进行识别.结果表明恰当设置SSVEP刺激目标数及其布局间距和刺激目标闪烁时间,可以有效提高被试/用户直接脑控机器人的性能;优化的SSVEP刺激范式三结合适应SSVEP解码的典型相关分析,8名被试脑控机器人到达终点平均用时为2分40秒,最少用时1分29秒;同时,在脑控机器人运动过程中触碰障碍平均次数为0.88,最少碰触次数为0.本研究显示基于SSVEP的脑机交互可以作为直接脑控机器人灵活运动的一种可选方法,能够实现对机器人多个运动方向和多级速度的控制;也证实了适当增加刺激目标间距可以有效提高SSVEP-BCI脑控指令识别的正确率,说明了该脑控方法的性能与刺激被试的范式有关;再次验证了CCA算法在基于SSVEP的脑机交互中具有优良的效果.最后,为克服单一SSVEP范式存在的局限,本研究也尝试把该范式与运动想象相结合的混合范式用于脑控机器人方向和速度,并进行了初步的研究,表明可以进一步改善控制速度和提高被试舒适度.本文可望为基于SSVEP或与运动想象混合的脑机交互应用于分级或精细控制机器人方向和速度提供思路,并为直接脑控机器人技术推向实际应用打下一定的基础.
  • 图  1  直接脑控机器人系统

    Fig.  1  Direct brain-controlled robot system

    图  2  脑控机器人测试平台及脑控机器人系统客户/服务器结构

    Fig.  2  The test platform and client/server architecture \\for brain-controlled robot system

    图  3  SSVEP脑机交互刺激范式一

    Fig.  3  The first SSVEP-based BCI stimulation paradigm

    图  4  SSVEP脑机交互刺激范式二和范式三

    Fig.  4  The second and the third SSVEP-based BCI stimulation paradigm

    图  5  基于SSVEP脑机交互直接脑控机器人系统结构

    Fig.  5  The structure of SSVEP-based BCI directly brain-controlled robot

    图  6  三种SSVEP刺激范式下被试脑控机器人达到终点平均用时及触碰障碍物次数及在1 $\sim$ 4秒刺激时间下的平均正确识别率

    Fig.  6  The averaged consuming time,number of touching obstacles and classification accuracies across 8 subjects when their controlling robot by their brains to reach the destination under three SSVEP stimulation paradigms

    表  1  一种脑控机器人策略: SSVEP 脑机交互刺激范式刺激目标对应的脑控制指令

    Table  1  A strategy for brain-controlled robot: control commands corresponding to the stimulus targets of SSVEP-based BCI stimulation paradigm

    F(Forward)BCIID01CA1000低速前进
    F+BCIID01CA2000中速前进
    F++BCIID01CA3000高速前进
    B(Backward)BCIID01CA0100低速后退
    B+BCIID01CA0200中速后退
    L(Left)BCIID01CA0010低速左转
    L+BCIID01CA0020中速左转
    R(Right)BCIID01CA0001低速右转
    R+BCIID01CA0002中速右转
    下载: 导出CSV

    表  2  三种SSVEP 刺激范式下被试脑控机器人达到终点用时及触碰障碍物次数

    Table  2  The consuming time and the number of touching obstacles when subjects controlling robot by their brains to reach the destination under three SSVEP stimulation paradigms

    SSSVEP刺激范式一SSVEP刺激范式二SSVEP刺激范式三
    用时(min)触碰次数 用时(min)触碰次数用时(min)触碰次数
    S1601854036330593
    602633054230360
    505644012230411
    603264057320461
    S2504834020130392
    502644029330010
    505034021220190
    S3500323039130020
    504343054220431
    603853004020370
    S4505544020330563
    505243039130172
    501843035110590
    S5404833026220321
    404433005120090
    504643022220470
    S6600963051230142
    505443025120250
    502752050210290
    S7405953009220271
    405562059210471
    701653053130422
    S8601143054230250
    604943013230241
    Aver50354.1730321.7520401.88
    Min404422050010290
    Var0.391.060.390.600.460.64
    下载: 导出CSV

    表  3  SSVEP脑机交互刺激范式一下刺激目标不同闪烁时间被试的正确识别率(%)

    Table  3  The correct recognition rate (%) at different flickering durations for 8 subjects under the first SSVEP-based BCI stimulation paradigm

    STd=1sTd=2sTd=3sTd=4s
    S133.3366.6783.3386.67
    S236.6770.0086.6793.33
    S330.0063.338086.67
    S430.0060.0076.6780
    S546.6780.0093.33100
    S643.3380.009096.67
    S736.6770.0086.6790
    S840.0066.6783.3386.67
    Aver37.0869.588590
    Max46.6780.0093.33100
    Var31.7845.6624.9936.11
    下载: 导出CSV

    表  4  SSVEP脑机交互刺激范式二下刺激目标不同闪烁时间被试的正确识别率(%)

    Table  4  The correct recognition rate (%) at different flickering durations for 8 subjects under the second SSVEP-based BCI stimulation paradigm stimulation paradigm

    STd=1sTd=2sTd=3sTd=4s
    S140.0073.3386.6793.33
    S243.3380.009096.67
    S336.6773.3386.6793.33
    S440.0076.679093.33
    S556.6786.6796.67100
    S653.3383.3393.33100
    S743.3376.679093.33
    S846.6780.0093.3396.67
    Aver45.0078.7590.8395.83
    Max56.6786.6796.67100
    Var41.6719.2810.417.65
    下载: 导出CSV

    表  5  脑机交互刺激范式三下刺激目标不同闪烁时间被试的正确识别率(%)

    Table  5  The correct recognition rate (%) at different flickering durations for 8 subjects under the third SSVEP-based BCI stimulation paradigm stimulation paradigm

    S Td=1 s Td=2 s Td=3 s Td=4 s45
    S1 43.33 80.00 90.00 93.3345
    S2 46.67 83.33 93.33 96.6745
    S3 43.33 80.00 90.00 93.3345
    S4 40.00 76.67 90.00 96.6745
    S5 60.00 86.67 96.67 10045
    S6 53.33 86.67 96.67 10045
    S7 46.67 83.33 93.33 93.3345
    S8 46.67 83.33 96.67 10045
    Aver 47.50 82.50 93.33 96.6745
    Max 60.00 86.67 96.67 10045
    Var 35.42 10.42 8.34 8.3445
    下载: 导出CSV
  • [1] Brain-computer interface from Wikipedia, the free encyclopedia [Online], available: http://en.wikipedia.org/wiki/ Brain-computer_interface, April 29, 2016.
    [2] Hochberg L R, Bacher D, Jarosiewicz B, Masse N Y, Simeral J D, Vogel J, Haddadin S, Liu J, Cash S S, van der Smagt P, Donoghue J P. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature, 2012, 485(7398): 372-375 doi: 10.1038/nature11076
    [3] 伏云发, 王越超, 李洪谊, 徐保磊, 李永程. 直接脑控机器人接口技术. 自动化学报, 2012, 38(8): 1229-1246 doi: 10.3724/SP.J.1004.2012.01229

    Fu Yun-Fa, Wang Yue-Chao, Li Hong-Yi, Xu Bao-Lei, Li Yong-Cheng. Direct brain-controlled robot interface technology. Acta Automatica Sinica, 2012, 38(8): 1229-1246 doi: 10.3724/SP.J.1004.2012.01229
    [4] McFarland D J, Wolpaw J R. Brain-computer interface operation of robotic and prosthetic devices. Computer, 2008, 41(10): 52-56 doi: 10.1109/MC.2008.409
    [5] Bell C J, Shenoy P, Chalodhorn R, Rao R P N. Control of a humanoid robot by a noninvasive brain-computer interface in humans. Journal of Neural Engineering, 2008, 5(2): 214-220 doi: 10.1088/1741-2560/5/2/012
    [6] Millan J R, Renkens F, Mourino J, Gerstner W. Noninvasive brain-actuated control of a mobile robot by human EEG. IEEE Transactions on Biomedical Engineering, 2004, 51(6): 1026-1033 doi: 10.1109/TBME.2004.827086
    [7] Comparison of consumer brain-computer interfaces from Wikipedia, the free encyclopedia [Online], available: http://en.wikipedia.org/wiki/Comparison_of_consumer_ brain-computer_interface_devices, December 20, 2015.
    [8] 伏云发, 徐保磊, 李永程, 李洪谊, 王越超, 余正涛. 基于运动相关皮层电位握力运动模式识别研究. 自动化学报, 2014, 40(6): 1045-1057 http://www.aas.net.cn/CN/abstract/abstract18374.shtml

    Fu Yun-Fa, Xu Bao-Lei, Li Yong-Cheng, Li Hong-Yi, Wang Yue-Chao, Yu Zheng-Tao. Recognition of actual grip force movement modes based on movement-related cortical potentials. Acta Automatica Sinica, 2014, 40(6): 1045-1057 http://www.aas.net.cn/CN/abstract/abstract18374.shtml
    [9] SchlÖgl A, Lee F, Bischof H, Pfurtscheller G. Characterization of four-class motor imagery EEG data for the BCI-competition 2005. Journal of Neural Engineering, 2005, 2(4): L14-L22 http://cn.bing.com/academic/profile?id=2057219090&encoded=0&v=paper_preview&mkt=zh-cn
    [10] 孙会文, 伏云发, 熊馨, 杨俊, 刘传伟, 余正涛. 基于HHT运动想象脑电模式识别研究. 自动化学报, 2015, 41(9): 1686-1692 http://www.aas.net.cn/CN/abstract/abstract18742.shtml

    Sun Hui-Wen, Fu Yun-Fa, Xiong Xin, Yang Jun, Liu Chuan-Wei, Yu Zheng-Tao. Identification of EEG induced by motor imagery based on Hilbert-Huang transform. Acta Automatica Sinica, 2015, 41(9): 1686-1692 http://www.aas.net.cn/CN/abstract/abstract18742.shtml
    [11] Fu Y F, Xu B L, Li Y C, Wang Y C, Yu Z T, Li H Y. Single-trial decoding of imagined grip force parameters involving the right or left hand based on movement-related cortical potentials. Chinese Science Bulletin, 2014, 59(16): 1907-1916 doi: 10.1007/s11434-014-0234-5
    [12] Lv J, Li Y Q, Gu Z H. Decoding hand movement velocity from electroencephalogram signals during a drawing task. Biomedical Engineering Online, 2010, 9: 64 doi: 10.1186/1475-925X-9-64
    [13] Yin X X, Xu B L, Jiang C H, Fu Y F, Wang Z D, Li H Y, Shi G. A hybrid BCI based on EEG and fNIRS signals improves the performance of decoding motor imagery of both force and speed of hand clenching. Journal of Neural Engineering, 2015, 12(3): 036004, DOI: 10.1088/1741-2560/12/3/036004
    [14] 尧德中, 刘铁军, 雷旭, 杨平, 徐鹏, 张杨松. 基于脑电的脑—机接口: 关键技术和应用前景. 电子科技大学学报, 2009, 38(5): 550-554 http://www.cnki.com.cn/Article/CJFDTOTAL-DKDX200905010.htm

    Yao De-Zhong, Liu Tie-Jun, Lei Xu, Yang Ping, Xu Peng, Zhang Yang-Song. Electroencephalogram based brain-computer interface: key techniques and application prospect. Journal of University of Electronic Science and Technology of China, 2009, 38(5): 550-554 http://www.cnki.com.cn/Article/CJFDTOTAL-DKDX200905010.htm
    [15] 王行愚, 金晶, 张宇, 王蓓. 脑控: 基于脑—机接口的人机融合控制. 自动化学报, 2013, 39(3): 208-221 doi: 10.1016/S1874-1029(13)60023-3

    Wang Xing-Yu, Jin Jing, Zhang Yu, Wang Bei. Brain control: human-computer integration control based on brain-computer interface. Acta Automatica Sinica, 2013, 39(3): 208-221 doi: 10.1016/S1874-1029(13)60023-3
    [16] Zhang R, Yao D, Valdés-Sosa P A, Li F L, Li P Y, Zhang T, Ma T, Li Y J, Xu P. Efficient resting-state EEG network facilitates motor imagery performance. Journal of Neural Engineering, 2015, 12(6): 066024, DOI: 10.1088/1741-2560/12/6/066024
    [17] Fazel-Rezai R, Allison B Z, Guger C, Sellers E W, Kleih S C, Kübler A. P300 brain computer interface: current challenges and emerging trends. Frontiers in Neuroengineering, 2012, 5: 14 http://cn.bing.com/academic/profile?id=2105438329&encoded=0&v=paper_preview&mkt=zh-cn
    [18] Gao S K, Wang Y J, Gao X R, Hong B. Visual and auditory brain-computer interfaces. IEEE Transactions on Biomedical Engineering, 2014, 61(5): 1436-1447 doi: 10.1109/TBME.2014.2300164
    [19] Chen X G, Wang Y J, Nakanishi M, Gao X R, Jung T P, Gao S K. High-speed spelling with a noninvasive brain-computer interface. Proceedings of the National Academy of Sciences of the United States of America, 2015, 112(44): E6058-E6067 http://cn.bing.com/academic/profile?id=2132876794&encoded=0&v=paper_preview&mkt=zh-cn
    [20] Bin G Y, Gao X R, Yan Z, Hong B, Gao S K. An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method. Journal of Neural Engineering, 2009, 6(4): 1771-1779 http://cn.bing.com/academic/profile?id=2105478324&encoded=0&v=paper_preview&mkt=zh-cn
    [21] The 2th competition of China brain-computer interface [Online], available: http://www.bcicompetition.cn, December 22, 2015.
    [22] Lin Z L, Zhang C S, Wu W, Gao X R. Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs. IEEE Transactions on Biomedical Engineering, 2006, 53(12): 2610-2614 doi: 10.1109/TBME.2006.886577
    [23] Long J Y, Li Y Q, Wang H T, Yu T Y, Pan J H, Li F. A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair. IEEE Transactions on Neural Systems & Rehabilitation Engineering, 2012, 20(5): 720-729 http://cn.bing.com/academic/profile?id=2001471870&encoded=0&v=paper_preview&mkt=zh-cn
    [24] Middendorf M, McMillan G, Calhoun G, Jones K S. Brain-computer interfaces based on the steady-state visual-evoked response. IEEE Transactions on Rehabilitation Engineering, 2000, 8(2): 211-214 doi: 10.1109/86.847819
    [25] Mak J N, Arbel Y, Minett J W, McCane L M, Yuksel B, Ryan D, Thompson D, Bianchi L, Erdogmus D. Optimizing the P300-based brain-computer interface: current status, limitations and future directions. Journal of Neural Engineering, 2011, 8(2): 025003, DOI: 10.1088/1741-2560/8/2/025003
    [26] Guger C, Edlinger G, Harkam W, Niedermayer I, Pfurtscheller G. How many people are able to operate an EEG-based brain-computer interface (BCI)? IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2003, 11(2): 145-147 doi: 10.1109/TNSRE.2003.814481
    [27] Guger C, Allison B Z, GroÜwindhager B, PrÜckl R, HintermÜller C, Kapeller C, Bruckner M, Krausz G, Edlinger G. How many people could use an SSVEP BCI? Frontiers in Neuroscience, 2012, 6: 169 http://cn.bing.com/academic/profile?id=2023666981&encoded=0&v=paper_preview&mkt=zh-cn
    [28] Jin J, Daly I, Zhang Y, Wang X Y, Cichocki A. An optimized ERP brain-computer interface based on facial expression changes. Journal of Neural Engineering, 2014, 11(3): 1082-1088 http://cn.bing.com/academic/profile?id=2089448644&encoded=0&v=paper_preview&mkt=zh-cn
  • 加载中
图(6) / 表(5)
计量
  • 文章访问数:  2326
  • HTML全文浏览量:  258
  • PDF下载量:  1747
  • 被引次数: 0
出版历程
  • 收稿日期:  2015-12-29
  • 录用日期:  2016-06-22
  • 刊出日期:  2016-11-01

目录

    /

    返回文章
    返回