2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于机器学习和几何变换的实时2D/3D脊椎配准

陈智强 王作伟 方龙伟 菅凤增 吴毅红 李硕 何晖光

陈智强, 王作伟, 方龙伟, 菅凤增, 吴毅红, 李硕, 何晖光. 基于机器学习和几何变换的实时2D/3D脊椎配准. 自动化学报, 2018, 44(7): 1183-1194. doi: 10.16383/j.aas.2017.c160711
引用本文: 陈智强, 王作伟, 方龙伟, 菅凤增, 吴毅红, 李硕, 何晖光. 基于机器学习和几何变换的实时2D/3D脊椎配准. 自动化学报, 2018, 44(7): 1183-1194. doi: 10.16383/j.aas.2017.c160711
CHEN Zhi-Qiang, WANG Zuo-Wei, FANG Long-Wei, JIAN Feng-Zeng, WU Yi-Hong, LI Shuo, HE Hui-Guang. Real-time 2D/3D Registration of Vertebra via Machine Learning and Geometric Transformation. ACTA AUTOMATICA SINICA, 2018, 44(7): 1183-1194. doi: 10.16383/j.aas.2017.c160711
Citation: CHEN Zhi-Qiang, WANG Zuo-Wei, FANG Long-Wei, JIAN Feng-Zeng, WU Yi-Hong, LI Shuo, HE Hui-Guang. Real-time 2D/3D Registration of Vertebra via Machine Learning and Geometric Transformation. ACTA AUTOMATICA SINICA, 2018, 44(7): 1183-1194. doi: 10.16383/j.aas.2017.c160711

基于机器学习和几何变换的实时2D/3D脊椎配准

doi: 10.16383/j.aas.2017.c160711
基金项目: 

国家高技术研究发展计划(863计划) 2013AA013803

中国自然科学基金 91520202

详细信息
    作者简介:

    陈智强  中国科学院自动化研究所类脑智能研究中心博士研究生.2014年获得北京科技大学智能科学与技术系学士学位.2017年获得中国科学院大学硕士学位.主要研究方向为医学影像分析, 类脑智能.E-mail:chenzhiqiang2014@ia.ac.cn

    王作伟  北京医院神经外科副主任医师.主要研究方向为脊柱微创手术.E-mail:wzw6855@163.com

    方龙伟  中国科学院自动化研究所类脑智能研究中心博士研究生.2010年获得中国矿业大学学士学位.2013年获得北京航空航天大学硕士学位.主要研究方向为医学影像.E-mail:fanglongwei2014@ia.ac.cn

    菅凤增  首都医科大学宣武医院神经外科主任医师.主要研究方向颈椎病, 腰椎病.E-mail:fengzengjian@hotmail.com

    吴毅红  中国科学院自动化研究所模式识别国家重点实验室研究员.主要研究方向为计算机视觉.E-mail:yihong.wu@ia.ac.cn

    李硕  加拿大西安大略大学医学生物物理系副教授.主要研究方向为医学影像.E-mail:slishuo@gmail.com

    通讯作者:

    何晖光  中国科学院自动化研究所类脑智能研究中心研究员, 中国科学院大学岗位教授.主要研究方向为类脑智能, 脑机接口, 医学影像分析.本文通信作者.E-mail:huiguang.he@ia.ac.cn

Real-time 2D/3D Registration of Vertebra via Machine Learning and Geometric Transformation

Funds: 

National High Technology Research and Development Program of China (863 Program) 2013AA013803

National Natural Science Foundation of China 91520202

More Information
    Author Bio:

     Ph. D. candidate at the Research Center for Brain-inspired Intelligence, Institute of Automation, Chinese Academy of Sciences. He received his bachelor degree from University of Science and Technology Beijing in 2014. He received his master degree from University of Chinese Academy of Sciences in 2017. His research interest covers medical image analysis, and brain-like intelligence

     Associate chief physician of Neurosurgery, Beijing Hospital. His main research interest is minimally invasive spine surgery

     Ph. D. candidate at the Research Center for Brain-inspired Intelligence, Institute of Automation, Chinese Academy of Sciences. He received his bachelor degree from China University of Mining and Technology in 2010. He received his master degree from Beihang University in 2013. His main research interest is medical image

     Chief physician of neurosurgery, Capital Medical University Xuanwu Hospital. His research interest covers cervical and lumbar spondylosis

     Professor at the State Key Laboratory of pattern recognition, Institute of automation, Chinese Academy of Sciences. Her main research interest is computer vision

     Associate professor in the Department of Medical Biophysics, University of Western Ontario, Canada. His main research interest is medical image

    Corresponding author: HE Hui-Guang  Professor at the Research Center for Brain-inspired Intelligence, Institute of automation, Chinese Academy of Sciences and professor in University of the Chinese Academy of Sciences. His research interest covers brain-inspire intelligence, brain computer interface, and medical image analysis. Corresponding author of this paper
  • 摘要: 在图像引导的脊柱手术中,实时高效的2D/3D配准是一项重要且具有挑战性的任务.通常的2D/3D配准一般是将三维图像投影到二维平面,然后进行2D/2D的配准.由于投影空间涉及到3个平移以及3个旋转参数,其投影空间的复杂度为O(n6),使得配准很难兼具高准确性和高实时性.本文提出了一个结合机器学习与几何变换的2D/3D配准方法,首先,使用统计形状模型对目标脊椎进行建模,并构建了一种新的投影方式,使得6个投影参数中的4个可以使用几何的方法计算出来;接下来利用回归学习的方法学习目标脊椎的形状与投影参数之间的关系;最终,结合学到的关系和几何变换完成配准.本方法的两个姿态参数的平均预测误差为0.84°和0.81°,平均目标配准误差(Mean target registration error,mTRE)为0.87mm,平均配准时间为0.9s.实验结果表明本方法具有很好的实时性和准确性.
    1)  本文责任编委 黄庆明
  • 图  1  传统2D/3D配准流程

    Fig.  1  Traditional 2D/3D registration process

    图  2  总体流程图

    Fig.  2  Overall flow chart

    图  3  提取标志点(我们对每幅图像手动提取93个标志点, 所有标志点总体分为6个部分: (a)为所有的标志点, (b)、(c)、(d)、(e)、(f)、(g)为6个部分每个部分的标志点, 其中(b)为椎体轮廓, (c)为中央灰度凹陷, (d)和(e)接近于生理结构的椎弓根, (f)和(g)为椎体左右下切角)

    Fig.  3  Extract landmarks (We manually extract 93 landmarks for each image, all landmarks are divided into 6 parts, in which (a) contains all landmarks, while (b), (c), (d), (e), (f), (g) contain one of 6 parts, among them, (b) is vertebral body contour, (c) is the central gray depression, (d) and (e) are close to the pedicle of the physiological structure, and (f) and (g) are the left and right bottom of the vertebral body.)

    图  4  投影变换((a)图是传统的投影方式, ($x, y, z$)为世界坐标系, ($t_x, t_y, t_z$)是三个平移参数($r_x, r_y, r_z$)为三个旋转参数. (b)图是本文构建的投影方式, ($x, y, z$)为世界坐标系, ($x'O'y'$)为投影平面坐标系, 此坐标系沿世界坐标系$z$轴向投影与($xOy$)重合. ($r, \theta, \phi$)为球形坐标系的参数. ($u, v, w$)为投影对象自身姿态坐标系, 此姿态坐标系与球形坐标系联动, 姿态坐标系的$u$轴平行于球坐标系的纬线切线指向如图 4(b)中所示方向, $v$轴平行于球坐标系的经线指向如图 4(b)中所示所示方向, $w$轴沿径向方法指向背离圆心的方向.)

    Fig.  4  Projection transformation ((a) is traditional projection transformation, and ($x, y, z$)is world coordinate system, and ($t_x, t_y, t_z$) are three translation parameters, while ($r_x, r_y, r_z$) are three rotation parameters. b) is the proposed projection method. ($x, y, z$) is the world coordinate system, and $x'O'y'$) is the coordinate system of projective plane. This coordinate system coincides with the axial projection of $z$ in the world coordinate system ($xOy$). ($r, \theta, \phi$) are parameters for the spherical coordinate system. ($u, v, w$) is pose projection coordinates of object, and it coact with spherical coordinates as the (b) shows.)

    图  5  本文配准示意图

    Fig.  5  Illustration of registration in this paper

    图  6  线性和高阶拟合((a)和(b))与不同采样间隔下预测误差的变化((c)和(d))

    Fig.  6  The difference of prediction error between linear model and high order model (a), (b) and by difference sampling intervals (c), (d)

    图  7  配准结果

    Fig.  7  The results of registration

    A1  第$\Delta r$对灰度的影响

    A1  The effect to image of $\Delta r$

    A2  $\Delta r$近似等效于相似变换

    A2  The effect of $\Delta r$ approximate to similar transformation

    A3  投影变换拆解

    A3  3 steps of projection

    表  1  配准结果

    Table  1  Results of registration

    对象 姿态误差ru(°) 姿态误差rv(°) mTRE (mm) 时间(s)
    PA1 0.92±0.69 0.88±0.71 0.88±0.73 0.96
    PA2 0.62±0.51 0.70±0.62 1.13±0.75 0.88
    PA3 0.52±0.44 0.70±0.58 1.01±0.62 0.88
    PA4 1.43±1.05 1.13±0.92 0.77±0.58 0.89
    PA5 0.78±0.61 0.62±0.48 0.73±0.45 0.88
    PA6 0.76±0.63 0.81±0.64 0.68±0.46 0.88
    平均 0.84 0.81 0.87 0.90
    下载: 导出CSV

    表  2  各种方法对比

    Table  2  Comparison with other methods

    作者 方法框架 相似度度量 mTRE (mm) 时间(s)
    Russakof 基于搜索 互信息 1.3 -
    Russakof 基于搜索 交叉相关 1.5 -
    Russakof 基于搜索 梯度相关 1.3 -
    Russakof 基于搜索 灰度模式 1.6 -
    Russakof 基于搜索 梯度差 1.3 -
    Russakof 基于搜索 Diff.图像熵 1.9 -
    Otake 基于搜索 NGI - 6.3 ~ 54
    Philipp 基于学习 纹理 1.05 0.02
    本文 基于学习 形状 0.87 0.90
    下载: 导出CSV
  • [1] Wu J, Kim M, Peters J, Chung H, Samant S S. Evaluation of similarity measures for use in the intensity-based rigid 2D-3D registration for patient positioning in radiotherapy. Medical Physics, 2009, 36(12):5391-5403 doi: 10.1118/1.3250843
    [2] Russakoff D B, Rohlfing T, Adler J R, Maurer C R. Intensity-based 2D-3D spine image registration incorporating a single fiducial marker. Academic Radiology, 2005, 12(1):37-50 doi: 10.1016/j.acra.2004.09.013
    [3] Otake Y, Wang A S, Webster S J, Uneri A, Kleinszig G, Vogt S, Khanna A J, Gokaslan Z L, Siewerdsen J H. Robust 3D-2D image registration:application to spine interventions and vertebral labeling in the presence of anatomical deformation. Physics in Medicine and Biology, 2013, 58(23):8535-8553 doi: 10.1088/0031-9155/58/23/8535
    [4] Russakoff D B, Rohlfing T, Ho A, Kim D H, Shahidi R, Adler J R, Maurer C R. Evaluation of intensity-based 2D-3D spine image registration using clinical gold-standard data. In:Proceedings of the 2nd International Workshop on Biomedical Image Registration (WBIR 2003). Philadelphia, PA, USA:Springer-Verlag, 2003. 151-160
    [5] Zollei L, Grimson E, Norbash A, Wells W. 2D-3D rigid registration of X-ray fluoroscopy and CT images using mutual information and sparsely sampled histogram estimators. In:Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001). Kauai, HI, USA:IEEE, 2001. Ⅱ-696-Ⅱ-703
    [6] Kim J, Yin F F, Zhao Y, Kim H J. Effects of X-ray and CT image enhancements on the robustness and accuracy of a rigid 3D/2D image registration. Medical Physics, 2005, 32(4):866-873 doi: 10.1118/1.1869592
    [7] Chen H M, Goela A, Garvin G J, Li S. A parameterization of deformation fields for diffeomorphic image registration and its application to myocardial delineation. In:Proceedings of the 13th international conference on Medical Image Computing and Computer-Assisted Intervention——(MICCAI 2010). Beijing, China:Springer-Verlag, 2010. 340-348
    [8] Pohl K M, Wells W M, Guimond A, Kasai K, Shenton M E, Kikinis R, et al. Incorporating non-rigid registration into expectation maximization algorithm to segment MR images. In:Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention——(MICCAI 2002). Tokyo, Japan:Springer-Verlag, 2002. 564-571
    [9] Chou C R, Frederick B, Mageras G, Chang S, Pizer S. 2D/3D Image Registration using Regression Learning. Computer Vision and Image Understanding, 2013, 117(9):1095-1106 doi: 10.1016/j.cviu.2013.02.009
    [10] Cyr C M, Kamal A F, Sebastian T B, Kimia B B. 2D-3D registration based on shape matching. In:Proceedings of the 2000 IEEE Workshop on Mathematical Methods in Biomedical Image Analysis. Washington, D.C., USA:IEEE Computer Society, 2000. 198
    [11] Philipp S, Markus N, Karl F, Felix S, Deutschmann H. A novel class of machine-learning-driven real-time 2D/3D tracking methods:texture model registration (TMR). Proceedings of the SPIE, 2011, DOI: 10.1117/12.878147
    [12] Gouveia A R, Metz C, Freire L, Klein S. 3D-2D image registration by nonlinear regression. In:Proceedings of the 9th IEEE International Symposium on Biomedical Imaging. Barcelona, Spain:IEEE, 2012. 1343-1346
    [13] Cootes T F, Taylor C J, Cooper D H, Graham J. Active shape models-their training and application. Computer Vision and Image Understanding, 1995, 61(1):38-59 doi: 10.1006/cviu.1995.1004
    [14] Cootes T F, Edwards G J, Taylor C J. Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(6):681-685 doi: 10.1109/34.927467
    [15] van Ginneken B, Frangi A F, Staal J J, Ter Haar Romeny B M, Viergever M A. Active shape model segmentation with optimal features. IEEE Transactions on Medical Imaging, 2002, 21(8):924-933 doi: 10.1109/TMI.2002.803121
    [16] Cootes T F, Hill A, Taylor C J, Haslam J. The use of active shape models for locating structures in medical images. In:Proceedings of the 13th International Conference on Information Processing in Medical Imaging. London, UK:Springer-Verlag, 1993. 33-47
    [17] Gower J G. Procrustes Analysis. International Encyclopedia of the Social and Behavioral Sciences, 2001, 39(4):12141-12143
    [18] Lee D T, Schachter B J. Two algorithms for constructing a Delaunay triangulation. International Journal of Parallel Programming, 1980, 9(3):219-242
  • 加载中
图(10) / 表(2)
计量
  • 文章访问数:  2937
  • HTML全文浏览量:  489
  • PDF下载量:  1120
  • 被引次数: 0
出版历程
  • 收稿日期:  2016-10-11
  • 录用日期:  2017-06-23
  • 刊出日期:  2018-07-20

目录

    /

    返回文章
    返回