2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于单字符注意力的全品类鲁棒车牌识别

穆世义 徐树公

穆世义, 徐树公. 基于单字符注意力的全品类鲁棒车牌识别. 自动化学报, 2023, 49(1): 122−134 doi: 10.16383/j.aas.c211210
引用本文: 穆世义, 徐树公. 基于单字符注意力的全品类鲁棒车牌识别. 自动化学报, 2023, 49(1): 122−134 doi: 10.16383/j.aas.c211210
Mu Shi-Yi, Xu Shu-Gong. Full-category robust license plate recognition based on character attention. Acta Automatica Sinica, 2023, 49(1): 122−134 doi: 10.16383/j.aas.c211210
Citation: Mu Shi-Yi, Xu Shu-Gong. Full-category robust license plate recognition based on character attention. Acta Automatica Sinica, 2023, 49(1): 122−134 doi: 10.16383/j.aas.c211210

基于单字符注意力的全品类鲁棒车牌识别

doi: 10.16383/j.aas.c211210
基金项目: 国家自然科学基金(61871262)资助
详细信息
    作者简介:

    穆世义:上海大学通信与信息工程学院硕士研究生. 主要研究方向为深度学习, 计算机视觉和光学字符识别. E-mail: mushiyishu@shu.edu.cn

    徐树公:上海大学通信与信息工程学院教授. 主要研究方向为无线通信和模式识别. 本文通信作者. E-mail: shugong@shu.edu.cn

Full-category Robust License Plate Recognition Based on Character Attention

Funds: Supported by National Natural Science Foundation of China (61871262)
More Information
    Author Bio:

    MU Shi-Yi Master student at the School of Communication and Information Engineering, Shanghai University. His research interest covers deep learning, computer vision and optical character recognition

    XU Shu-Gong Professor at the School of Communication and Info-rmation Engineering, Shanghai University. His research interest covers wireless communication and pattern recognition. Corresponding author of this paper

  • 摘要: 复杂场景下的高精度车牌识别仍然存在着许多挑战, 除了光照、分辨率不可控和运动模糊等因素导致的车牌图像质量低之外, 还包括车牌品类多样产生的行数不一和字数不一等困难, 以及因拍摄角度多样出现的大倾角等问题. 针对这些挑战, 提出了一种基于单字符注意力的场景鲁棒的高精度车牌识别算法, 在无单字符位置标签信息的情况下, 使用注意力机制对车牌全局特征图进行单字符级特征分割, 以处理多品类车牌和倾斜车牌中的二维字符布局问题. 另外, 该算法通过使用共享参数的多分支结构代替现有算法的串行解码结构, 降低了分类头参数量并实现了并行化推理. 实验结果表明, 该算法在公开车牌数据集上实现了超越现有算法的精度, 同时具有较快的识别速度.
  • 图  1  注意力机制改进

    Fig.  1  Evolution of attention mechanism

    图  2  CARNet算法结构图

    Fig.  2  Framework of the proposed algorithm CARNet

    图  3  轻量化特征提取

    Fig.  3  Lightweight feature extraction

    图  4  单字符注意力网络

    Fig.  4  Single character attention network

    图  5  单字符特征分割

    Fig.  5  Single character feature segmentation

    图  6  脚本生成的车牌样本

    Fig.  6  License plate samples generated by script

    图  7  常见7字符车牌注意力图

    Fig.  7  Attention maps of seven-character license plates

    图  8  双行黄牌、新能源车牌及黑色车牌注意力图

    Fig.  8  Attention maps of double-line and new energy and black plate licenses

    图  9  真实复杂场景下的检测识别测试

    Fig.  9  Detection and recognition test in complex scene

    图  10  识别错误示例

    Fig.  10  Recognition error cases

    表  1  在CCPD上的车牌识别准确率(%)

    Table  1  License plate recognition accuracy on CCPD (%)

    算法平均基础集明暗集远近集旋转集倾斜集天气集挑战集
    Li 等[1]94.497.894.894.587.992.186.881.2
    Xu 等[27]95.598.596.994.390.892.587.985.1
    Wang 等[23]96.698.996.196.491.993.795.483.1
    Zou 等[8]97.899.398.598.692.594.499.386.6
    Yang 等[4]97.599.196.995.997.198.097.585.9
    Qin 等[33]97.599.593.393.798.295.998.992.9
    Qiao 等[34]96.999.097.195.595.096.595.983.1
    Zhang 等[20]98.599.698.898.896.497.698.588.9
    Liu 等[35]98.7499.7399.0599.2397.6298.4098.8988.51
    GCN98.7999.7099.0798.9698.3398.8298.6689.42
    CARNet99.50
    (0.02)
    99.89
    (0.01)
    99.57
    (0.08)
    99.56
    (0.04)
    99.68
    (0.04)
    99.80
    (0.01)
    99.38
    (0.06)
    94.92
    (0.09)
    下载: 导出CSV

    表  2  本文算法有效性评估(%)

    Table  2  Evaluation of the effectiveness of the algorithm of this paper (%)

    评估指标算法平均基础集明暗集远近集旋转集倾斜集天气集挑战集
    $ {R_{LP}} $GCN98.7999.7099.0798.9698.3398.8298.6689.42
    (0.10)(0.03)(0.11)(0.11)(0.25)(0.19)(0.13)(0.54)
    CARNet99.5099.8999.5799.5699.6899.8099.3894.92
    (0.02)(0.01)(0.08)(0.04)(0.04)(0.01)(0.06)(0.09)
    $ {R_{Char}} $GCN99.7499.9599.8399.7999.6899.7899.7797.28
    (0.02)(0.01)(0.02)(0.01)(0.05)(0.03)(0.02)(0.14)
    CARNet99.9099.9899.9499.9399.9599.9799.9098.89
    (0.01)(0.01)(0.01)(0.01)(0.01)(0.01)(0.01)(0.01)
    $ {R_{C\_Char}} $GCN99.7299.8799.7899.7899.5699.7199.7098.18
    (0.02)(0.01)(0.02)(0.01)(0.06)(0.03)(0.06)(0.07)
    CARNet99.9299.9999.9399.8999.9599.9899.9599.13
    (0.01)(0.01)(0.02)(0.02)(0.01)(0.01)(0.02)(0.01)
    $ {R_{W\_Char}} $GCN99.7499.9799.8399.8099.7099.8099.7897.13
    (0.02)(0.01)(0.02)(0.01)(0.05)(0.03)(0.01)(0.16)
    CARNet99.9099.9899.9499.9399.9599.9799.8998.85
    (0.01)(0.01)(0.01)(0.01)(0.01)(0.01)(0.01)(0.01)
    下载: 导出CSV

    表  3  在CLPD上的车牌识别准确率

    Table  3  License plate recognition accuracy on CLPD

    算法训练数据${R_{LP} }$(%)
    Zhang 等[20]CCPD 基础集70.8
    GCN[22]CCPD 基础集74.4
    Zou 等[8]CCPD 基础集80.7
    Liu 等[35]CCPD 基础集80.3
    CARNetCCPD 基础集82.2
    CARNet混合数据集83.9
    下载: 导出CSV

    表  4  在混合品类车牌上的识别准确率

    Table  4  Recognition accuracy on mixed types of license plates

    车牌类别数量 (张)GCN[22] (%)CARNet (%)
    蓝牌车牌105096.399.0
    新能源绿牌101041.978.5
    大型车后牌66035.862.1
    教练车牌86044.974.8
    港澳车牌158053.774.9
    大型车前牌98063.275.8
    下载: 导出CSV

    表  5  各算法速度比较

    Table  5  Comparison of algorithm speed

    车牌识别算法车牌识别耗时 (ms)
    Zhang 等[20]7.9
    GCN[22]18.7
    CARNet4.9
    下载: 导出CSV

    表  6  低功耗嵌入式硬件测试

    Table  6  Low-power embedded device test

    算法硬件平台推理引擎耗时 (ms)
    Qin 等[9]Jetson NanoTensorFlow68
    CARNetJetson NanoPytorch41
    CARNetJetson TX2Pytorch30
    CARNetHi3516DV300NNIE46
    下载: 导出CSV

    表  7  特征提取网络消融实验

    Table  7  Feature extraction ablation experiment

    特征提取$ {R_{LP}} $(%)参数量
    计算复杂度 (GMacs)
    Resnet45[25]99.513.94 M14.66
    Xception1999.51.87 M1.71
    下载: 导出CSV

    表  8  分类头参数共享消融实验

    Table  8  Classification head weight sharing ablation experiment

    参数共享$ {R_{LP}} $(%)参数量
    计算复杂度 (GMacs)
    99.51.87 M1.71
    99.43.82 M1.71
    下载: 导出CSV

    表  9  单字符注意力消融实验

    Table  9  Ablation experiments for single-character attention

    单字符注意力$ {R_{LP}} $(%)参数量
    计算复杂度 (GMacs)
    99.51.87 M1.71
    99.13.04 M1.02
    下载: 导出CSV
  • [1] Li H, Wang P, You M Y, Shen C H. Reading car license plates using deep neural networks. Image and Vision Computing, 2018, 72: 14-23 doi: 10.1016/j.imavis.2018.02.002
    [2] Wu C H, Xu S G, Song G C, Zhang S Q. How many labeled license plates are needed. In: Proceedings of the Chinese Conference on Pattern Recognition and Computer Vision. Guangzhou, China: 2018. 334−346
    [3] Huang G, Liu Z, Maaten L V, Weinberger K Q. Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017. 2261−2269
    [4] Yang Y X, Xi W, Zhu C K, Zhao Y H. HomoNet: Unified license plate detection and recognition in complex scenes. In: Proceedings of the International Conference on Collaborative Computing: Networking, Applications and Worksharing. Shanghai, China: 2020. 268−282
    [5] He M X, Hao P. Robust automatic recognition of Chinese license plates in natural scenes. IEEE Access, 2020, 8: 173804-173814 doi: 10.1109/ACCESS.2020.3026181
    [6] Lee C Y, Osindero S. Recursive recurrent nets with attention modeling for ocr in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 2231−2239
    [7] Cheng Z Z, Bai F, Xu Y L, Zheng G, Pu S L, Zhou S G. Focusing attention: Towards accurate text recognition in natural images. In: Proceedings of the IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017. 5086−5094
    [8] Zou Y J, Zhang Y J, Yan J, Jiang X X, Huang T J, Fan H S, et al. A robust license plate recognition model based on bi-LSTM. IEEE Access, 2020, 8: 211630-211641 doi: 10.1109/ACCESS.2020.3040238
    [9] Qin S X, Liu S J. Efficient and unified license plate recognition via lightweight deep neural network. IET Image Processing, 2020, 14(16): 4102-4109 doi: 10.1049/iet-ipr.2020.1130
    [10] Guo J M, Liu Y F. License plate localization and character segmentation with feedback self-learning and hybrid binarization techniques. IEEE Transactions on Vehicular Technology, 2008, 57(3): 1417-1424 doi: 10.1109/TVT.2007.909284
    [11] Silva S M, Jung C R. License plate detection and recognition in unconstrained scenarios. In: Proceedings of the European Conference on Computer Vision. Munich, Germany: 2018. 580−596
    [12] Gou C, Wang K F, Yao Y J, Li Z X. Vehicle license plate recognition based on extremal regions and restricted boltzmann machines. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(4): 1096-1107 doi: 10.1109/TITS.2015.2496545
    [13] Park S H, Yu S B, Kim J, Yoon H. An all-in-one vehicle type and license plate recognition system using yolov4. Sensors, 2022, 22(3): 921-939 doi: 10.3390/s22030921
    [14] Wang D, Tian Y M, Geng W H, Zhao L, Gong C. LPR-Net: Recognizing Chinese license plate in complex environments. Pattern Recognition Letters, 2020, 130: 148-156 doi: 10.1016/j.patrec.2018.09.026
    [15] Luo C J, Jin L W, Sun Z H. Moran: A multi-object rectified attention network for scene text recognition. Pattern Recognition, 2019, 90: 109-118 doi: 10.1016/j.patcog.2019.01.020
    [16] Jaderberg M, Simonyan K, Zisserman A, Kavukcuoglu K. Spatial transformer networks. In: Proceedings of the Annual Conference on Neural Information Processing Systems. Montreal, Can-ada: 2015. 2017−2025
    [17] 周晓君, 高媛, 李超杰, 阳春华. 基于多目标优化多任务学习的端到端车牌识别方法. 控制理论与应用, 2021, 38(5): 676−688

    Zhou Xiao-Jun, Gao Yuan, Li Chao-Jie, Yang Chun-Hua. Multi-objective optimization based multi-task learning for end-to-end car license plates recognition. Control Theory and Applications. 2021, 38(5): 676−688
    [18] Li H, Wang P, Shen C H, Zhang G Y. Show, attend and read: A simple and strong baseline for irregular text recognition. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Honolulu, USA: 2019. 8610−8617
    [19] Xu H, Guo Z H, Wang D H, Zhou X D, Shi Y. 2D license plate recognition based on automatic perspective rectification. In: Proceedings of the 25th International Conference on Pattern Recognition. Milan, Italy: IEEE, 2021. 202−208
    [20] Zhang L J, Wang P, Li H, Li Z, Shen C H. A robust attentional framework for license plate recognition in the wild. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(11): 6967-6976 doi: 10.1109/TITS.2020.3000072
    [21] Hu W Y, Cai X C, Hou J, Yi S, Lin Z P. GTC: Guided training of ctc towards efficient and accurate scene text recognition. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York, USA: 2020. 11005−11012
    [22] Yan R J, Peng L R, Xiao S Y, Yao G. Primitive representation learning for scene text recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Nashville, USA: IEEE, 2021. 284−293
    [23] Wang T W, Zhu Y Z, Jin L W, Luo C J, Chen X X, Wu Y Q, et al. Decoupled attention network for text recognition. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York, USA: AAAI, 2020. 12216−12224
    [24] Chollet F. Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017. 1251−1258
    [25] Shi B G, Yang M K, Wang X G, Lyu P Y, Yao C, Bai X. Aster: An attentional scene text recognizer with flexible rectification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 41(9): 2035-2048
    [26] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In: Proceedings of the International Conference on Medical Image Computing and Computer-assisted Intervention. Munich, Germany: 2015. 234− 241
    [27] Xu Z B, Yang W, Meng A J, Lu N X, Huang H, Ying C C, et al. Towards end-to-end license plate detection and recognition: A large dataset and baseline. In: Proceedings of the European Conference on Computer Vision. Munich, Germany: 2018. 255− 271
    [28] Zhao Y T, Yu Z, Li X Y. Evaluation methodology for license plate recognition systems and experimental results. IET Intelligent Transport Systems, 2018, 12(5): 375-385 doi: 10.1049/iet-its.2017.0138
    [29] Sun M, Zhou F, Yang C, Yin X C. Image generation framework for unbalanced license plate data Set. In: Proceedings of the International Conference on Data Mining Workshops. Beijing, China: IEEE, 2019. 883−889
    [30] Han B G, Lee J T, Lim K T, Choi D H. License plate image generation using generative adversarial networks for end-to-end license plate character recognition from a small set of real images. Applied Sciences, 2020, 10(8): 2780-2796 doi: 10.3390/app10082780
    [31] Sun Y F, Liu Q, Chen S L, Zhou F, Yin X C. Robust Chinese license plate generation via foreground text and background separation. In: Proceedings of the 11th International Conference on Image and Graphics. Haikou, China: 2021. 290−302
    [32] Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B. Generative adversarial nets. In: Proceedings of the Advances in Neural Information Processing Systems. Montreal, Canada: 2014. 2672− 1680
    [33] Qin S X, Liu S J. Towards end-to-end car license plate location and recognition in unconstrained scenarios. Neural Computing and Applications, to be published
    [34] Qiao L, Chen Y, Cheng Z Z, Xu Y L, Niu Y, Pu S L, et al. Mango: A mask attention guided one-stage scene text spotter. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence. Virtual Event: 2021. 2467−2476
    [35] Liu Q, Chen S L, Li Z J, Yang C, Chen F, Yin X C. Fast recognition for multidirectional and multi-type license plates with 2D spatial attention. In: Proceedings of the 16th International Conference on Document Analysis and Recognition. Lausanne, Swi-tzerland: 2021. 125−139
  • 加载中
图(10) / 表(9)
计量
  • 文章访问数:  742
  • HTML全文浏览量:  192
  • PDF下载量:  223
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-12-20
  • 录用日期:  2022-04-07
  • 网络出版日期:  2022-05-06
  • 刊出日期:  2023-01-07

目录

    /

    返回文章
    返回