2.845

2023影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

“结构−内容”框架下融合时空特征的技术预测模型

袭希 许伟 刘传斌 刘玮倩 苏忻洁

袭希, 许伟, 刘传斌, 刘玮倩, 苏忻洁. “结构−内容”框架下融合时空特征的技术预测模型. 自动化学报, xxxx, xx(x): x−xx doi: 10.16383/j.aas.c250094
引用本文: 袭希, 许伟, 刘传斌, 刘玮倩, 苏忻洁. “结构−内容”框架下融合时空特征的技术预测模型. 自动化学报, xxxx, xx(x): x−xx doi: 10.16383/j.aas.c250094
Xi Xi, Xu Wei, Liu Chuan-Bin, Liu Wei-Qian, Su Xin-Jie. A technology forecasting model integrating spatiotemporal features under the “structure-content” framework. Acta Automatica Sinica, xxxx, xx(x): x−xx doi: 10.16383/j.aas.c250094
Citation: Xi Xi, Xu Wei, Liu Chuan-Bin, Liu Wei-Qian, Su Xin-Jie. A technology forecasting model integrating spatiotemporal features under the “structure-content” framework. Acta Automatica Sinica, xxxx, xx(x): x−xx doi: 10.16383/j.aas.c250094

“结构−内容”框架下融合时空特征的技术预测模型

doi: 10.16383/j.aas.c250094 cstr: 32138.14.j.aas.c250094
基金项目: 国家自然科学基金面上项目 (72473142; 72374089) 资助
详细信息
    作者简介:

    袭希:中国人民大学信息学院博士后. 2013年获得哈尔滨工程大学管理学博士学位. 主要研究方向为商务智能与技术挖掘. E-mail: xix@ruc.edu.cn

    许伟:中国人民大学信息学院教授. 2009年获得中国科学院管理学博士学位.主要研究方向为数字金融、电子商务与智能社会治理. E-mail: weixu@ruc.edu.cn

    刘传斌:中国石油大学(北京)经济管理学院教授. 2023年获得哈尔滨工程大学管理学博士学位. 主要研究方向为数据挖掘、评价科学. E-mail: liuchuanbingl@163.com

    刘玮倩:哈尔滨商业大学管理学院硕士. 2025年获得管理学硕士学位. 主要研究方向为科技管理与技术挖掘. E-mail: 1527364361@qq.com

    苏忻洁:哈尔滨工程大学经济管理学院博士研究生. 2023年获得哈尔滨商业大学管理学硕士学位. 主要研究方向为商务智能与技术挖掘. E-mail: sxj_su@163.com

  • 中图分类号: Y

A Technology Forecasting Model Integrating Spatiotemporal Features under the “Structure-Content” Framework

Funds: National Natural Science Foundation of China(General Program)(72473142; 72374089)
More Information
    Author Bio:

    XI Xi Postdoctoral Researcher at the School of Information, Renmin University of China. She received her Ph.D. degree in Management from Harbin Engineering University in 2013. Her research interest covers business intelligence and technology mining

    XU Wei Professor at the School of Information, Renmin University of China. He received his Ph.D. degree in Management from Chinese Academy of Sciences in 2009. His research interest covers digital finance, e-commerce and intelligent social governance

    LIU Chuan-Bin Professor at the School of Economics and Management, China University of Petroleum. He received his Ph.D. degree in Management from Harbin Engineering University in 2023. His research interest covers data mining and evaluating science

    LIU Wei-Qian Master at the School of Management, Harbin University of Commerce. She received her Master degree in Management in 2025. Her research interest covers S&T management and technology mining

    SU Xin-Jie Ph.D candidate at the School of Economics and Management, Harbin Engineering University. She received her Master degree in Management from Habrin University of Commerce in 2023. Her research interest covers business intelligence and technology mining

  • 摘要: 科学技术发展是一种动态非线性的复杂演进过程.为了提升技术发展的精准预测, 本文基于大语言模型(Large Language Model, LLM)、图卷积神经网络(Graph Convolutional Networks, GCN)、双向长短期记忆神经网络(Bi-directional Long Short-Term Memory, BiLSTM)以及鲁棒随机配置网络(Robust Stochastic Configuration Networks, RSCN), 提出了一种全新的"结构−内容"时空技术预测模型(Spatiotemporal Technological Forecasting Model with LLM as Representation, STTeFL 模型).首先, 通过结合图卷积神经网络和双向长短期记忆神经网络, 分别捕捉技术网络中的空间依赖关系和时间演化规律, 从而突破了传统预测模型在动态性和结构表征上的局限性, 克服了传统技术预测模型的“伪动态”和“静态”限制; 其次, 引入大语言模型对技术网络中的节点特征和边特征进行双重语义表征, 将预测框架从单一的结构维度扩展至“结构−内容”双维度分析, 显著增强了模型对技术发展信息的理解能力和表征深度. 最后, 通过集成RSCN, 模型能够有效应对极端不均衡数据分布的挑战, 进一步提升了预测的鲁棒性和准确性.本文提出的预测框架在多个指标上均优于当前多种技术预测方法, 为推动技术预测建模和评估未来技术发展轨迹提供了有力的支持.
  • 图  1  本文提出的STTeFL模型整体架构

    Fig.  1  Overall architecture of the proposed STTeFL model

    图  2  分类性能的消融实验比较

    Fig.  2  Ablation experiment comparison of classification performance

    图  3  查准率和查全率及其不均衡处理性能的消融实验比较

    Fig.  3  Ablation comparison of precision, recall ratio and non-equilibrium treatment performance

    图  4  基于预测结果构建的深度学习技术网络

    Fig.  4  Deep learning technology network based on prediction results

    图  5  网络社群检测分析后的规模结果

    Fig.  5  Scale of each community after network' s community detection

    表  1  模型性能指标的计算方法

    Table  1  Calculation method of model performance measure inde

    测度指标 计算公式
    Recall $\dfrac{TP}{TP + FN}$
    Precision $\dfrac{TP}{TP + FP}$
    AUC $\sum\limits_{i=1}^{n-1} \dfrac{(FPR_{i+1} - FPR_i) \times (TPR_{i+1} + TPR_i)}{2}$
    AUPR $\sum\limits_{i=1}^{n-1} \dfrac{(R_{i+1} - R_i) \times (P_{i+1} + P_i)}{2}$
    F1 Score $2 \times \dfrac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}$
    下载: 导出CSV

    表  2  所提模型与基线模型性能比较

    Table  2  Performance comparison between the proposed model and the baseline models

    模型 特征 AUC Precision Recall F1 AUPR
    STTeFL 内容LLM表征 0.8837±0.0024 0.8361±0.0166 0.8066±0.0134 0.8206±0.0084 0.9054±0.0026
    LogR 链路相似性 0.6279±0.0141 0.7383±0.0370 0.2565±0.0282 0.3807±0.0313 0.5002±0.0265
    SVM 链路相似性 0.5113±0.0137 0.7778±0.1070 0.0227±0.0321 0.0442±0.0371 0.4039±0.0332
    RF 链路相似性 0.6915±0.0115 0.6650±0.0236 0.3844±0.0231 0.4867±0.0649 0.5270±0.0368
    XGBoost 链路相似性 0.8004±0.0149 0.4773±0.0170 0.6058±0.0301 0.5335±0.0301 0.5431±0.0135
    AdaBoost 链路相似性 0.7085±0.0267 0.6324±0.1254 0.4188±0.0406 0.5039±0.0351 0.5278±0.0483
    下载: 导出CSV

    表  3  对比模型的实验结果

    Table  3  Experimental results under different comparative models

    对比方式 模型 AUC Precision Recall F1 AUPR
    大模型 STTeFL_gpt 0.8781±0.0042 0.7031±0.0199 0.8855±0.0098 0.7835±0.0089 0.9000±0.0062
    STTeFL_bert 0.8803±0.0052 0.8163±0.0126 0.8061±0.0242 0.8135±0.0085 0.8959±0.0083
    特征表达 STTeFL_doc 0.7872±0.0606 0.6306±0.1107 0.8477±0.1253 0.7052±0.0400 0.8026±0.0630
    STTeFL_sim 0.8677±0.0076 0.6037±0.0221 0.9333±0.0093 0.7328±0.0135 0.8842±0.0105
    维数选择 STTeFL_32N 0.8700±0.0001 0.5553±0.0201 0.9595±0.0006 0.7017±0.0088 0.8870±0.0024
    STTeFL_128L 0.8708±0.0000 0.6289±0.0014 0.9255±0.0002 0.7480±0.0005 0.8882±0.0000
    下载: 导出CSV

    表  4  深度学习领域未来潜在技术举例

    Table  4  Examples of potential future technologies in the deep learning area

    序号 核心节点 潜在可能技术 涉及领域
    1A43D-001; A47B-091; H01Q-001; H04S-003天线设计与优化、立体声与多声道音频处理、塑料成型与模具5G通信、虚拟现实、汽车制造
    2G01J-005;B65D-025;G07C-015;G05G-005光学测量与传感器、航空设备与飞行控制、时间记录与考勤系统环境监测、航空航天、企业管理
    3B21D-037;G08B-007;E04H-012;F28F-013计算机硬件与软件、机械测试与测量、金属成型与加工设备监测、零部件制造、金融科技
    下载: 导出CSV

    A1  LLM 模型的参数设置

    A1  LLM model parameter settings

    模型 参数设置
    activation _ function (激活函数): gelu _ new
    n _ embd (嵌入维度): 1600
    n_layer (Transformer 层的数量): 48
    n_head (每个 Transformer 层中的自注意力头数): 25
    vocab_size (词汇表大小): 50257
    n_ctx (上下文窗口大小): 1024
    n_positions (最大位置编码数): 1024
    gpt2-xl initializer_range (初始化范围): 0.02
    layer_norm_epsilon (层归一化的 epsilon 值): 1e-5
    attn_pdrop (注意力层的丢弃率): 0.1
    embd_pdrop (嵌入层的丢弃率): 0.1
    resid_pdrop (残差连接的丢弃率): 0.1
    bos_token_id (开始标记 ID): 50256
    eos_token_id (结束标记 ID): 50256
    max_length (生成文本的最大长度): 16
    hidden_size (隐藏层大小): 2048
    num_attention_heads(每个 Transformer 层中的自注意力头数):16
    num_hidden_layers(Transformer 层的数量): 24
    vocab_size (词汇表大小): 32000
    max_position_embeddings (最大位置编码数): 4096
    intermediate_size (前馈神经网络的隐藏层大小): 5504
    Sheared-LLaMA-1.3B hidden_act (隐藏层激活函数): silu
    bos_token_id (开始标记 ID): 1
    eos_token_id (结束标记 ID): 2
    pad_token_id (填充标记 ID): 0
    pretraining_tp (预训练 TP): 1
    rms_norm_eps (RMS 层归一化的 epsilon 值): le-5
    rope_theta (位置编码的 theta 值): 10000.0
    max_length (生成文本的最大长度): 16
    activation_function (激活函数): gelu
    n_embd (嵌入维度): 768
    n_layer (Transformer 层的数量): 12
    n_head (每个 Transformer 层中的自注意力头数): 12
    vocab_size (词汇表大小): 30522
    n_ctx (上下文窗口大小): 512
    bert-base-uncased n_positions (最大位置编码数): 512
    layer_norm_epsilon (层归一化的 epsilon 值): 1e-12
    attn_pdrop (注意力层的丢弃率): 0.1
    hidden_dropout_prob (隐藏层丢弃率): 0.1
    bos_token_id (开始标记 ID): 101
    eos_token_id (结束标记 ID): 102
    pad_token_id (填充标记 ID): 0
    position_embedding_type (位置编码类型): absolute
    Linear Layer:
    输入维度: 768
    输出维度: 16
    下载: 导出CSV
  • [1] Zhao J Y, Dong, Z J, Yao X L, et al. Optimizing collaboration decisions in technological innovation through machine learning: identify trend and partners in collaboration-knowledge interdependent networks. Annals of Operations Research, 2024, 1-42
    [2] 连芷萱, 王芳, 康佳, 等. 基于图神经网络和粒子群算法的技术预测模型. 情报学报, 2023, 42(4): 420−435

    Lian Z X, Wang F, Kang J et al. Graph Neural Network-based and Particle Swarm Optimization Technological Prediction Model (in Chinese). Journal of the China Society for Scientific and Technical Information, 2023, 42(4): 420−435
    [3] Xi X, Ren F F, Yu L A, Yang J. Detecting the technology's evolutionary pathway using HiDS-trait-driven tech mining strategy. Technological Forecasting & Social Change, 2023, 195: 122777
    [4] 潘文雯, 赵洲, 俞俊, 吴飞. 基于文本引导的注意力图像转发预测排序网络. 自动化学报, 2021, 47(11): 2547−2556

    Pan W W, Zhao Z, Yu J, Wu F. Textually Guided Ranking Network for Attentional Image Retweet Modeling. Automatica Sinica, 2021, 47(11): 2547−2556
    [5] Daud N N, Ab Hamid S H, Saadoon M et al. Applications of link prediction in social networks: A review. Journal of Network and Computer Applications, 2020, 166: 102716 doi: 10.1016/j.jnca.2020.102716
    [6] Lee W S, Han E J, Sohn S Y. Predicting the pattern of technology convergence using big-data technology on large-scale triadic patents. Technological Forecasting and Social change, 2015, 22(7): 317−329
    [7] 张斌, 李亚婷. 学科合作网络链路预测结果的排序鲁棒性. 信息资源管理学报, 2018, 8(04): 89−97

    Zhang B, Li Y T. Ranking robustness of link prediction results in disciplinary collaboration network (in Chinese). Journal of Information Resources Management, 2018, 8(04): 89−97
    [8] Xi X, Zhao J Y, Yu L A et al. Exploring the Potentials of Artificial Intelligence Towards Carbon Neutrality: Technological Convergence Forecasting Through Link Prediction and Community Detection. Computers & Industrial Engineering, 2024, 190: 110015
    [9] Sasaki H, Sakata I. Identifying potential technological spin-offs using hierarchical information in international patent classification. Technovation, 2021, 100: 102192 doi: 10.1016/j.technovation.2020.102192
    [10] 贾承丰, 韩华, 吕亚楠, 张路. 基于 Word2vec 和粒子群的链路预测算法. 自动化学报, 2020, 46(8): 1703−1713

    Jia C F, Han H, Lv Y N, Zhang L. Link Prediction Algorithm Based on Word2vec and Particle Swarm. Acta Automatica Sinica, 2020, 46(8): 1703−1713
    [11] 王守辉, 于洪涛, 黄瑞阳, 马青青. 基于模体演化的时序链路预测方法. 自动化学报, 2016, 42(5): 735−745

    Wang S H, Yu H T, Huang R Y, Ma Q Q. A Temporal Link Prediction Method Based on Motif Evolution. Automatica Sinica, 2016, 42(5): 735−745
    [12] Kwon O, An Y, Kim M, Lee et al. Anticipating technology-driven industry convergence: evidence from large-scale patent analysis. Technology Analysis & Strategic Management, 2020, 32(4): 363−378
    [13] Kipf T N, Welling M. Semi-Supervised Classification with Graph Convolutional Networks, In: Proceedings of the 5th International Conference on Learning Representations, 2017
    [14] Zhou L X, Schellaert W, Martinez-Plumed F et al. Larger and more instructable language models become less reliable. Nature, 2024, 634: 61−68 doi: 10.1038/s41586-024-07930-y
    [15] 宁传峰. 随机配置网络不平衡数据分类算法研究. 中国矿业大学[D], 2023.
    [16] Lee C, Hong S, Kim J. Anticipating multi-technology convergence: a machine learning approach using patent information. Scientometrics, 2021, 126: 1867−1896 doi: 10.1007/s11192-020-03842-6
    [17] Wang J, Lee J J. Predicting and Analyzing Technology Convergence for Exploring Technological Opportunities in the Smart Health Industry. Computers & Industrial Engineering, 2023109352
    [18] Hong S C, Lee C Y. Effective Indexes and Classification Algorithms for Supervised Link Prediction Approach to Anticipating Technology Convergence: A Comparative Study. IEEE transactions on engineering, 2023, 70(4): 1430 doi: 10.1109/TEM.2021.3098602
    [19] Grover A, Leskovec J. Node2vec: scalable feature learning for networks. KDD '16: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016855−864
    [20] Dastile, X., Celik, T., & Potsane, M. Statistical and machine learning models in credit scoring: A systematic literature survey. Applied Soft Computing, 2000, 91: 106263
    [21] Kim T S, Sohn S Y. Machine-learning-based deep semantic analysis approach for forecasting new technology convergence. Technological Forecasting & Social Change, 2020, 157: 120095
    [22] Zhao H, Zhao C, Zhang X et al. An ensemble learning approach with gradient resampling for class-imbalance problems. INFORMS Journal on Computing, 2023, 35(4): 747−763 doi: 10.1287/ijoc.2023.1274
    [23] Kumar A, Singh S S, Singh K et al. Link prediction techniques, applications, and performance: A survey. Physica A: Statistical Mechanics and its Applications, 2020, 553: 124289 doi: 10.1016/j.physa.2020.124289
    [24] Kim K S, Cho N W. Predicting the patterns of technology convergence in defense technologies. In 2022 IEEE International Conference on Big Data and Smart Computing (BigComp). 2022: 72-75
  • 加载中
计量
  • 文章访问数:  21
  • HTML全文浏览量:  23
  • 被引次数: 0
出版历程
  • 收稿日期:  2025-03-11
  • 录用日期:  2025-07-13
  • 网络出版日期:  2025-09-22

目录

    /

    返回文章
    返回