Argument Pair Extraction Method Based on Bidirectional Multi-Perspective Relational Graph Convolutional Network
-
摘要: 论辩对抽取是论辩挖掘领域中的一项重要研究任务, 旨在从对话文档的两个段落中抽取互动论辩对. 现有研究通常将其分为序列标记和关系分类两个子任务, 通过预测段落间的句子级关系来抽取论辩对. 然而, 这些研究在整体论点级语义及句子内部细粒度语义逻辑信息的显式建模上仍存在不足, 且未充分考虑两个段落间复杂的上下文感知交互关系. 基于此, 提出一种双向多视角关系图卷积网络. 首先, 从段落内、依存语法和段落间视角分别构建论点关系图, 利用图结构表示文本的逻辑结构和语义交互关系, 为模型提供丰富的上下文语义信息. 然后, 通过引入多视角关系图卷积和图匹配模块, 在两个段落之间进行双向交互, 充分利用不同层次的论点间互动关系, 增强模型对跨段落论点间语义联系的捕捉能力和论点关系的识别精度. 实验结果表明, 相较于基线模型, 该方法在性能上有了显著提升.Abstract: Argument pair extraction, which aims to extract interactive argument pairs from two paragraphs of dialogical documents, is an important research task in argument mining field. Existing research decomposes this task into sequence tagging and relation classification, extracting argument pairs by predicting sentence-level relationships between paragraphs. However, these studies lack in explicit modeling of overall argument-level semantics and fine-grained semantic logical information within sentences, and failure to adequately account for the complex context-aware interaction relationship between two paragraphs. Based on this we propose a bidirectional multi-perspective relation graph convolutional network. Firstly, we construct argument relation graphs from intra-paragraph, dependency syntactic, and inter-paragraph perspectives, respectively, leveraging graph structures to better represent the logical structure and semantic interaction relationships within the text, to enrich the model with rich contextual semantic information. Then, by introducing multi-perspective relation graph convolutional module and graph matching module, achieve bidirectional interactions between two paragraphs, fully utilizing different levels of interaction between arguments to enhance the model's ability to capture semantic connections across paragraph-level arguments and improve the accuracy of argument relation recognition. Experimental results show that the method proposed in this paper significantly improves performance compared to baseline models.1)
1 1https://www.hanlp.com/ 2https://github.com/tensorflow/models/tree/master/syntaxnet 2)2 3https://github.com/fxsjy/jieba 4https://github.com/chatopera/Synonyms 3)3 5https://github.com/LiyingCheng95/ArgumentPairExtraction/tree/master/data 6http://cail.cipsc.org.cn/task_summit.html?raceID=5&cail_tag=2023 4)4 7https://huggingface.co/sentence-transformers/all-mpnet-base-v2 8https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 -
图 7 不同MRGC层数L对模型效果的影响
Fig. 7 Impact of different MRGC layer numbers L on model performance
图 9 不同GCN层数$ l$对模型效果的影响
Fig. 9 Impact of different GCN layer numbers $ l$ on model performance
图 10 RR-S数据集上不同$ \varphi$和$ \gamma$对模型效果的影响
Fig. 10 Impact of different $ \varphi$ and $ \gamma$ on model performance on the RR-S dataset
表 1 评估数据集的统计信息, 其中SPA表示每个论点的句子数量
Table 1 Evaluation dataset statistics, where SPA indicates the number of sentences of per argument
数据集 RR-S RR-P Cail 训练集中论辩对的数量 3817 3811 2740 开发集中论辩对的数量 473 477 343 测试集中论辩对的数量 474 476 342 句子总数 217.8 K 190.5 K 8094 $D_a$中包含的论点数 23.2 K 23.0 K 2833 $D_b$中包含的论点数 17.7 K 17.5 K 2464 $D_a$中的SPA均值 2.84 2.53 1.21 $D_b$中的SPA均值 4.16 3.82 1.39 SPA的均值 3.41 3.09 1.29 
下载: 导出CSV
表 2 实验结果
Table 2 Experimental results
数据集 方法 AM APE Pre Rec F1 Pre Rec F1 RR-S PL-H-LSTM-CRF 67.02 68.49 67.75 19.74 19.13 19.43 MT-H-LSTM-CRF 70.74 69.46 70.09 27.24 26.00 26.61 MLMC 69.53 73.27 71.35 37.15 29.38 32.81 MGF 70.82 73.19 71.99 40.45 30.77 34.95 MCRN 70.52 69.53 70.02 39.62 32.03 35.42 MRC-APE 71.83 73.05 72.43 41.83 38.17 39.92 HGMN 73.16 74.07 73.61 44.42 39.19 41.64 MMR-GCN 71.68 70.67 71.16 44.69 38.44 41.31 PIEGON 72.29 73.22 72.75 41.06 44.84 42.86 SADGCN 73.18 72.88 73.03 45.67 47.32 46.48 BMRGCN (ours) 74.25 74.56 74.40 44.31 44.82 44.56 RR-P PL-H-LSTM-CRF 73.10 67.65 70.27 21.24 19.30 20.22 MT-H-LSTM-CRF 71.85 71.01 71.43 30.08 29.55 29.81 MLMC 66.79 72.17 69.38 40.27 29.53 34.07 MGF 73.62 70.88 72.22 38.03 35.68 36.82 MRC-APE 76.39 70.62 73.39 37.70 44.00 40.61 MCRN 69.27 71.39 70.32 38.20 32.37 35.04 HGMN 74.15 76.14 75.13 44.95 40.69 42.71 MMR-GCN 71.47 71.60 71.54 45.52 39.70 42.41 PIEGON 73.30 73.56 73.43 43.18 44.16 43.66 SADGCN 73.31 73.69 73.50 43.25 47.53 45.29 BMRGCN (ours) 75.95 76.32 76.13 45.16 45.48 45.32 Cail Bert-CNN-CRF 67.15 66.23 66.74 28.13 27.86 27.98 Bert-BiGRU-CRF 68.90 69.04 69.00 29.93 30.18 30.02 Bert-BiLSTM-CRF 69.56 68.62 69.08 30.08 30.25 30.16 MLMC 71.92 77.12 74.43 49.87 38.24 43.29 MGF 79.58 75.96 77.73 47.86 44.72 46.24 MRC-APE 82.64 76.45 79.42 46.89 53.86 50.13 BMRGCN (ours) 82.61 82.84 82.72 55.27 55.48 55.37
下载: 导出CSV
表 3 大语言模型上的实验结果, 其中T1表示用模板1进行提示、T2表示用模板2进行提示
Table 3 Experimental results of LLMs, where T1 refers to the prompt using Template 1 and T2 refers to the prompt using Template 2
数据集 方法 AM APE Pre Rec F1 Pre Rec F1 RR-S GPT-3 (concrete)$^*$ — — 39.86 — — 18.58 GPT-4 (concrete)$^*$ — — 64.51 — — 53.84 GPT-3 (symbolic)$^*$ — — 62.00 — — 20.15 GPT-4 (symbolic)$^*$ — — 70.63 — — 49.85 GPT-4o (T1) 70.41 70.69 70.54 40.61 40.74 40.67 ERNIE-4.0-8K (T1) 64.79 64.94 64.87 34.89 35.07 34.98 GPT-4o (T2) 72.26 72.37 72.31 47.76 47.87 47.82 ERNIE-4.0-8K (T2) 67.48 67.57 67.53 43.46 43.58 43.52 Llama3-8B (T2) 41.77 41.89 41.83 19.30 19.40 19.35 BMRGCN (ours) 74.25 74.56 74.40 44.31 44.82 44.56 RR-P GPT-4o (T1) 71.21 71.36 71.29 41.68 41.82 41.75 ERNIE-4.0-8K (T1) 66.08 66.21 66.14 35.91 36.01 35.96 GPT-4o (T2) 73.22 73.31 73.27 48.16 48.25 48.21 ERNIE-4.0-8K (T2) 68.90 69.15 69.03 44.28 44.39 44.34 Llama3-8B (T2) 41.97 42.08 42.03 21.06 21.21 21.13 BMRGCN (ours) 75.95 76.32 76.13 45.16 45.48 45.32 Cail GPT-4o (T1) 78.79 78.88 78.84 50.72 50.80 50.76 ERNIE-4.0-8K (T1) 70.98 71.21 71.09 42.75 42.89 42.82 GPT-4o (T2) 80.88 80.97 80.93 56.81 56.93 56.87 ERNIE-4.0-8K (T2) 74.78 74.92 74.85 51.22 51.31 51.27 Llama3-8B (T2) 46.47 46.55 46.51 25.24 25.39 25.31 BMRGCN (ours) 82.61 82.84 82.72 55.27 55.48 55.37
下载: 导出CSV
表 4 不同损失权重设置对模型效果的影响
Table 4 Impact of different loss weight settings on model performance
权重 F1 $\ell_{\mathrm{start}}$ $\ell_{\mathrm{end}}$ $\ell_{\mathrm{pair}}$ $\ell_{reg}$ AM APE 0.15 0.15 0.60 0.10 71.23 42.16 0.20 0.20 0.50 0.10 72.59 42.57 0.25 0.25 0.40 0.10 73.15 43.04 0.30 0.30 0.30 0.10 73.62 43.76 0.35 0.35 0.25 0.05 74.40 44.56 0.40 0.40 0.15 0.05 73.79 43.89
下载: 导出CSV
表 5 消融实验结果(表中两个Δ(F1)分别表示在AM和APE任务上F1的变化值)
Table 5 The results of ablation experiment (the two Δ(F1) values represent F1-score changes for the AM and APE tasks, respectively)
数据集 方法 AM $\Delta($F1) APE $\Delta($F1) RR-S BMRGCN 74.40 — 44.56 — w/o $G^{Intra}$ 69.63 −4.77 39.82 −4.74 w/o $G^{Dep}$ 70.18 −4.22 40.61 −3.95 w/o $G^{Inter}$ 71.31 −3.09 38.24 −6.32 w/o GM 72.41 −1.99 42.44 −2.12 w/o CA 72.23 −2.17 42.52 −2.04 w/o DR 73.09 −1.31 43.09 −1.47 RR-P BMRGCN 76.13 — 45.32 — w/o $G^{Intra}$ 71.05 −5.08 40.07 −5.25 w/o $G^{Dep}$ 71.30 −4.83 40.86 −4.46 w/o $G^{Inter}$ 72.49 −3.64 38.71 −6.61 w/o GM 73.94 −2.19 42.89 −2.43 w/o CA 73.75 −2.38 42.95 −2.37 w/o DR 74.68 −1.45 43.56 −1.76 Cail BMRGCN 82.72 — 55.37 — w/o $G^{Intra}$ 76.83 −5.89 48.41 −6.96 w/o $G^{Dep}$ 77.22 −5.50 49.10 −6.27 w/o $G^{Inter}$ 77.97 −4.75 47.23 −8.14 w/o GM 79.54 −3.18 52.14 −3.23 w/o CA 79.51 −3.21 52.32 −3.05 w/o DR 80.08 −2.64 52.56 −2.81
下载: 导出CSV
表 6 不同相似度计算方式下的实验结果
Table 6 The experimental results under different similarity calculation methods
数据集 相似度计算方式 AM APE Pre Rec F1 Pre Rec F1 RR-S Sentence-Bert 74.25 74.56 74.40 44.31 44.82 44.56 Cos 73.38 73.95 73.66 43.79 43.85 43.82 RR-P Sentence-Bert 75.95 76.32 76.13 45.16 45.48 45.32 Cos 75.42 75.68 75.55 44.85 45.01 44.94 Cail Sentence-Bert 82.61 82.84 82.72 55.27 55.48 55.37 Cos 81.83 81.92 81.89 54.02 54.11 54.03
下载: 导出CSV
表 7 不同$ G^{Inter} $构建方式下的实验结果
Table 7 Experimental results under different $ G^{Inter} $ construction methods
数据集 $ G^{Inter} $构建方式 AM APE Pre Rec F1 Pre Rec F1 RR-S $ G_{ab}^{Inter}+G_{ba}^{Inter} $ 74.73 74.94 74.83 45.41 45.61 45.51 $ G^{Inter} $ 72.82 73.10 72.96 43.65 43.79 43.72 RR-P $ G_{ab}^{Inter}+G_{ba}^{Inter} $ 76.62 76.85 76.73 46.56 46.75 46.66 $G^{Inter}$ 74.47 74.60 74.53 44.79 44.91 44.85 Cail $G^{Inter}_{ab} + G^{Inter}_{ba}$ 82.61 82.84 82.72 55.27 55.48 55.37 $G^{Inter}$ 79.80 79.91 79.85 52.18 52.29 52.24
下载: 导出CSV
表 8 不同依存语法工具下的实验结果
Table 8 Experimental results under different dependency grammer tools
数据集 依存语法工具 AM APE Pre Rec F1 Pre Rec F1 RR-S Spacy 74.25 74.56 74.40 44.31 44.82 44.56 CoreNLP 73.91 74.25 74.08 44.16 44.34 44.25 SyntaxNet 74.73 74.94 74.83 45.41 45.61 45.51 RR-P Spacy 75.95 76.32 76.13 45.16 45.48 45.32 CoreNLP 76.08 76.26 76.17 45.09 45.25 45.17 SyntaxNet 76.62 76.85 76.73 46.56 46.75 46.66 Cail DDParser 82.29 82.40 82.34 54.79 54.90 54.84 LTP 82.61 82.61 82.64 54.98 55.19 55.09 hanlp 82.61 82.84 82.72 55.27 55.48 55.37
下载: 导出CSV
-
[1] Cheng L Y, Bing L D, Yu Q, Lu W, Si L. APE: Argument pair extraction from peer review and rebuttal via multi-task learning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Virtual Event: ACL, 2020. 7000−7011 [2] Bao J Z, Sun J Y, Zhu Q L, Xu R F. Have my arguments been replied to? Argument pair extraction as machine reading comprehension. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Dublin, Ireland: ACL, 2022. 29−35 [3] Cheng L Y, Wu T Y, Bing L D, Si L. Argument pair extraction via attention-guided multi-layer multi-cross encoding. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Virtual Event: ACL, 2021. 6341−6353 [4] Sun J Y, Zhu Q L, Bao J Z, Wu J P, Yang C H, Wang R, et al. A hierarchical sequence labeling model for argument pair extraction. In: Proceedings of the 10th CCF International Conference on Natural Language Processing and Chinese Computing. Qingdao, China: Springer, 2021. 472−483 [5] He N X, Chen Q F, Yu Q, Han Z Z. Multi-fusion recurrent network for argument pair extraction. In: Proceedings of the 32nd International Conference on Artificial Neural Networks and Machine Learning-ICANN 2023. Heraklion, Greece: Springer, 2023. 103−114 [6] Bao J Z, Liang B, Sun J Y, Zhang Y C, Yang M, Xu R F. Argument pair extraction with mutual guidance and inter-sentence relation graph. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Punta Cana, Dominican Republic: ACL, 2021. 3923−3934 [7] Zhu X F, Liu Y D, Chen Z, Chen X, Guo J F, Dietze S. A mutually enhanced multi-scale relation-aware graph convolutional network for argument pair extraction. Journal of Intelligent Information Systems, 2024, 62(2): 555−574 [8] Mao T Z, Yoshie O, Fu J L, Mao W X. Seeing both sides: Context-aware heterogeneous graph matching networks for extracting-related arguments. Neural Computing and Applications, 2024, 36(9): 4741−4762 doi: 10.1007/s00521-023-09250-0 [9] Sun Y, Liang B, Bao J Z, Zhang Y C, Tu G, Yang M, et al. Probing graph decomposition for argument pair extraction. In: Proceedings of the Findings of the Association for Computational Linguistics: ACL 2023. Toronto, Canada: ACL, 2023. 13075−13088 [10] Jo Y, Visser J, Reed C, Hovy E H. A cascade model for proposition extraction in argumentation. In: Proceedings of the 6th Workshop on Argument Mining. Florence, Italy: ACL, 2019. 11−24 [11] Ein-Dor L, Shnarch E, Dankin L, Halfon A, Sznajder B, Gera A, et al. Corpus wide argument mining——A working solution. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York, USA: AAAI, 2020. 7683−7691 [12] Gleize M, Shnarch E, Choshen L, Dankin L, Moshkowich G, Aharonov R, et al. Are you convinced? Choosing the more convincing evidence with a Siamese network. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: ACL, 2019. 967−976 [13] Toledo A, Gretz S, Cohen-Karlik E, Friedman R, Venezian E, Lahav D, et al. Automatic argument quality assessment-new datasets and methods. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China: ACL, 2019. 5625−5635 [14] Cocarascu O, Cabrio E, Villata S, Toni F. Dataset independent baselines for relation prediction in argument mining. In: Proceedings of the Computational Models of Argument-Proceedings of COMMA 2020. Perugia, Italy: IOS Press, 2020. 45−52 [15] Jo Y, Bang S, Reed C, Hovy E. Classifying argumentative relations using logical mechanisms and argumentation schemes. Transactions of the Association for Computational Linguistics, 2021, 9: 721−739 [16] Hua X Y, Hu Z, Wang L. Argument generation with retrieval, planning, and realization. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: ACL, 2019. 2661−2672 [17] Schiller B, Daxenberger J, Gurevych I. Aspect-controlled neural argument generation. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Virtual Event: ACL, 2021. 380−396 [18] Morio G, Ozaki H, Morishita T, Koreeda Y, Yanai K. Towards better non-tree argument mining: Proposition-level biaffine parsing with task-specific parameterization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Virtual Event: ACL, 2020. 3259−3266 [19] Bao J Z, Fan C, Wu J P, Dang Y X, Du J C, Xu R F. A neural transition-based model for argumentation mining. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Virtual Event: ACL, 2021. 6354−6364 [20] Ji L, Wei Z Y, Li J, Zhang Q, Huang X J. Discrete argument representation learning for interactive argument pair identification. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Virtual Event: ACL, 2021. 5467−5478 [21] Tan C H, Niculae V, Danescu-Niculescu-Mizil C, Lee L. Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In: Proceedings of the 25th International Conference on World Wide Web. Montréal, Canada: International World Wide Web Conferences Steering Committee, 2016. 613−624 [22] Guan M Z, Qiu Z X, Li F H, Xue Y. Semantics-aware dual graph convolutional networks for argument pair extraction. In: Proceedings of the Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). Torino, Italia: ACL, 2024. 14652−14663 [23] Zhou Y X, Liao L J, Gao Y, Jie Z M, Lu W. To be closer: Learning to link up aspects with opinions. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Punta Cana, Dominican Republic: ACL, 2021. 3899−3909 [24] Li X Y, Feng J R, Meng Y X, Han Q H, Wu F, Li J W. A unified MRC framework for named entity recognition. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Virtual Event: ACL, 2020. 5849−5859 [25] de Wynter A, Yuan T M. “ I'd like to have an argument, please”: Argumentative reasoning in large language models. In: Proceedings of the Computational Models of Argument-Proceedings of COMMA 2024. Hagen, Germany: IOS Press, 2024. 73−84 -
图(10) / 表(8)
计量
- 文章访问数: 35
- HTML全文浏览量: 18
- PDF下载量: 11
- 被引次数: 0
