###
计算机系统应用英文版:2024,33(4):246-253
本文二维码信息
码上扫一扫!
基于指针生成网络和扩展Transformer的多属性可控文本摘要模型
(华南师范大学 软件学院, 佛山 528225)
Multi-attribute Controllable Text Summary Model Based on Pointer Generator Network and Extended Transformer
(School of Software, South China Normal University, Foshan 528225, China)
摘要
图/表
参考文献
相似文献
本文已被:浏览 70次   下载 274
Received:September 28, 2023    Revised:November 09, 2023
中文摘要: 可控文本摘要模型可以生成符合用户偏好的摘要. 之前的摘要模型侧重于单独控制某个属性, 而不是多个属性的组合. 传统的Seq2Seq多属性可控文本摘要模型在满足多个控制属性时, 存在无法整合所有控制属性、无法准确再现文本中关键信息和无法处理单词表外单词等问题. 为此, 本文提出了一种基于扩展Transformer和指针生成网络(pointer generator network, PGN)的模型. 模型中的扩展Transformer将Transformer单编码器-单解码器的模型形式扩展成具有双重文本语义信息提取的双编码器和单个可融合指导信号特征的解码器形式. 然后利用指针生成网络模型选择从源文本中复制单词或利用词汇表生成新的摘要信息, 以解决摘要任务中常出现的OOV (out of vocabulary)问题. 此外, 为高效完成位置信息编码, 模型在注意力层中使用相对位置表示来引入文本的序列信息. 模型可以用于控制摘要的许多重要属性, 包括长度、主题和具体性等. 通过在公开数据集MACSum上的实验表明, 相较以往方法, 本文提出的模型在确保摘要质量的同时, 更加符合用户给定的属性要求.
Abstract:The controllable text summary models can generate summaries that conform to user preferences. Previous summary models focus on controlling a certain attribute alone, rather than the combination of multiple attributes. When multiple control attributes are satisfied, the traditional Seq2Seq multi-attribute controllable text summary model cannot integrate all control attributes, accurately reproduce key information in the texts, and handle words outside the word lists. Therefore, this study proposes a model based on the extended Transformer and pointer generator network (PGN). The extended Transformer in the model extends the Transformer single encoder-single decoder model form into a dual encoder with dual text semantic information extraction and a single decoder form that can fuse guidance signal features. Then the PGN model is employed to select the source from the source copy words in the text or adopt vocabulary to generate new summary information to solve the OOV (out of vocabulary) problem that often occurs in summary tasks. Additionally, to efficiently complete position information encoding, the model utilizes relative position representation in the attention layer to introduce sequence information of the texts. The model can be leveraged to control many important summary attributes, including lengths, topics, and specificity. Experiments on the public dataset MACSum show that compared with previous methods, the proposed model performs better at ensuring the summary quality. At the same time, it is more in line with the attribute requirements given by users.
文章编号:     中图分类号:    文献标志码:
基金项目:
引用文本:
冼广铭,李凡龙,郑兆明.基于指针生成网络和扩展Transformer的多属性可控文本摘要模型.计算机系统应用,2024,33(4):246-253
XIAN Guang-Ming,LI Fan-Long,ZHENG Zhao-Ming.Multi-attribute Controllable Text Summary Model Based on Pointer Generator Network and Extended Transformer.COMPUTER SYSTEMS APPLICATIONS,2024,33(4):246-253