面向小样本意图识别的分步式阶段性数据增强
作者:
基金项目:

陕西省自然科学基金(2019JQ-849)


Stepwise and Phased Data Augmentation for Few-shot Intent Detection
Author:
  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [20]
  • |
  • 相似文献 [20]
  • | | |
  • 文章评论
    摘要:

    文本意图识别任务中常面临训练数据不足的问题, 且由于文本数据离散性导致在标签不变的条件下进行数据增强并提高原模型性能具有一定困难, 为解决小样本意图识别任务中的上述问题, 提出一种分步式数据增强与阶段性训练策略相结合的方法. 该方法从全局和局部两个角度将原始数据在全体语句和同类别中的样本对上进行递进式增强, 并在模型训练期间根据递进层次的不同划分阶段进行学习, 最后在多个意图识别数据集上进行实验以评估其有效性. 实验结果表明, 该方法可以有效提高小样本环境中意图识别模型的准确率, 同时模型的稳定性也得到了提升.

    Abstract:

    Insufficient training data is often faced in the task of text intent detection, and due to the discreteness of text data, it is difficult to perform data augmentation and improve the performance of the original model with the unchanged label. This study proposes a method combining stepwise data augmentation with a phased training strategy to solve the above problems in the few-shot intent detection. The method progressively augments the original data on whole statements and sample pairs in the same category from both global and local perspectives. During model training, the original data is learned according to different partition stages of the progressive level. Finally, experiments are performed on multiple intent detection datasets to evaluate the validity of the method. The experimental results show that the proposed method can effectively improve the accuracy and the stability of the few-shot intent detection model.

    参考文献
    [1] Hou YT, Lai YK, Wu YS, et al. Few-shot learning for multi-label intent detection. Proceedings of the 35th AAAI Conference on Artificial Intelligence. Online:AAAI Press, 2021. 13036-13044.
    [2] Dopierre T, Gravier C, Logerais W. Protaugment:Unsupervised diverse short-texts paraphrasing for intent detection meta-learning. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Stroudsburg:Association for Computational Linguistics, 2021. 2454-2466.
    [3] 王新哲, 于泽沛, 时斌, 等. 基于元学习的小样本数据生成算法. 计算机系统应用, 2021, 30(9):161-170.[doi:10.15888/j.cnki.csa.008063
    [4] Malandrakis N, Shen MM, Goyal A, et al. Controlled text generation for data augmentation in intelligent artificial agents. Proceedings of the 3rd Workshop on Neural Generation and Translation. Hong Kong:Association for Computational Linguistics, 2019. 90-98.
    [5] Goodfellow IJ, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets. Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal:MIT Press, 2014. 2672-2680.
    [6] Madry A, Makelov A, Schmidt L, et al. Towards deep learning models resistant to adversarial attacks. Proceedings of the 6th International Conference on Learning Representations. Vancouver:OpenReview.net, 2018. 1-28.
    [7] Shafahi A, Najibi M, Ghiasi A, et al. Adversarial training for free! Proceedings of the 33rd Conference on Neural Information Processing Systems. Vancouver:Curran Associates Inc., 2019. 3358-3369.
    [8] Zhu C, Cheng Y, Gan Z, et al. FreeLB:Enhanced adversarial training for natural language understanding. Proceedings of the 8th International Conference on Learning Representations. Addis Ababa:OpenReview.net, 2020. 1-14.
    [9] Zhang RY, Chen CY, Gan Z, et al. Improving adversarial text generation by modeling the distant future. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg:Association for Computational Linguistics, 2020. 2516-2531.
    [10] Wu X, Lv SW, Zang LJ, et al. Conditional BERT contextual augmentation. Proceedings of the 19th International Conference on Computational Science. Faro:Springer, 2019. 84-95.
    [11] Devlin J, Chang MW, Lee K, et al. BERT:Pre-training of deep bidirectional transformers for language understanding. Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Minneapolis:Association for Computational Linguistics, 2019. 4171-4186.
    [12] Anaby-Tavor A, Carmeli B, Goldbraich E, et al. Do not have enough data? Deep learning to the rescue! Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York:AAAI Press, 2020. 7383-7390.
    [13] Wei J, Zou K. EDA:Easy data augmentation techniques for boosting performance on text classification tasks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Hong Kong:Association for Computational Linguistics, 2019. 6382-6388.
    [14] Xie QZ, Dai ZH, Hovy E, et al. Unsupervised data augmentation for consistency training. Proceedings of the 34th Conference on Neural Information Processing Systems. Vancouver:Curran Associates Inc., 2020. 6256-6268.
    [15] Haralabopoulos G, Torres MT, Anagnostopoulos I, et al. Text data augmentations:Permutation, antonyms and negation. Expert Systems with Applications, 2021, 177:114769.[doi:10.1016/j.eswa.2021.114769
    [16] DeVries T, Taylor GW. Dataset augmentation in feature space. Proceedings of the 5th International Conference on Learning Representations. Toulon:OpenReview.net, 2017. 1-12.
    [17] Zhang HY, Cissé M, Dauphin YN, et al. Mixup:Beyond empirical risk minimization. Proceedings of the 6th International Conference on Learning Representations. Vancouver:OpenReview.net, 2018. 1-13.
    [18] Xu BF, Zhang LC, Mao ZD, et al. Curriculum learning for natural language understanding. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg:Association for Computational Linguistics, 2020. 6095-6104.
    [19] Wei J, Huang CY, Vosoughi S, et al. Few-shot text classification with triplet networks, data augmentation, and curriculum learning. Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg:Association for Computational Linguistics, 2021. 5493-5500.
    [20] Wang XY, Pham H, Dai ZH, et al. SwitchOut:An efficient data augmentation algorithm for neural machine translation. Proceedings of 2018 Conference on Empirical Methods in Natural Language Processing. Brussels:Association for Computational Linguistics, 2018. 856-861.
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

李玉茹,张晓滨.面向小样本意图识别的分步式阶段性数据增强.计算机系统应用,2023,32(1):406-412

复制
分享
文章指标
  • 点击次数:767
  • 下载次数: 1871
  • HTML阅读次数: 1444
  • 引用次数: 0
历史
  • 收稿日期:2022-05-15
  • 最后修改日期:2022-06-15
  • 在线发布日期: 2022-08-26
文章二维码
您是第11418545位访问者
版权所有:中国科学院软件研究所 京ICP备05046678号-3
地址:北京海淀区中关村南四街4号 中科院软件园区 7号楼305房间,邮政编码:100190
电话:010-62661041 传真: Email:csa (a) iscas.ac.cn
技术支持:北京勤云科技发展有限公司

京公网安备 11040202500063号