本文已被:浏览 97次 下载 1042次
Received:February 21, 2024 Revised:March 19, 2024
Received:February 21, 2024 Revised:March 19, 2024
中文摘要: 目前, 在多模态情感分析任务上, 存在着单一模态特征提取不充分、数据融合方法缺乏稳定性的问题. 本文提出一种利用插值优化模态特征的方法, 用于解决这些问题. 首先利用插值优化BERT和GRU模型提取特征的方式, 并使用这两种模型挖掘文本、音频、视频的信息. 其次, 用改进的注意力机制融合文本、音频和视频信息, 从而更稳定地实现模态融合. 该方法在MOSI和MOSEI数据集上进行实验. 实验结果表明, 使用插值能够在优化模态特征的基础上, 提高对多模态情感分析任务的准确率, 该结果验证了插值的有效性.
Abstract:Currently, in multimodal sentiment analysis tasks, there are problems such as insufficient single modal feature extraction and lack of stability in data fusion methods. This study proposes a method of optimizing modal features that uses interpolation to solve these problems. Firstly, the interpolation-optimized BERT and GRU models are applied to extract features, and both of the models are used to mine text, audio, and video information. Secondly, an improved attention mechanism is used to fuse text, audio, and video information, thus achieving modal fusion more stably. This method is tested on the MOSI and MOSEI datasets. The experimental results show that using interpolation can improve the accuracy of multi-modal sentiment analysis tasks based on optimizing modal features. This result verifies the effectiveness of interpolation.
文章编号: 中图分类号: 文献标志码:
基金项目:国家自然科学基金重点项目(62237001); 广东省哲学社会科学青年项目(GD23YJY08)
引用文本:
唐业凯,冯广,杨芳捷,林浩泽.利用插值优化特征的多模态情感分析.计算机系统应用,2024,33(10):255-262
TANG Ye-Kai,FENG Guang,YANG Fang-Jie,LIN Hao-Ze.Multimodal Sentiment Analysis Using Interpolation Optimization Features.COMPUTER SYSTEMS APPLICATIONS,2024,33(10):255-262
唐业凯,冯广,杨芳捷,林浩泽.利用插值优化特征的多模态情感分析.计算机系统应用,2024,33(10):255-262
TANG Ye-Kai,FENG Guang,YANG Fang-Jie,LIN Hao-Ze.Multimodal Sentiment Analysis Using Interpolation Optimization Features.COMPUTER SYSTEMS APPLICATIONS,2024,33(10):255-262