###
计算机系统应用英文版:2024,33(10):66-74
本文二维码信息
码上扫一扫!
双向融合CNN与Transformer的三维视线估计
(西安工业大学 计算机科学与工程学院, 西安 710021)
3D Gaze Estimation by Bidirectional Fusion of CNN and Transformer
(School of Computer Science and Engineering, Xi’an Technological University, Xi’an 710021, China)
摘要
图/表
参考文献
相似文献
本文已被:浏览 169次   下载 920
Received:March 18, 2024    Revised:April 16, 2024
中文摘要: 针对当前视线估计任务在无约束环境中易受影响因素干扰, 准确度不高的问题, 提出一种卷积与注意力双分支并行的特征交叉融合视线估计方法, 提升了特征融合的有效性和网络性能. 首先, 对Mobile-Former网络进行改进, 引入了线性注意力机制和部分卷积, 有效提高了特征提取能力并且降低了计算成本; 其次, 增加了基于300W-LP数据集预训练的ResNet50头部姿态特征估计网络分支来增强视线估计的准确度, 并使用Sigmoid函数作为门控单元来筛选有效特征; 最后, 将面部图像输入神经网络进行特征提取和融合, 输出三维视线估计方向. 在MPIIFace-Gaze和Gaze360数据集上评估模型, 该方法的视线平均角度误差为3.70°和10.82°, 通过与其他主流三维视线估计方法比较, 验证了该网络模型能够比较准确的估计三维视线方向并降低计算复杂度.
Abstract:To address the issue of low accuracy and susceptibility to interference from external factors in unconstrained environments, a convolution and attention double-branch parallel feature cross-fusion gaze estimation method is proposed to enhance feature fusion effectiveness and network performance. Firstly, the Mobile-Former network is enhanced by introducing a linear attention mechanism and partial convolution. This effectively improves the feature extraction capability while reducing computing costs. Additionally, a branch of the ResNet50 head pose feature estimation network, pre-trained on the 300W-LP dataset, is added to enhance gaze estimation accuracy. A Sigmoid function is used as a gating unit to screen effective features. Finally, facial images are inputted into the neural network for feature extraction and fusion, and the 3D gaze estimation direction is outputted. The model is evaluated on the MPIIFaceGaze and Gaze360 datasets, and the average angle error of the proposed method is 3.70° and 10.82°, respectively. The network model is verified to accurately estimate the 3D gaze direction and reduce computational complexity compared to other mainstream 3D gaze estimation methods.
文章编号:     中图分类号:    文献标志码:
基金项目:国家自然科学基金(52072293)
引用文本:
吕嘉琦,王长元.双向融合CNN与Transformer的三维视线估计.计算机系统应用,2024,33(10):66-74
LYU Jia-Qi,WANG Chang-Yuan.3D Gaze Estimation by Bidirectional Fusion of CNN and Transformer.COMPUTER SYSTEMS APPLICATIONS,2024,33(10):66-74