Classroom Behavior Recognition of Students Based on Improved YOWO Algorithm
CSTR:
Author:
  • Article
  • | |
  • Metrics
  • |
  • Reference [21]
  • |
  • Related
  • | | |
  • Comments
    Abstract:

    At present, since the recognition of most students’ classroom behavior is mainly based on a single frame image and ignores behavior coherence, video information cannot be made full use of to accurately depict students’ classroom behavior. Therefore, this study proposes an improved YOWO algorithm model to effectively employ video information to identify students’ classroom behavior. First, this paper collects teaching videos from real classroom teaching in a university and produces an AVA format video dataset containing five types of students’ classroom behavior. Second, the temporal shift module (TSM) is adopted to enhance the ability of this model to obtain time context information. Finally, a non-local operation module is utilized to improve the ability of the model to extract key location information. The experimental results show that by optimizing the YOWO model, the recognition performance of the network is better. In the classroom behavior dataset, the mAP value of the improved algorithm is 95.7%, 4.6% higher than that of the original YOWO algorithm. The parameter number in the model is reduced by 32.3% at 81.97×106 and the calculation amount is decreased by 9.6% at 22.6 GFLOPs. The detection speed is 24.03 f/s, an increase of about 3 f/s.

    Reference
    [1] 周鹏霄, 邓伟, 郭培育, 等. 课堂教学视频中的S-T行为智能识别研究. 现代教育技术, 2018, 28(6): 54–59.
    [2] 党冬利. 人体行为识别及在教育录播系统中的应用 [硕士学位论文]. 西安: 西安科技大学, 2017.
    [3] 廖鹏, 刘宸铭, 苏航, 等. 基于深度学习的学生课堂异常行为检测与分析系统. 电子世界, 2018(8): 97–98.
    [4] 秦道影. 基于深度学习的学生课堂行为识别 [硕士学位论文]. 武汉: 华中师范大学, 2019.
    [5] 蒋沁沂, 张译文, 谭思琪, 等. 基于残差网络的学生课堂行为识别. 现代计算机, 2019(20): 23–27.
    [6] 周叶. 基于Faster R-CNN的小学生课堂行为检测研究 [硕士学位论文]. 成都: 四川师范大学, 2021.
    [7] 杜俊强. 基于深度学习的学生课堂行为识别方法研究 [硕士学位论文]. 烟台: 山东工商学院, 2022.
    [8] 董琪琪, 刘剑飞, 郝禄国, 等. 基于改进SSD算法的学生课堂行为状态识别. 计算机工程与设计, 2021, 42(10): 2924–2930.
    [9] 孙绍涵, 张运楚, 王超, 等. 基于深度学习的学生课堂注意力评价. 计算机系统应用, 2022, 31(6): 307–314.
    [10] 林灿然, 许伟亮, 李逸. 基于多模态数据的课堂学生行为识别技术的探究. 现代计算机, 2020, (6): 69–75.
    [11] 郭俊奇, 吕嘉昊, 王汝涵, 等. 深度学习模型驱动的师生课堂行为识别. 北京师范大学学报(自然科学版), 2021, 57(6): 905–912.
    [12] 郑丹. 基于双流卷积神经网络的学生课堂行为识别研究[硕士学位论文]. 沈阳: 沈阳师范大学, 2022.
    [13] Köpüklü O, Wei XY, Rigoll G. You only watch once: A unified cnn architecture for real-time spatiotemporal action localization. arXiv:1911.06644, 2019.
    [14] Wu T, Cao MQ, Gao ZT, et al. STMixer: A one-stage sparse action detector. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023. 14720–14729.
    [15] Chen SF, Sun PZ, Xie EZ, et al. Watch only once: An end-to-end video action detection framework. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021. 8158–8167.
    [16] Chi HG, Lee K, Agarwal N, et al. AdamsFormer for spatial action localization in the future. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023. 17885–17895.
    [17] Lin J, Gan C, Han S. TSM: Temporal shift module for efficient video understanding. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019. 7082–7092.
    [18] Yang CY, Xu YH, Shi JP, et al. Temporal pyramid network for action recognition. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020. 588–597.
    [19] Feichtenhofer C. X3D: Expanding architectures for efficient video recognition. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020. 200–210.
    [20] Wang LM, Tong Z, Ji B, et al. TDN: Temporal difference networks for efficient action recognition. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021. 1895–1904.
    [21] Wang XL, Girshick R, Gupta A, et al. Non-local neural networks. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018. 7794–7803.
    Related
    Cited by
Get Citation

徐鑫磊,张景异.基于改进YOWO算法的学生课堂行为识别.计算机系统应用,2024,33(4):113-122

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:September 09,2023
  • Revised:October 08,2023
  • Online: January 18,2024
Article QR Code
You are the first990394Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-3
Address:4# South Fourth Street, Zhongguancun,Haidian, Beijing,Postal Code:100190
Phone:010-62661041 Fax: Email:csa (a) iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063