Review on Non-contact Detection Methods of Heart Rate and SpO2 Based on Video Signal
Author:
  • Article
  • | |
  • Metrics
  • |
  • Reference [56]
  • |
  • Related
  • |
  • Cited by
  • | |
  • Comments
    Abstract:

    Heart rate and saturation of peripheral capillary oxygenation (SpO2) are very important physiological indicators of human health. In recent years, non-contact heart rate and SpO2 detection methods based on imaging photoplethysmography (IPPG) have gradually become a research focus as they are convenient and freely-applied. The main work is as follows. First, the study introduces the background and research significance of non-contact detection methods. Secondly, two aspects of target region detection and region of interest (ROI) are selected to summarize and clarify the research status and future improvement direction. Thirdly, the detection methods of heart rate and SpO2 are summarized from three aspects: traditional method, signal processing combined with deep learning method and end-to-end method, and the data sets used in deep learning method and the detection effects displayed in each data set are sorted out. Finally, the paper points out the problems that need to be solved and the future research direction in this field.

    Reference
    [1] Qi L, Yu HD, Xu LS, et al. Robust heart-rate estimation from facial videos using Project_ICA. Physiological Measurement, 2019, 40(8): 085007.
    [2] Jain PK, Tiwari AK. Heart monitoring systems—A review. Computers in Biology and Medicine, 2014, 54: 1–13.
    [3] Tulppo MP, Kiviniemi AM, Junttila MJ, et al. Home monitoring of heart rate as a predictor of imminent cardiovascular events. Frontiers in Physiology, 2019, 10: 341.
    [4] Mather M, Thayer JF. How heart rate variability affects emotion regulation brain networks. Current Opinion in Behavioral Sciences, 2018, 19: 98–104.
    [5] Poh MZ, McDuff DJ, Picard RW. Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Transactions on Biomedical Engineering, 2011, 58(1): 7–11.
    [6] Poh MZ, McDuff DJ, Picard RW. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Optics Express, 2010, 18(10): 10762–10774.
    [7] 周秦武, 隋芳芳, 白平, 等. 嵌入式无接触视频心率检测方法. 西安交通大学学报, 2013, 47(12): 55–60.
    [8] Li XB, Chen J, Zhao GY, et al. Remote heart rate measurement from face videos under realistic situations. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014. 4264–4271.
    [9] Al-Naji A, Khalid GA, Mahdi JF, et al. Non-contact SpO2 prediction system based on a digital camera. Applied Sciences, 2021, 11(9): 4255.
    [10] Viola P, Jones M. Robust real-time object detection. Proceedings of the 2nd International Workshop on Statistical and Computational Theories of Vision—Modeling, Learning, Computing, and Sampling. Vancouver, 2001. 137–154.
    [11] 贾海鹏, 张云泉, 袁良, 等. 基于OpenCL的Viola-Jones人脸检测算法性能优化研究. 计算机学报, 2016, 39(9): 1775–1789.
    [12] Wu BF, Chu YW, Huang PW, et al. Neural network based luminance variation resistant remote-photoplethysmography for driver’s heart rate monitoring. IEEE Access, 2019, 7: 57210–57225.
    [13] 汪秀军, 曹大平, 曹宜, 等. 非接触式生理参数测量设备的研究. 信息技术, 2018, 42(8): 17–22.
    [14] 郑银环, 王备战, 王嘉珺, 等. 深度卷积神经网络应用于人脸特征点检测研究. 计算机工程与应用, 2019, 55(4): 173–178.
    [15] Hsu GSJ, Xie RC, Ambikapathi A, et al. A deep learning framework for heart rate estimation from facial videos. Neurocomputing, 2020, 417: 155–166.
    [16] Verkruysse W, Svaasand LO, Nelson JS. Remote plethysmographic imaging using ambient light. Optics Express, 2008, 16(26): 21434–21445.
    [17] 赵飞帆, 方路平, 陈仕骁. 基于人脸视频测量脉搏方法的初步研究. 生物医学工程学杂志, 2012, 29(5): 876–879, 918.
    [18] Xie K, Fu CH, Liang HG, et al. Non-contact heart rate monitoring for intensive exercise based on singular spectrum analysis. Proceedings of the 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR). San Jose: IEEE, 2019. 228–233.
    [19] Kong LQ, Zhao YJ, Dong LQ, et al. Non-contact detection of oxygen saturation based on visible light imaging device using ambient light. Optics Express, 2013, 21(15): 17464–17471.
    [20] Shao DD, Liu CB, Tsow F, et al. Noncontact monitoring of blood oxygen saturation using camera and dual-wavelength imaging system. IEEE Transactions on Biomedical Engineering, 2016, 63(6): 1091–1098.
    [21] Wei B, He X, Zhang C, et al. Non-contact, synchronous dynamic measurement of respiratory rate and heart rate based on dual sensitive regions. BioMedical Engineering OnLine, 2017, 16(1): 17.
    [22] Casalino G, Castellano G, Zaza G. Evaluating the robustness of a contact-less mHealth solution for personal and remote monitoring of blood oxygen saturation. Journal of Ambient Intelligence and Humanized Computing, 2023, 14(7): 8871–8880.
    [23] Jonathan E, Leahy M. Investigating a smartphone imaging unit for photoplethysmography. Physiological Measurement, 2010, 31(11): N79–N83.
    [24] Gregoski MJ, Mueller M, Vertegel A, et al. Development and validation of a smartphone heart rate acquisition application for health promotion and wellness telehealth applications. International Journal of Telemedicine and Applications, 2012, 2012: 696324.
    [25] Nam Y, Kong Y, Reyes B, et al. Monitoring of heart and breathing rates using dual cameras on a smartphone. PLoS One, 2016, 11(3): e0151013.
    [26] Wu R, Gao HY, Zhang C, et al. To investigate the feasibility of palm-video-based IPPG technique. Proceedings of the 5th International Conference on Information Communication and Signal Processing (ICICSP). Shenzhen: IEEE, 2022. 296–302.
    [27] 曾旭, 代俊丽, 赵建, 等. IPPG信号的噪声估计与分析. 航天医学与医学工程, 2021, 34(1): 52–58.
    [28] Sun Y, Hu S, Azorin-Peris V, et al. Noncontact imaging photoplethysmography to effectively access pulse rate variability. Journal of Biomedical Optics, 2012, 18(6): 061205.
    [29] Lenskiy AA, Lee JS. Driver’s eye blinking detection using novel color and texture segmentation algorithms. International Journal of Control, Automation and Systems, 2012, 10(2): 317–327.
    [30] Wieringa FP, Mastik F, Van Der Steen AFW. Contactless multiple wavelength photoplethysmographic imaging: A first step toward “SpO2 camera” technology. Annals of Biomedical Engineering, 2005, 33(8): 1034–1041.
    [31] Monkaresi H, Calvo RA, Yan H. A machine learning approach to improve contactless heart rate monitoring using a webcam. IEEE Journal of Biomedical and Health Informatics, 2014, 18(4): 1153–1160.
    [32] Al-Naji A, Perera AG, Chahl J. Remote measurement of cardiopulmonary signal using an unmanned aerial vehicle. IOP Conference Series: Materials Science and Engineering, 2018, 405: 012001.
    [33] 嵇晓强, 刘振瑶, 李炳霖, 等. 面部视频非接触式生理参数感知. 中国光学, 2022, 15(2): 276–285.
    [34] Hsu GS, Ambikapathi AM, Chen MS. Deep learning with time-frequency representation for pulse estimation from facial videos. Proceedings of the 2017 IEEE International Joint Conference on Biometrics. Denver: IEEE, 2017. 383–389.
    [35] Qiu Y, Liu Y, Arteaga-Falconi J, et al. EVM-CNN: Real-time contactless heart rate estimation from facial video. IEEE Transactions on Multimedia, 2019, 21(7): 1778–1787.
    [36] Niu XS, Han H, Shan SG, et al. SynRhythm: Learning a deep heart rate estimator from general to specific. Proceedings of the 24th International Conference on Pattern Recognition. Beijing: IEEE, 2018. 3580–3585.
    [37] 吕浩原. 基于深度学习的非接触式心率检测系统研究 [硕士学位论文]. 大庆: 东北石油大学, 2023.
    [38] Wu CL, Yuan ZY, Wan SH, et al. Anti-jamming heart rate estimation using a spatial-temporal fusion network. Computer Vision and Image Understanding, 2022, 216: 103327.
    [39] Yu SN, Wang CS, Chang YP. Heart rate estimation from remote photoplethysmography based on light-weight U-Net and attention modules. IEEE Access, 2023, 11: 54058–54069.
    [40] Jaiswal KB, Meenpal T. Heart rate estimation network from facial videos using spatiotemporal feature image. Computers in Biology and Medicine, 2022, 151: 106307.
    [41] Hu M, Qian F, Guo D, et al. ETA-rPPGNet: Effective time-domain attention network for remote heart rate measurement. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 2506212.
    [42] Yin RN, Jia RS, Cui Z, et al. PulseNet: A multitask learning network for remote heart rate estimation. Knowledge-based Systems, 2021, 239: 108048.
    [43] Hu M, Wu X, Wang XH, et al. Contactless blood oxygen estimation from face videos: A multi-model fusion method based on deep learning. Biomedical Signal Processing and Control, 2023, 81: 104487.
    [44] Ouzar Y, Djeldjli D, Bousefsaf F, et al. X-iPPGNet: A novel one stage deep learning architecture based on depthwise separable convolutions for video-based pulse rate estimation. Computers in Biology and Medicine, 2023, 154: 106592.
    [45] Zhang Z, Girard JM, Wu Y, et al. Multimodal spontaneous emotion corpus for human behavior analysis. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016. 3438–3446.
    [46] Soleymani M, Lichtenauer J, Pun T, et al. A multimodal database for affect recognition and implicit tagging. IEEE Transactions on Affective Computing, 2012, 3(1): 42–55.
    [47] Bennett S, El Harake TN, Goubran R, et al. Adaptive eulerian video processing of thermal video: An experimental analysis. IEEE Transactions on Instrumentation and Measurement, 2017, 66(10): 2516–2524.
    [48] Niu XS, Han H, Shan SG, et al. VIPL-HR: A multi-modal database for pulse estimation from less-constrained face video. Proceedings of the 14th Asian Conference on Computer Vision. Perth: Springer, 2019. 562–576.
    [49] Stricker R, Müller S, Gross HM. Non-contact video-based pulse rate measurement on a mobile service robot. Proceedings of the 23rd IEEE International Symposium on Robot and Human Interactive Communication. Edinburgh: IEEE, 2014. 1056–1062.
    [50] Li XB, Alikhani I, Shi JG, et al. The OBF database: A large face video database for remote physiological signal measurement and atrial fibrillation detection. Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition. Xi’an: IEEE, 2018. 242–249.
    [51] Pilz CS, Zaunseder S, Krajewski J, et al. Local group invariance for heart rate estimation from face videos in the wild. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City: IEEE, 2018. 1335–13358.
    [52] Heusch G, Anjos A, Marcel S. A reproducible study on remote heart rate measurement. arXiv:1709.00962, 2017.
    [53] Spetlík R, Franc V, Cech J, et al. Visual heart rate estimation with convolutional neural network. Proceedings of the 2018 British Machine Vision Conference. Newcastle: BMVC, 2018. 1–12.
    [54] Yu ZT, Peng W, Li XB, et al. Remote heart rate measurement from highly compressed facial videos: An end-to-end deep learning solution with video enhancement. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul: IEEE, 2019. 151–160.
    [55] Sabokrou M, Pourreza M, Li XB, et al. Deep-HR: Fast heart rate estimation from face video under realistic conditions. Expert Systems with Applications, 2021, 186: 115596.
    [56] Li B, Zhang PP, Peng JY, et al. Non-contact PPG signal and heart rate estimation with multi-hierarchical convolutional network. Pattern Recognition, 2023, 139: 109421.
    Related
    Cited by
    Comments
    Comments
    分享到微博
    Submit
Get Citation

赵娅,王世铎,贾迪.基于视频信号的非接触式心率和血氧饱和度检测方法综述.计算机系统应用,2024,33(10):26-36

Copy
Share
Article Metrics
  • Abstract:
  • PDF:
  • HTML:
  • Cited by:
History
  • Received:February 27,2024
  • Revised:May 06,2024
  • Online: August 21,2024
Article QR Code
You are the first990552Visitors
Copyright: Institute of Software, Chinese Academy of Sciences Beijing ICP No. 05046678-3
Address:4# South Fourth Street, Zhongguancun,Haidian, Beijing,Postal Code:100190
Phone:010-62661041 Fax: Email:csa (a) iscas.ac.cn
Technical Support:Beijing Qinyun Technology Development Co., Ltd.

Beijing Public Network Security No. 11040202500063