-
基于人脸视频的生理信号检测面临的主要挑战是运动伪影噪声. 针对受试者头部刚性旋转运动引起的伪影噪声, 本文提出利用头部运动信息构建自适应滤波器的非接触式心率检测方法. 该方法利用人脸二维和三维的特征点计算受试者运动中头部的偏航和俯仰欧拉角度, 并将其作为调控过程噪声协方差的信号质量指数, 进而构建了自适应Kalman滤波器, 实现了稳健的心率估计. 实验结果表明: 本文提出的方法可有效抑制头部刚性旋转运动引起的噪声, 平均绝对误差为2.22 beat/min, 均方根误差为2.76 beat/min, 与现有方法相比准确度分别提升9%与24.6%, 具有统计显著性. 本文提出的头部旋转角度自适应非接触鲁棒性心率检测方法在自发运动的真实场景下能有效提升检测的准确性, 扩大了成像式光电容积描记技术在视频健康监测领域的使用场景.
-
关键词:
- 人脸视频 /
- 非接触式心率检测 /
- 头部旋转运动 /
- 自适应Kalman滤波器
The dominant challenge of vital signal monitoring based on facial video is to eliminate the interference of motion artifacts. In this paper, we propose a non-contact heart rate detection method based on an adaptive filter constructed by head movement information to tackle the noise of motion artifacts caused by the rigid rotation of the subject's head. The two-dimensional and three-dimensional feature points of the subject’s face are used to calculate the yaw and pitch Euler angles of the head movement, then the yaw and pitch Euler angles are used as a novel signal quality index (SQI) for modulating process noise covariance to construct an adaptive Kalman filter, and finally robust heart rate is estimated by this method. The experimental results show that the proposed method can effectively suppress the noise caused by the head rigid rotation with an average absolute error of 2.22 beat/min and a root mean square error of 2.76 beat/min, which are statistically significant with an accuracy improvement of 9% and 24.6%, respectively, compared with the existing methods. The adaptive non-contact robust heart rate detection technique based on head rigid rotation may effectively enhance the accuracy in real-world motion situations, as well as broaden the range of applications for IPPG in the field of the video-based monitoring of health conditions.-
Keywords:
- facial video /
- non-contact heart rate detection method /
- head rotation /
- adaptive Kalman filter
[1] Franco M, Cooper R S, Bilal U, Fuster V 2011 Am. J. Med. 124 95
Google Scholar
[2] Allen J 2007 Physiol. Meas. 28 R1
Google Scholar
[3] Shelley K H 2007 Anesth. Analg. 105 S31
Google Scholar
[4] Sun Y, Thakor N 2016 IEEE Trans. Biomed. Eng. 63 463
Google Scholar
[5] Hulsbusch M, Blazek V 2002 Medical Imaging San Diego, California, USA, April 24, 2002 110
[6] Verkruysse W, Svaasand L O, Nelson J S 2008 Opt. Express 16 21434
[7] Wieringa F P, Mastik F, Steen A 2005 Ann. Biomed. Eng. 33 1034
Google Scholar
[8] Shao D, Yang Y, Liu C, Tsow F, Yu H, Tao N 2014 IEEE Trans. Biomed. Eng. 61 2760
Google Scholar
[9] Hülsbusch M 2008 Ph. D. Dissertation (Aachen: RWTH Aachen University) (in German)
[10] Poh M Z, McDuff D J, Picard R W 2010 Opt. Express 18 10762
Google Scholar
[11] Poh M Z, McDuff D J, Picard R W 2010 IEEE Trans. Biomed. Eng. 58 7
Google Scholar
[12] Lewandowska M, Ruminski J, Kocejko T, Nowak J 2011 Federated Conference on Computer Science and Information Systems-FedCSIS 2011 Szczecin, Poland, September 18–21, 2011 p405
[13] Haan G D, Jeanne V 2013 IEEE Trans. Biomed. Eng. 60 2878
Google Scholar
[14] Wang W, Brinker A C D, Stuijk S, Haan G D 2017 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) Washington, USA, May 30–June 3, 2017 p71
[15] Wang W, Brinker A C D, Stuijk S, Haan G D 2016 IEEE Trans. Biomed. Eng. 64 1479
Google Scholar
[16] Wang W, Brinker A C D, Stuijk S, Haan G D 2017 Physiol. Meas. 38 1023
Google Scholar
[17] Sun Y, Hu S, Azorin-Peris V, Greenwald S, Chambers J, Zhu Y 2011 J. Biomed. Opt. 16 077010
Google Scholar
[18] Wang W, Stuijk S, Haan G D 2014 IEEE Trans. Biomed. Eng. 62 415
Google Scholar
[19] Wu B F, Huang P W, Lin C H, Chung M L, Tsou T Y, Wu Y L 2018 IEEE Access 6 21621
Google Scholar
[20] Kong L, Wu Y, Zhao Y, Dong L, Hui M, Liu M, Liu X 2020 IEEE Photonics J. 12 1
Google Scholar
[21] Viola P A, Jones M J 2001 Computer Vision and Pattern Recognition Kauai, Hawaii, USA, December 8–14, 2001 p511
[22] Kwon S, Kim J, Lee D, Park K 2015 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) Milan, Italy, August 25–29, 2015 p4938
[23] 杨萍, 侯威, 封国林 2008 物理学报 57 5333
Google Scholar
Yang P, Hou W, Feng G L 2008 Acta Phys. Sin. 57 5333
Google Scholar
[24] Bousefsaf F, Maaoui C, Pruski A 2013 Biomed. Signal Process. Control 8 568
Google Scholar
[25] Haan G D, Leest A V 2014 Physiol. Meas. 35 1913
Google Scholar
[26] 孔令琴 2014 博士学位论文 (北京: 北京理工大学)
Kong L 2014 Ph. D. Dissertation (Beijing: Beijing Institute of Technology) (in Chinese)
[27] Wang W 2017 Ph. D. Dissertation (Eindhoven, The Netherlands: Eindhoven University of Technology)
[28] Smith W J 2008 Modern Optical Engineering: The Design of Optical Systems (4th Ed.) (New York: The McGraw-Hill Companies, Inc) p253
[29] Asthana A, Zafeiriou S, Cheng S, Pantic M 2013 Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on Portland, Oregon, USA, June 23–28, 2013 p3444
[30] Baltrusaitis T, Robinson P, Morency L 2012 IEEE Conference on Computer Vision & Pattern Recognition Providence, Rhode island, USA, June 16–21, 2012 p2610
[31] Baltrusaitis T, Robinson P, Morency L 2016 IEEE Winter Conference on Applications of Computer Vision Lake Placid, New York, USA, March 7–10, 2016 p1
[32] Baltrusaitis T, Zadeh A, Lim Y C, Morency L 2018 IEEE International Conference on Automatic Face & Gesture Recognition Xi’an, China, May 15–19, 2018 p59
[33] Andreotti F, Trumpp A, Malberg H, Zaunseder S 2015 2015 IEEE 35th International Conference on Electronics and Nanotechnology (ELNANO) Kyiv, Ukraine, April 21–24, 2015 p428
[34] 张玉燕, 殷东哲, 温银堂, 罗小元 2021 物理学报 70 118102
Google Scholar
Zhang Y Y, Yin D Z, Wen Y T, Luo S Y 2021 Acta Phys. Sin. 70 118102
Google Scholar
[35] Nemati S, Malhotra A, Clifford G 2010 EURASIP J. Adv. Signal Process. 2010 926305
Google Scholar
[36] Tarvainen M P, Ranta-Aho P O, Karjalainen P A 2002 IEEE Trans. Biomed. Eng. 49 172
Google Scholar
[37] Fallet S, Moser V, Braun F, Vesin J M 2017 Computing in Cardiology Conference Vancouver, BC, Canada, September 11–14, 2016 p341
-
图 2 头部旋转角度信息作为自适应滤波器质量控制的流程图
Fig. 2. Flow chart of head rotation angles as a quality control of our proposed adaptive filter.
图 3 实验方案图 (a) 实验装置示意图; (b) 采集到的视频片段((i)头部姿势的欧拉角为0°; (ii) 头部俯仰运动; (iii) 头部偏航)
Fig. 3. Experimental plan diagram: (a) Experiment set-up; (b) video clips ((i) the Euler angle of the head pose is 0°; (ii) the head pitch segment; (iii) the head yaw segment).
图 4 矩形ROI区域选取示意图 (a)面部68个关键点检测; (b)脸颊矩形ROI区域; (c)偏航运动时ROI区域
Fig. 4. Flow chart of rectangular ROI area selection: (a) 68 feature points detection; (b) rectangular ROI area; (c) ROI area during yaw motion.
图 6 本文提出的滤波器与CHROM结合的对比结果
Fig. 6. Comparison results of the filter proposed in this paper combined with CHROM.
图 7 21组实验结果相关性图(图中红线表示y=x的线性关系) (a) CHROM方法的相关性图; (b)本文提出的滤波器与CHROM结合的相关性图
Fig. 7. Correlation plots of 21 groups of experimental results (the red lines in the plots indicate linear relationship of y = x): (a) Correlation plot of CHROM method; (b) correlation plot of combining CHROM with our proposed adaptive filter.
表 1 本文提出的自适应滤波器与不同的传统方法结合前后实验结果对比
Table 1. Comparison of experimental results before and after combining the filter proposed in this paper with different traditional methods.
方法类型 传统的方法 传统方法结合本文的滤波器 MAE/(beat·min–1) RMSE/(beat·min–1) $ {\mathit{R}}^{2} $ MAE/(beat·min–1) RMSE/(beat·min–1) $ {\mathit{R}}^{2} $ GREEN 5.98 8.65 0.5471 3.88 4.95 0.7217 CDF 5.46 9.22 0.5993 3.18 4.16 0.8402 POS 3.22 4.85 0.7644 2.86 3.61 0.8181 POS+CDF 6.69 11.59 0.4774 3.25 4.42 0.8517 CHROM 2.44 3.66 0.8142 2.22 2.74 0.8401 CHROM+CDF 4.71 8.90 0.7514 2.62 3.46 0.8901 
下载: 导出CSV
-
[1] Franco M, Cooper R S, Bilal U, Fuster V 2011 Am. J. Med. 124 95
Google Scholar
[2] Allen J 2007 Physiol. Meas. 28 R1
Google Scholar
[3] Shelley K H 2007 Anesth. Analg. 105 S31
Google Scholar
[4] Sun Y, Thakor N 2016 IEEE Trans. Biomed. Eng. 63 463
Google Scholar
[5] Hulsbusch M, Blazek V 2002 Medical Imaging San Diego, California, USA, April 24, 2002 110
[6] Verkruysse W, Svaasand L O, Nelson J S 2008 Opt. Express 16 21434
[7] Wieringa F P, Mastik F, Steen A 2005 Ann. Biomed. Eng. 33 1034
Google Scholar
[8] Shao D, Yang Y, Liu C, Tsow F, Yu H, Tao N 2014 IEEE Trans. Biomed. Eng. 61 2760
Google Scholar
[9] Hülsbusch M 2008 Ph. D. Dissertation (Aachen: RWTH Aachen University) (in German)
[10] Poh M Z, McDuff D J, Picard R W 2010 Opt. Express 18 10762
Google Scholar
[11] Poh M Z, McDuff D J, Picard R W 2010 IEEE Trans. Biomed. Eng. 58 7
Google Scholar
[12] Lewandowska M, Ruminski J, Kocejko T, Nowak J 2011 Federated Conference on Computer Science and Information Systems-FedCSIS 2011 Szczecin, Poland, September 18–21, 2011 p405
[13] Haan G D, Jeanne V 2013 IEEE Trans. Biomed. Eng. 60 2878
Google Scholar
[14] Wang W, Brinker A C D, Stuijk S, Haan G D 2017 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017) Washington, USA, May 30–June 3, 2017 p71
[15] Wang W, Brinker A C D, Stuijk S, Haan G D 2016 IEEE Trans. Biomed. Eng. 64 1479
Google Scholar
[16] Wang W, Brinker A C D, Stuijk S, Haan G D 2017 Physiol. Meas. 38 1023
Google Scholar
[17] Sun Y, Hu S, Azorin-Peris V, Greenwald S, Chambers J, Zhu Y 2011 J. Biomed. Opt. 16 077010
Google Scholar
[18] Wang W, Stuijk S, Haan G D 2014 IEEE Trans. Biomed. Eng. 62 415
Google Scholar
[19] Wu B F, Huang P W, Lin C H, Chung M L, Tsou T Y, Wu Y L 2018 IEEE Access 6 21621
Google Scholar
[20] Kong L, Wu Y, Zhao Y, Dong L, Hui M, Liu M, Liu X 2020 IEEE Photonics J. 12 1
Google Scholar
[21] Viola P A, Jones M J 2001 Computer Vision and Pattern Recognition Kauai, Hawaii, USA, December 8–14, 2001 p511
[22] Kwon S, Kim J, Lee D, Park K 2015 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) Milan, Italy, August 25–29, 2015 p4938
[23] 杨萍, 侯威, 封国林 2008 物理学报 57 5333
Google Scholar
Yang P, Hou W, Feng G L 2008 Acta Phys. Sin. 57 5333
Google Scholar
[24] Bousefsaf F, Maaoui C, Pruski A 2013 Biomed. Signal Process. Control 8 568
Google Scholar
[25] Haan G D, Leest A V 2014 Physiol. Meas. 35 1913
Google Scholar
[26] 孔令琴 2014 博士学位论文 (北京: 北京理工大学)
Kong L 2014 Ph. D. Dissertation (Beijing: Beijing Institute of Technology) (in Chinese)
[27] Wang W 2017 Ph. D. Dissertation (Eindhoven, The Netherlands: Eindhoven University of Technology)
[28] Smith W J 2008 Modern Optical Engineering: The Design of Optical Systems (4th Ed.) (New York: The McGraw-Hill Companies, Inc) p253
[29] Asthana A, Zafeiriou S, Cheng S, Pantic M 2013 Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on Portland, Oregon, USA, June 23–28, 2013 p3444
[30] Baltrusaitis T, Robinson P, Morency L 2012 IEEE Conference on Computer Vision & Pattern Recognition Providence, Rhode island, USA, June 16–21, 2012 p2610
[31] Baltrusaitis T, Robinson P, Morency L 2016 IEEE Winter Conference on Applications of Computer Vision Lake Placid, New York, USA, March 7–10, 2016 p1
[32] Baltrusaitis T, Zadeh A, Lim Y C, Morency L 2018 IEEE International Conference on Automatic Face & Gesture Recognition Xi’an, China, May 15–19, 2018 p59
[33] Andreotti F, Trumpp A, Malberg H, Zaunseder S 2015 2015 IEEE 35th International Conference on Electronics and Nanotechnology (ELNANO) Kyiv, Ukraine, April 21–24, 2015 p428
[34] 张玉燕, 殷东哲, 温银堂, 罗小元 2021 物理学报 70 118102
Google Scholar
Zhang Y Y, Yin D Z, Wen Y T, Luo S Y 2021 Acta Phys. Sin. 70 118102
Google Scholar
[35] Nemati S, Malhotra A, Clifford G 2010 EURASIP J. Adv. Signal Process. 2010 926305
Google Scholar
[36] Tarvainen M P, Ranta-Aho P O, Karjalainen P A 2002 IEEE Trans. Biomed. Eng. 49 172
Google Scholar
[37] Fallet S, Moser V, Braun F, Vesin J M 2017 Computing in Cardiology Conference Vancouver, BC, Canada, September 11–14, 2016 p341
下载:
计量
- 文章访问数: 5146
- PDF下载量: 73
- 被引次数: 0
