对于PAD （pleasure：愉悦度；arousal：唤醒度；dominance：优势度）维度情感预测和分析中的数值预测问题，结合心率变异性（heart rate variability，HRV）特点，提出了基于主成分分析（principal component analysis，PCA）和支持向量回归（support vector regression，SVR）的PAD维度情感预测模型（PCA-SVR）。通过柔性离子传感器以音乐和视频的诱导方式采集了12名志愿者在放松和焦虑两种情感状态下的心率和心率间期数据，利用PAD量表进行标注，通过均值和方差计算等统计方法、Welch功率谱、Poincaré散点图等分别提取HRV的时域、频域和非线性特征，然后利用PCA模型对HRV特征降维，最后利用降维后的HRV特征作为SVR模型的输入特征进行训练和预测。实验结果表明，结合HRV特征的PCA-SVR模型在PAD的3个维度上均有良好的预测效果，其平均一致性相关系数达到了0.51。同时对比了SVR、极限学习机（extreme learning machine，ELM）和基于PCA的ELM这3种预测方法，结果显示所提方法相对于以上3种方法在一致性相关系数上分别提升了0.14、0.10和0.04，表明该方法能够细致地划分情感，结合可穿戴设备，在情感识别和分析方面有一定补充作用，为在日常生活中针对情感的识别和预测带来了可能。
Abstract
In order to solve the numerical prediction problem in PAD (pleasure, arousal and dominance) dimensional emotion prediction, a PAD dimensional emotion prediction model integrating heart rate variability (HRV) based on principal component analysis (PCA) and support vector regression (SVR) is proposed in this paper. The heart rate and heart interval data of 12 volunteers in two emotion states with relaxation and anxiety induced by music and video were collected by flexible iontronic sensing, and labeled on a PAD emotion scale. The time-domain, frequency-domain and nonlinear features of HRV were then extracted by different statistical methods, namely mean and variance, Welch power spectrum and Poincaré scatter diagram, respectively. Moreover, the PCA model was used to reduce the dimension of HRV features. The HRV features after dimensionality reduction were used as the input features of the SVR model for training and prediction. The experimental results show that the PCA-SVR model combined with HRV features had good prediction effects for the three dimensions of PAD, and its average consistency correlation coefficient (CCC) reached 0.51. The three prediction methods of the SVR, extreme learning machine (ELM) and the ELM based the PCA were compared, and the results showed that the proposed method resulted in improvements in CCC of 0.14, 0.10, and 0.04, respectively. Furthermore, the proposed method can divide emotions in detail, and has a certain complementary role in emotion recognition and analysis. Thus using the method in combination with wearable devices, it is possible to identify and predict emotions in daily life.
关键词
PAD维度情感 /
心率变异性 /
主成分分析 /
支持向量回归
{{custom_keyword}} /
Key words
PAD dimensional emotion /
heart rate variability /
principal component analysis /
support vector regression
{{custom_keyword}} /
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] PICARD R W. Affective computing[M]. Cambridge, USA:MIT press, 2000.
[2] 权学良, 曾志刚, 蒋建华, 等. 基于生理信号的情感计算研究综述[J]. 自动化学报, 2021, 47:1-16. QUAN X L, ZENG Z G, JIANG J H, et al. Physiological signals based affective computing:a systematic review[J]. Acta Automatica Sinica, 2021, 47:1-16. (in Chinese)
[3] ZHU J P, JI L Z, LIU C Y. Heart rate variability monitoring for emotion and disorders of emotion[J]. Physiological Measurement, 2019, 40(6):064004.
[4] 李幼军. 生理信号的情感计算研究及其应用[D]. 北京:北京工业大学, 2018. LI Y J. The study and application of affective computing based on bio-signals[D]. Beijing:Beijing University of Technology, 2018. (in Chinese)
[5] EKMAN P, FRIESEN W V. Constants across cultures in the face and emotion[J]. Journal of Personality and Social Psychology, 1971, 17(2):124-129.
[6] HE C, YAO Y J, YE X S. An emotion recognition system based on physiological signals obtained by wearable sensors[M]//YANG C, VIRK G, YANG H. Wearable sensors and robots. Lecture notes in electrical engineering. Singapore:Springer, 2017:15-25.
[7] GUO H W, HUANG Y S, LIN C H, et al. Heart rate variability signal features for emotion recognition by using principal component analysis and support vectors machine[C]//2016 IEEE 16th International Conference on Bioinformatics and Bioengineering (BIBE). Taichung:IEEE, 2016:274-277.
[8] CAI H S, QU Z D, LI Z, et al. Feature-level fusion approaches based on multimodal EEG data for depression recognition[J]. Information Fusion, 2020, 59:127-138.
[9] GUNES H, SCHULLER B. Categorical and dimensional affect analysis in continuous input:current trends and future directions[J]. Image and Vision Computing, 2013, 31(2):120-136.
[10] RUSSELL J A. A circumplex model of affect[J]. Journal of Personality and Social Psychology, 1980, 39(6):1161-1178.
[11] SONG T F, ZHENG W M, SONG P, et al. EEG emotion recognition using dynamical graph convolutional neural networks[J]. IEEE Transactions on Affective Computing, 2020, 11(3):532-541.
[12] SAMARA A, MENEZES M L R, GALWAY L. Feature extraction for emotion recognition and modelling using neurophysiological data[C]//201615th International Conference on Ubiquitous Computing and Communications and 20168th International Symposium on Cyberspace and Security (IUCC-CSS). Granada:IEEE, 2016:138-144.
[13] KEREN G, KIRSCHSTEIN T, MARCHI E, et al. End-to-end learning for dimensional emotion recognition from physiological signals[C]//2017 IEEE International Conference on Multimedia and Expo (ICME). Hong Kong:IEEE, 2017:985-990.
[14] YU W Y, DING S, YUE Z J, et al. Emotion recognition from facial expressions and contactless heart rate using knowledge graph[C]//2020 IEEE International Conference on Knowledge Graph (ICKG). Nanjing:IEEE, 2020:64-69.
[15] 胡艳香, 孙颖, 张雪英, 等. 基于聚类PSO-LSSVM模型的PAD维度预测[J]. 计算机应用研究, 2020, 37(4):994-998. HU Y X, SUN Y, ZHANG X Y, et al. Forecast of PAD dimensions using clustering PSO-LSSVM model[J]. Application Research of Computers, 2020, 37(4):994-998. (in Chinese)
[16] MEHRABIAN A, RUSSELL J A. An approach to environmental psychology[M]. Cambridge, USA:MIT Press, 1974.
[17] MEHRABIAN A. Framework for a comprehensive description and measurement of emotional states[J]. Genetic, Social, and General Psychology Monographs, 1995, 121(3):339-361.
[18] 李晓明, 傅小兰, 邓国峰. 中文简化版PAD情绪量表在京大学生中的初步试用[J]. 中国心理卫生杂志, 2008, 22(5):327-329. LI X M, FU X L, DENG G F. Preliminary application of the abbreviated PAD emotion scale to Chinese undergraduates[J]. Chinese Mental Health Journal, 2008, 22(5):327-329. (in Chinese)
[19] NICOLAOU M A, GUNES H, PANTIC M. Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space[J]. IEEE Transactions on Affective Computing, 2011, 2(2):92-105.
[20] KU W F, STORER R H, GEORGAKIS C. Disturbance detection and isolation by dynamic principal component analysis[J]. Chemometrics and Intelligent Laboratory Systems, 1995, 30(1):179-196.
[21] CORTES C, VAPNIK V. Support-vector networks[J]. Machine Learning, 1995, 20(3):273-297.
[22] ZHANG Z C, ZHU Z J, BAZOR B, et al. FeetBeat:a flexible lontronic sensing wearable detects pedal pulses and muscular activities[J]. IEEE Transactions on Biomedical Engineering, 2019, 66(11):3072-3079.
[23] FUKUNUSHI M, MCDUFF D, TSUMURA N. Improvements in remote video based estimation of heart rate variability using the Welch FFT method[J]. Artificial Life and Robotics, 2018, 23(1):15-22.
[24] HOSHI R A, PASTRE C M, VANDERLEI L C M, et al. Poincaré plot indexes of heart rate variability:relationships with other nonlinear variables[J]. Autonomic Neuroscience:Basic and Clinical, 2013, 177(2):271-274.
[25] LI J, YAN J Q, LIU X Z, et al. Using permutation entropy to measure the changes in EEG signals during absence seizures[J]. Entropy, 2014, 16(6):3049-3061.
[26] MISHRA A K, RAGHAV S. Local fractal dimension based ECG arrhythmia classification[J]. Biomedical Signal Processing and Control, 2010, 5(2):114-123.
[27] 孙颖, 胡艳香, 张雪英, 等. 面向情感语音识别的情感维度PAD预测[J]. 浙江大学学报(工学版), 2019, 53(10):2041-2048. SUN Y, HU Y X, ZHANG X Y, et al. Prediction of emotional dimensions PAD for emotional speech recognition[J]. Journal of Zhejiang University (Engineering Science), 2019, 53(10):2041-2048. (in Chinese)
[28] HUANG G B, ZHU Q Y, SIEW C K. Extreme learning machine:theory and applications[J]. Neurocomputing, 2006, 70(1-3):489-501.
{{custom_fnGroup.title_cn}}
脚注
{{custom_fn.content}}
基金
中央高校基本科研业务费（XK1802-4）；贵州科技计划项目重大专项（GuizhouBranch[2018]3002）
{{custom_fund}}
PDF(1026 KB)
