Facial Expressions as Behavioral Indicators for Assessing Pain using Machine Learning Models

Abedi, WMS., Ahmed T. S., and Nadher, I., "Modified CNN-LSTM for Pain Facial Expressions Recognition." (2020).

Ahmed A., Yang A., and Taati. B. "Pain expression recognition using occluded faces." 2019 14th IEEE international conference on Automatic Face & Gesture Recognition (FG 2019). IEEE, 2019. https://doi.org/10.1109/FG.2019.8756594

Alghamdi, T., and Alaghband, G. "Facial expressions based automatic pain assessment system." Applied Sciences 12.13 (2022): 6423. 10.3390/app12136423

Anwar, D. A. "Real Time Pain Detection Using Facial Action Units in Telehealth System." http://dx.doi.org/10.21271/zjpas

Aung, M. S., Kaltwang, S., Romera-Paredes, B., Martinez, B., Singh, A., Cella, M., ... & Bianchi-Berthouze, N. (2015). The automatic detection of chronic pain-related expression: requirements, challenges and the multimodal EmoPain dataset. IEEE transactions on affective computing, 7(4), 435-451. 10.1109/TAFFC.2015.2462830

Bargshady, G., Soar, J., Zhou, X., Deo, R., Whittaker, F., and Wang, H. "A joint deep neural network model for pain recognition from face." 2019 IEEE 4th International Conference on Computer and Communication Systems (ICCCS). IEEE, 2019. 10.1109/CCOMS.2019.8821779

Barua, P. D., Baygin, N., Dogan, S., Baygin, M., Arunkumar, N., Fujita, H., Tuncer, T., Tan, R., Palmer, E., Azizan, M. M. B., Kadri, N. B., Acharya, U. R. "Automated detection of pain levels using deep feature extraction from shutter blinds-based dynamic-sized horizontal patches with facial images." Scientific Reports 12.1 (2022): 17297. 10.1038/s41598-022-21380-4

Bellantonio, M., Haque, M. A., Rodriguez, P., Nasorollahi, K., Telve, T., Escalera, S., Gonzalez, J., Moeslund, T. B., Rasti, P., and Anbarjafari, G. "Spatio-temporal pain recognition in cnn-based super-resolved facial images." Video Analytics. Face and Facial Expression Recognition and Audience Measurement: Third International Workshop, VAAM 2016, and Second International Workshop, FFER 2016, Cancun, Mexico, December 4, 2016, Revised Selected Papers 2. Springer International Publishing, 2017.

Breivik, H., Borchgrevink, P. C., Allen, S. M., Rosseland, L. A., Romundstad, L., Breivik Hals, E. K., ... & Stubhaug, A. (2008). Assessment of pain. BJA: British Journal of Anaesthesia, 101(1), 17-24. https://doi.org/10.1093/bja/aen103

Chen, J., Chi, Zheru., and Fu, H.. "A new apprach for pain event detection in video." 2015 International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 2015. https://doi.org/10.1109/ACII.2015.7344579

Chen, Z., Ansari, R., and Wilkie, D. J. "Learning pain from action unit combinations: a weakly supervised approach via multiple instance learning." IEEE transactions on affective computing 13.1 (2019): 135-146. 10.1109/taffc.2019.2949314

Csurka, G. (2017). Domain adaptation for visual applications: A comprehensive survey. arXiv preprint arXiv:1702.05374.

https://doi.org/10.48550/arXiv.1702.05374

Darwin, C., and Prodger, P. (1998). The expression of the emotions in man and animals. Oxford University Press, USA. https://doi.org/10.1037/10001-000

Dehshibi, M. M., Olugbade, T., Diaz de Maria, F., Berthouze, N., and Jimenez, Ana. "Pain level and pain-related behaviour classification using GRU-based sparsely-connected RNNs." IEEE Journal of Selected Topics in Signal Processing (2023). https://doi.org/10.1109/jstsp.2023.3262358

Ekman, P., & Friesen, W. V. (1978). Facial action coding system. Environmental Psychology & Nonverbal Behavior. https://doi.org/10.1037/t27734-000

Elgendy, F., Alshewimy, M., and Sarhan, A. M. "Pain detection/classification framework including face recognition based on the analysis of facial expressions for E-health systems." Int. Arab J. Inf. Technol. 18.1 (2021): 125-132.

Fordyce W. E. (1976). Behavioral methods for chronic pain and illness. St. Louis, MO; Mosby.

Hadelina, R., Rusydi, M. I., Firza, M., Samuel, O. W. "ANN Models for Shoulder Pain Detection based on Human Facial Expression Covered by Mask." JITCE (Journal of Information Technology and Computer Engineering) 7.01 (2023): 49-55

Huang, D., Xia, Z., Li, L., and Ma, Y. "Pain estimation with integrating global‐wise and region‐wise convolutional networks." IET Image Processing 17.3 (2023): 637-648 https://doi.org/10.1049/ipr2.12639

Huang, D., Xia, Z., Mwesigye, J., and Feng, X. "Pain-attentive network: a deep spatio-temporal attention model for pain estimation." Multimedia Tools and Applications 79 (2020): 28329-28354. https://doi.org/10.1007/s11042-020-09397-1

Huang, Y., Qing, L., Xu, S., Wang, L., Peng, Y. "HybNet: a hybrid network structure for pain intensity estimation." The Visual Computer (2022): 1-12. https://doi.org/10.1007/s00371-021-02056-y

Karamitsos, I., Seladji, I., and Modak, S. "A modified CNN network for automatic pain identification using facial expressions." Journal of Software Engineering and Applications 14.8 (2021): 400-417. 10.4236/jsea.2021.148024

Kharghanian, R., Peiravi, A., and Moradi, F. "Pain detection from facial images using unsupervised feature learning approach." 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2016. 10.1109/EMBC.2016.7590729

Li, Y., Ghosh, S., and Joshi, J. "PLAAN: Pain Level Assessment with Anomaly-detection based Network." Journal on Multimodal User Interfaces (2021): 1-14. https://doi.org/10.1007/s12193-020-00362-8

Lopez-Martinez, D., and Picard, R. "Multi-task neural networks for personalized pain recognition from physiological signals." 2017 Seventh International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW). IEEE, 2017. 10.1109/ACIIW.2017.8272611

Lucey, P., Cohn, J. F., Prkachin, K. M., Solomon, P. E., & Matthews, I. (2011, March). Painful data: The UNBC-McMaster shoulder pain expression archive database. In 2011 IEEE International Conference on Automatic Face & Gesture Recognition (FG) (pp. 57-64). IEEE. https://doi.org/10.1016/j.imavis.2009.05.007

Lucey, P., Cohn, J., Lucey, S., Matthews, I., Sridharan, S., and Prkachin, K. M. (2009, September). Automatically detecting pain using facial actions. In 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops (pp. 1-8). IEEE.

Mallol-Ragolta, A., Liu, S., Cummins, N., Schuller, B. "A curriculum learning approach for pain intensity recognition from facial expressions." 2020 15th IEEE international conference on automatic face and gesture recognition (FG 2020). IEEE, 2020. 10.1109/FG47880.2020.00083

Mavadati, S. M., Mahoor, M. H., Bartlett, K., Trinh, P., & Cohn, J. F. (2013). Disfa: A spontaneous facial action intensity database. IEEE Transactions on Affective Computing, 4(2), 151-160. 10.1109/T-AFFC.2013.4

Meawad, F., Yang, S., and Loy, F. L. "Automatic detection of pain from spontaneous facial expressions." Proceedings of the 19th ACM International Conference on Multimodal Interaction. 2017. 10.1145/3136755.3136794

Niu, Z., Zhong, G., & Yu, H. (2021). A review on the attention mechanism of deep learning. Neurocomputing, 452, 48-62. https://doi.org/10.1016/j.neucom.2021.03.091

Othman, E., Werner, P., Saxen, F., Al-Hamadi, A., Walter, S. "Cross-database evaluation of pain recognition from facial video." 2019 11th International Symposium on Image and Signal Processing and Analysis (ISPA). IEEE, 2019 10.1109/ISPA.2019.8868562

Peng, X., Huang, D., and Zhang, H. "Pain intensity recognition via multi‐scale deep network." IET Image Processing 14.8 (2020): 1645-1652 https://doi.org/10.1049/iet-ipr.2019.1448

Pouromran, F., Radhakrishnan, S., and Kamarthi, S. "Exploration of physiological sensors, features, and machine learning models for pain intensity estimation." Plos one 16.7 (2021): e0254108. https://doi.org/10.1371/journal.pone.0254108

Prkachin, K. M. (1992). The consistency of facial expressions of pain: a comparison across modalities. Pain, 51(3), 297-306. 10.1016/0304-3959(92)90213-U

Rathee, N., and Ganotra, D. "Multiview distance metric learning on facial feature descriptors for automatic pain intensity detection." Computer Vision and Image Understanding 147 (2016): 77-86 https://doi.org/10.1016/j.cviu.2015.12.004

Rodriguez, P., Cucurull, G., Gonzalez, J., Gonfaus, J. M., Nasrollahi, K., Moeslund, T. B., Roca, F. X. "Deep pain: Exploiting long short-term memory networks for facial expression classification." IEEE transactions on cybernetics 52.5 (2017): 3314-3324. 10.1109/TCYB.2017.2662199

Semwal, A., and Londhe, N.D. "Automated Facial Expression based Pain Assessment Using Deep Convolutional Neural Network." 2020 3rd International Conference on Intelligent Sustainable Systems (ICISS). IEEE, 2020. 10.1109/ICISS49785.2020.9316099

Singh, S. K., Tiwari, S., Abidi, A. I., Singh, A. "Prediction of pain intensity using multimedia data." Multimedia Tools and Applications 76 (2017): 19317-19342. https://doi.org/10.1007/s11042-017-4718-6

Subramaniam, S. D., and Dass, B. "Automated nociceptive pain assessment using physiological signals and a hybrid deep learning network." IEEE Sensors Journal 21.3 (2020): 3335-3343. 10.1109/JSEN.2020.3023656

Tavakolian, M., Lopez, M. D., and Liu, L. "Self-supervised pain intensity estimation from facial videos via statistical spatiotemporal distillation." Pattern Recognition Letters 140 (2020): 26-33. https://doi.org/10.1016/j.patrec.2020.09.012

Walter, S., Gruss, S., Ehleiter, H., Tan, J., Traue, H. C., Werner, P., ... & da Silva, G. M. (2013, June). The biovid heat pain database data for the advancement and systematic validation of an automated pain recognition system. In 2013 IEEE international conference on cybernetics (CYBCO) (pp. 128-131). IEEE. 10.1109/CYBConf.2013.6617456

Warden, V., Hurley, A. C., & Volicer, L. (2003). Development and psychometric evaluation of the Pain Assessment in Advanced Dementia (PAINAD) scale. Journal of the American Medical Directors Association, 4(1), 9-15. https://doi.org/10.1097/01.JAM.0000043422.31640.F7

Werner, P., Al-Hamadi, A., Gruss, S., & Walter, S. (2019, September). Twofold-multimodal pain recognition with the X-ITE pain database. In 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) (pp. 290-296). IEEE. 10.1109/ACIIW.2019.8925061

Werner, P., Al-Hamadi, A., Limbrecht-Ecklundt, K., Walter, S., Gruss, S., Traue, H. C. "Automatic pain assessment with facial activity descriptors." IEEE Transactions on Affective Computing 8.3 (2016): 286-299. https://doi.org/10.1109/TAFFC.2016.2537327

Xin, X., Lin, X., Yang, S., Zheng, Xin. "Pain intensity estimation based on a spatial transformation and attention CNN." Plos one 15.8 (2020): e0232412. 10.1371/journal.pone.0232412

Xu, X., and de Sa, V. R. "Personalized Pain Detection in Facial Video with Uncertainty Estimation." 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 2021. 10.1109/EMBC46164.2021.9631056

Zhou, J., Liu, K., Xu, L, Lai, S. 2016. "Recurrent convolutional neural network regression for continuous pain intensity estimation in video." Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 10.1609/aaai.v29i1.9513