Published on: July 2026
EXPLAINABLE MULTIMODAL INTELLIGENCE FOR COGNITIVE FATIGUE DETECTION USING SPEECH ACOUSTICS, FACIAL DYNAMICS, AND HUMAN INTERACTION BEHAVIOUR: A SYSTEMATIC REVIEW
Sarbjeet Kaur
Article Status
Available Documents
Abstract
Keywords: cognitive fatigue detection; multimodal fusion; explainable artificial intelligence; speech prosody; facial action units
How to Cite this Paper
Kaur, S. (2026). Explainable Multimodal Intelligence for Cognitive Fatigue Detection Using Speech Acoustics, Facial Dynamics, and Human Interaction Behaviour: A Systematic Review. International Journal of Creative and Open Research in Engineering and Management, <i>02</i>(7). https://doi.org/10.55041/ijcope.v2i6.395
Kaur, Sarbjeet. "Explainable Multimodal Intelligence for Cognitive Fatigue Detection Using Speech Acoustics, Facial Dynamics, and Human Interaction Behaviour: A Systematic Review." International Journal of Creative and Open Research in Engineering and Management, vol. 02, no. 7, 2026, pp. . doi:https://doi.org/10.55041/ijcope.v2i6.395.
Kaur, Sarbjeet. "Explainable Multimodal Intelligence for Cognitive Fatigue Detection Using Speech Acoustics, Facial Dynamics, and Human Interaction Behaviour: A Systematic Review." International Journal of Creative and Open Research in Engineering and Management 02, no. 7 (2026). https://doi.org/https://doi.org/10.55041/ijcope.v2i6.395.
References
[1] M. A. S. Boksem and M. Tops, "Mental fatigue: Costs and benefits," Brain Research Reviews, vol. 59, no. 1, pp. 125–139, 2008, doi: 10.1016/j.brainresrev.2008.07.001.[2] M. A. Al Imran, F. Nasirzadeh, and C. Karmakar, "Designing a practical fatigue detection system: A review on recent developments and challenges," Journal of Safety Research, vol. 90, pp. 100–114, 2024, doi: 10.1016/j.jsr.2024.07.002.
[3] A. B. Rafaih and K. Ari, "AI-driven approaches to managing surgeon fatigue and improving performance," Cureus, vol. 16, no. 12, Art. no. e75717, 2024, doi: 10.7759/cureus.75717.
[4] J. S. Virk, M. Singh, M. Singh, U. Panjwani, and K. Ray, "A multimodal feature fusion framework for sleep-deprived fatigue detection to prevent accidents," Sensors, vol. 23, no. 8, Art. no. 4129, 2023, doi: 10.3390/s23084129.
[5] M. Dias, F. Dörr, S. Garthof, S. Schäfer, J. Elmers, L. Schwed, and T. Ziemssen, "Detecting fatigue in multiple sclerosis through automatic speech analysis," Frontiers in Human Neuroscience, vol. 18, Art. no. 1449388, 2024, doi: 10.3389/fnhum.2024.1449388.
[6] E. Karim, H. R. Pavel, S. Nikanfar, A. Hebri, A. Roy, H. R. Nambiappan, and F. Makedon, "Cognitive fatigue assessment using physiological sensors during human-robot interaction for activities of daily living," Technologies, vol. 12, no. 3, Art. no. 38, 2024, doi: 10.3390/technologies12030038.
[7] F. Far Poor, H. H. Dodge, and M. H. Mahoor, "A multimodal cross-transformer-based model to predict mild cognitive impairment using speech, language, and vision," Computers in Biology and Medicine, vol. 182, Art. no. 109199, 2024, doi: 10.1016/j.compbiomed.2024.109199.
[8] M. Zhao, H. Gao, X. Qi, H. Yin, Y. Song, Y. Bai, and L. Zhao, "Multi-query cross-modal attention fusion for cognitive impairment recognition," IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 33, pp. 2520–2530, 2025, doi: 10.1109/TNSRE.2025.3583110.
[9] Y. Xie and B. Guo, "CogMamba: Multi-task driver cognitive load and physiological non-contact estimation with multimodal facial features," Sensors, vol. 25, no. 18, Art. no. 5620, 2025, doi: 10.3390/s25185620.
[10] Y. Sun, M. Zhou, F. Cai, X. Yang, C. Li, and Y. Yi, "Noncontact multimodal sensing for driver distraction detection via behavioral-physiological fusion," IEEE Sensors Journal, vol. 25, no. 20, pp. 38576–38584, 2025, doi: 10.1109/JSEN.2025.3608125.
Ethical Compliance & Review Process
- •All submissions are screened under plagiarism detection.
- •Review follows editorial policy.
- •Authors retain copyright.
- •Peer Review Type: Double-Blind Peer Review
- •Published on: Jul 05 2026
This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. You are free to share and adapt this work for non-commercial purposes with proper attribution.

