A Unified XAI-LLM Approach for EndotrachealSuctioning Activity Recognition

TL;DR

This work tackles automated recognition and feedback for endotracheal suctioning training by introducing a unified LLM-centered HAR framework that fuses video data, SHAP-based attributions, and procedure-driven prompts. Using Gemini 2.5 Pro, it performs spatiotemporal reasoning, generates explanations, and delivers natural-language feedback to learners, yielding approximately 15–20% gains in accuracy and macro F1 over baselines. A pilot student-support module combines anomaly detection with explainable AI, enabling interpretable feedback that highlights correct actions and targeted improvements. The study discusses prompting strategies, LLM justification, and limitations (notably real-time capability and the lack of user studies), outlining pathways for scalable, interpretable, data-driven nursing education that can enhance training efficiency and patient safety.

Abstract

Endotracheal suctioning (ES) is an invasive yet essential clinical procedure that requires a high degree of skill to minimize patient risk - particularly in home care and educational settings, where consistent supervision may be limited. Despite its critical importance, automated recognition and feedback systems for ES training remain underexplored. To address this gap, this study proposes a unified, LLM-centered framework for video-based activity recognition benchmarked against conventional machine learning and deep learning approaches, and a pilot study on feedback generation. Within this framework, the Large Language Model (LLM) serves as the central reasoning module, performing both spatiotemporal activity recognition and explainable decision analysis from video data. Furthermore, the LLM is capable of verbalizing feedback in natural language, thereby translating complex technical insights into accessible, human-understandable guidance for trainees. Experimental results demonstrate that the proposed LLM-based approach outperforms baseline models, achieving an improvement of approximately 15-20\% in both accuracy and F1 score. Beyond recognition, the framework incorporates a pilot student-support module built upon anomaly detection and explainable AI (XAI) principles, which provides automated, interpretable feedback highlighting correct actions and suggesting targeted improvements. Collectively, these contributions establish a scalable, interpretable, and data-driven foundation for advancing nursing education, enhancing training efficiency, and ultimately improving patient safety.

Figures (6)

• Figure 1: The comparison between the traditional method (upper) and our LLM-based HAR method (lower). In our method, the LLM model works as a central model for interpretation and activity recognition, which moves beyond the traditional single-task model. This approach ensures transparency and a reliable recognition framework for nursing activity recognition.
• Figure 2: The diagram of the ES procedure and the average duration of each activity. The activity consists of 4 major steps: preparation, intervention, clean up, and post-procedure. In addition, in each video, two rounds of the activity were performed.
• Figure 3: The prompt breakdown for the zero-shot classification task of our study. In prompt A scenarios, the LLM was given a prompt with instructions, objectives, descriptions, output, video, and especially a procedure description, which prompt B does not have. This procedure description will be further analyzed for its effect on the LLM recognition framework in Subsection \ref{['LLM_in_HAR']}
• Figure 4: The confusion matrix of recognition in the two-prompt strategy.
• Figure 5: The PCA projection of the students' and the nurses' posture features. It is clear that nurses and students have performed certain movements that the other group did not in a specific activity (see red circle).