A Multimodal Dangerous State Recognition and Early Warning System for Elderly with Intermittent Dementia

TL;DR

This work tackles the problem of elderly missing incidents among those with intermittent dementia by proposing a 5G-enabled wearable helmet that streams first-person images and precise location data to a cloud-based, scene-location multimodal risk-recognition network. The method combines a Map Generation Module, Unimodal Feature Extraction, Multimodal Feature Fusion, and Hazardous State Recognition to automatically assess danger levels without requiring elder input, enabling proactive caregiver alerts. Experiments on a 2930-sample dataset demonstrate that multimodal fusion with SAN-LSTM and SENet yields superior precision, recall, and accuracy compared to unimodal baselines and existing multimodal methods, with R50_S18 achieving around 0.73 precision, 0.72 recall, and 72.11% accuracy. Real-world testing confirms the system’s practical viability, offering real-time location visualization, first-person imagery, and four-level warnings that enhance elderly safety while reducing caregiver burden.

Abstract

In response to the social issue of the increasing number of elderly vulnerable groups going missing due to the aggravating aging population in China, our team has developed a wearable anti-loss device and intelligent early warning system for elderly individuals with intermittent dementia using artificial intelligence and IoT technology. This system comprises an anti-loss smart helmet, a cloud computing module, and an intelligent early warning application on the caregiver's mobile device. The smart helmet integrates a miniature camera module, a GPS module, and a 5G communication module to collect first-person images and location information of the elderly. Data is transmitted remotely via 5G, FTP, and TCP protocols. In the cloud computing module, our team has proposed for the first time a multimodal dangerous state recognition network based on scene and location information to accurately assess the risk of elderly individuals going missing. Finally, the application software interface designed for the caregiver's mobile device implements multi-level early warnings. The system developed by our team requires no operation or response from the elderly, achieving fully automatic environmental perception, risk assessment, and proactive alarming. This overcomes the limitations of traditional monitoring devices, which require active operation and response, thus avoiding the issue of the digital divide for the elderly. It effectively prevents accidental loss and potential dangers for elderly individuals with dementia.

Figures (9)

• Figure 1: Overall architecture and operation flowchart of the anti-loss and dangerous state recognition early warning system based on a smart helmet
• Figure 2: The structure diagram of the scene-location multimodal dangerous state recognition network.
• Figure 3: The structure diagram of the scene-location multimodal dangerous state recognition network.
• Figure 4: Sample Map Generated by the Pseudo Map Generator
• Figure 5: Assembly Effect of the Smart Helmet and Specific Details of the Intelligent Camera Module