Real-Time Sleep Disorder Monitoring Design Using Dynamic Temporal Graphs with Facial and Acoustic Feature Fusion

Fei Pei¹Ying Zhou²Qiangqiang Fu³Hong Zhou^2*

• ¹University of Shanghai for Science and Technology, Shanghai, China
• ²Shanghai Shidong Hospital of Yangpu District, Shanghai, China
• ³Tongji University, Shanghai, China

Sleep disorders and related health complications pose significant risks to patient safety and wellbeing. This study presents a novel multimodal approach for identifying and predicting pathological conditions during sleep by analyzing facial expressions and sleep audio signals. We propose a dynamic graph modeling framework that captures the temporal evolution of these multimodal features over time t, enabling real-time monitoring and early warning of sleep-related medical events. Facial expression data is processed using deep convolutional neural networks to extract emotional and physiological indicators, while audio signals are analyzed for breathing patterns, sleep apnea events, and other acoustic biomarkers. These heterogeneous data streams are integrated into a unified dynamic graph representation where nodes represent different modalities and temporal states, and edges capture their interdependencies across time. The temporal dynamics are modeled using graph neural networks that can effectively learn complex spatiotemporal patterns indicative of various sleep pathologies. Our experimental validation demonstrates the system's capability to accurately detect and predict critical sleep events, including sleep apnea, restless leg syndrome, and cardiovascular irregularities during sleep. This multimodal framework addresses a critical gap in sleep medicine by providing continuous, non-invasive monitoring that maintains diagnostic accuracy comparable to polysomnography while eliminating patient discomfort and laboratory constraints. The system's rapid detection capabilities (average 10.7-second delay) and robust clinical agreement rates (94.6%) position it as a viable tool for both point-of-care screening and long-term home-based sleep disorder management, potentially expanding access to sleep medicine services for underserved populations.