Deep Multimodal Biomechanical Analysis for Lower Back Pain Rehabilitation to Improve Patients Stability

Abrar Ashraf¹Yanfeng Wu²Shaheryar Najam³Mohammed Alshehri⁴Yahya Alqahtani⁴Hanan Aljuaid⁵Ahmad Jalal^6*Hui Liu^7*

• ¹Riphah International University, Rawalpindi, Punjab, Pakistan
• ²Guodian Nanjing Automation Co., LTD, Nanjing, China
• ³Bahria University, Islamabad, Islamabad, Pakistan
• ⁴King Khalid University, Abha, Saudi Arabia
• ⁵Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia
• ⁶Air University, Islamabad, Pakistan
• ⁷University of Bremen, Bremen, Bremen, Germany

Advancements in artificial intelligence are transforming rehabilitation by enabling scalable, patient-centric solutions. This study presents 3D-PoseFormer, a deep multimodal framework specifically designed for the telerehabilitation of individuals with lower back pain (LBP). The system leverages synchronized RGB and depth video streams to perform real-time, markerless, and sensor-free analysis of physiotherapy exercises. From depth data, it extracts 3D body joint positions and generates SMPL-based mesh vertices to capture detailed biomechanical and postural information. Simultaneously, RGB frames are processed using keypoint detection techniques—including Shi-Tomasi, AKAZE, BRISK, SIFT, and Harris corner. These features are then enhanced via semantic contour analysis of segmented body parts to extract localized appearance-based features relevant to LBP therapy. These multimodal features are fused and passed to a Transformer network that models temporal motion patterns for accurate classification and performance assessment. The system removes the need for wearable sensors and complements clinician supervision by enabling autonomous and continuous monitoring in home settings as a supplementary tool for rehabilitation. Validation on the KIMORE dataset (baseline, including rehabilitation exercises by patients with lower back pain), mRI dataset (rehabilitation exercises), and UTKinect-Action3D dataset (comprising diverse subjects and activity scenarios) yielded state-of-the-art accuracies (94.73%, 91%, and 94.2%, respectively), demonstrating the framework's robustness, generalizability, and clinical potential in AI-assisted rehabilitation for musculoskeletal disorders.