UAV-Based Intelligent Traffic Surveillance using Recurrent Neural Networks and Swin Transformer for Dynamic Environments

Nouf Abdullah Almujally¹Ting Wu²Haifa F Alhasson³Muhammad Hanzla⁴Ahmad Jalal^4*Hui Liu^5*

• ¹Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia
• ²Southeast University, Nanjing, China
• ³Qassim University, Buraydah, Saudi Arabia
• ⁴Air University, Islamabad, Pakistan
• ⁵University of Bremen, 28359, Bremen, Germany, Bremen, Germany

Abstract Introduction Urban traffic congestion, environmental degradation, and road safety challenges necessitate intelligent aerial robotic systems capable of real-time adaptive decision-making. Unmanned Aerial Vehicles (UAVs), with their flexible deployment and high vantage point, offer a promising solution for large-scale traffic surveillance in complex urban environments. This study introduces a UAV-based neural framework that addresses challenges such as asymmetric vehicle motion, scale variations, and spatial inconsistencies in aerial imagery. Methods The proposed system integrates a multi-stage pipeline encompassing contrast enhancement and region-based clustering to optimize segmentation while maintaining computational efficiency for resource-constrained UAV platforms. Vehicle detection is carried out using a Recurrent Neural Network (RNN), optimized via a hybrid loss function combining cross-entropy and mean squared error to improve localization and confidence estimation. Upon detection, the system branches into two neural submodules: (i) a classification stream utilizing SURF and BRISK descriptors integrated with a Swin Transformer backbone for precise vehicle categorization, and (ii) a multi-object tracking stream employing DeepSORT, which fuses motion and appearance features within an affinity matrix for robust trajectory association. Results Comprehensive evaluation on three benchmark UAV datasets—AU-AIR, UAVDT, and VAID shows consistent and high performance. The model achieved detection precisions of 0.913, 0.930, and 0.920; tracking precisions of 0.901, 0.881, and 0.890; and classification accuracies of 92.14%, 92.75%, and 91.25%, respectively. Discussion These findings highlight the adaptability, robustness, and real-time viability of the proposed architecture in aerial traffic surveillance applications. By effectively integrating detection, classification, and tracking within a unified neural framework, the system contributes significant advancements to intelligent UAV-based traffic monitoring and supports future developments in smart city mobility and decision-making systems.