Design of an AI-Driven Secure 5G-SDN Framework with Federated Reinforcement Learning for Anomaly Detection, Mitigation and Attack Forensics

Shameli RSujatha Rajkumar^*

• School of Electronics Engineering, Vellore Institute of Technology, Vellore, India

The increasing adoption of Software-Defined Networking (SDN) in 5G networks has revolutionized network management, moreover, introduced severe security vulnerabilities, including data plane anomalies, control layer intrusions, Denial-of-service (DDoS) attacks. Conventional intrusion detection approaches based on CNNs and LSTMs suffer from high computational overhead, increased detection time and limited scalability, rendering them inefficiently suited for real-time 5G-SDN environments. Despite this, the paper proposes a novel multi-layered security framework that integrates EfficientNet with Knowledge Distillation (KD), Transformer Networks, Spiking Neural Networks (SNN), Federated Reinforcement Learning (FRL), and Blockchain. EfficientNet-KD performs lightweight anomaly detection in the data plane layer with high accuracy, while Transformer capture long-range temporal dependencies for enhanced sequential control layer attack detection. For ultra-low-latency classification, SNNs replicate the processing of human brain networks in real time for recognizing attacks instantaneously. FRL supports decentralized, privacy-preserving mitigation across SDN controllers, which enhances scalability. Blockchain ensures the integrity and immutability of attack logs, thus ensuring forensic integrity. Experiments on CICIDS2017, UNSW-NB15, IoT-23, InSDN datasets demonstrated and achieved an average 97.75% accuracy, less than 5% throughput degradation, and 15 ms detection time; each detection requires just 0.25 joules, representing a 40% reduction in energy usage compared to existing CNN and LSTM approaches. The results demonstrate a scalable, low-latency, and high-accuracy security solution for 5G-SDN environments.