A Deep Learning/Machine Learning Approach for Anomaly Based Network Intrusion Detection

Samia Dardouri^*Reem Almuhanna

• Department of Computer Science, College of Computing and Information Technology, Shaqra University, Saudi Arabia, shaqra, Saudi Arabia

The increasing complexity and frequency of cybersecurity threats demand advanced detection systems capable of identifying both known and novel attacks. This study presents a hybrid anomaly-based Network Intrusion Detection System (NIDS) that integrates multiple machine learning and deep learning techniques, including XGBoost, Random Forest, Graph Neural Networks (GNN), Long Short-Term Memory (LSTM), and Autoencoders. Trained on a largescale dataset of over 5.6 million network traffic records, the proposed system employs comprehensive preprocessing, feature engineering, and the Synthetic Minority Over-sampling Technique (SMOTE) to address class imbalance. Leveraging a weighted soft-voting ensemble strategy, the model combines individual predictions to improve robustness and generalization. Experimental results demonstrate near-perfect performance with accuracy, precision, recall, and F1-score approaching 100% on the primary dataset, supported by rigorous 5-fold crossvalidation. Furthermore, evaluation on an additional benchmark dataset confirms the model's strong generalizability and robustness across diverse intrusion scenarios. These findings highlight the effectiveness of the proposed hybrid ensemble framework in enhancing intrusion detection capabilities in complex network environments.