Beyond Performance Metrics: Demonstrating Generative AI's Unique Value Propositions in Cybersecurity Threat Detection through Hybrid Pipeline Integration

Abstract

This study evaluates the use of generative artificial intelligence (GenAI) in cybersecurity threat detection to examine its benefits in a workflow where people must make decisions. The experiments used the BODMAS dataset (134,435 samples) and a smaller exploratory subset from UNSW-NB15. State-of-the-art machine learning (ML) classifiers were compared with a zero-shot large language model (LLM). On standard classification metrics, ML consistently outperformed the LLM-based systems. This comparison is not the main contribution but establishes a performance baseline. The relevant point is how the LLM-based system behaves when the case is unclear. Although it is not reliable as a primary detector in this setting, it can support the analyst-side work that follows detection. In particular, it can generate short explanations in plain language, organize context around an alert, and provide an initial interpretation when the instance does not match the learned classes. This is valuable because it affects the triage speed and quality of escalation. Therefore, a hybrid pipeline is proposed. ML is used by default for high-confidence and time-sensitive decisions. The LLM is only called when the confidence is low or when an explanation is needed for review. Latency and cost are treated as deployment constraints, and the hallucination risk is handled as a reliability limitation that requires human oversight. Overall, GenAI is unlikely to replace ML-based detection methods. Its contribution is better framed as interpretive support for ambiguous or unfamiliar alerts.