The Road of Adaptive AI for Precision in Cybersecurity

TL;DR

The paper addresses the need for precision in cybersecurity amid rapidly evolving knowledge and noisy data. It advocates a dual strategy of inference-time adaptation (ICL, RAG, knowledge graphs) and model-level continual learning (unsupervised pretraining/fine-tuning, LoRA) to build robust GenAI pipelines. Practical guidance is drawn from production deployments, detailing how to pair retrieval strategies with domain knowledge and how to maintain currency through continual learning and task-specialized models. The work highlights open research directions in robustness, auditability, and secure deployment to advance practical, auditable AI for cyber defense.

Abstract

Cybersecurity's evolving complexity presents unique challenges and opportunities for AI research and practice. This paper shares key lessons and insights from designing, building, and operating production-grade GenAI pipelines in cybersecurity, with a focus on the continual adaptation required to keep pace with ever-shifting knowledge bases, tooling, and threats. Our goal is to provide an actionable perspective for AI practitioners and industry stakeholders navigating the frontier of GenAI for cybersecurity, with particular attention to how different adaptation mechanisms complement each other in end-to-end systems. We present practical guidance derived from real-world deployments, propose best practices for leveraging retrieval- and model-level adaptation, and highlight open research directions for making GenAI more robust, precise, and auditable in cyber defense.

Figures (4)

• Figure 1: AI for Precision in Cybersecurity: An overview of the challenges and adaptation mechanisms discussed in this paper. On the left, we show core obstacles—such as evolving knowledge bases, conflicting and noisy assessments, countless alerts, and sparse-signal contexts—that make precision in cybersecurity both essential and difficult. On the right are the main approaches for continual adaptation: inference-time adaptation (with in-context learning, retrieval-augmented generation, and knowledge graphs), and model-level adaptation (through continual learning and fine-tuning). Together, these orthogonal levers enable GenAI systems to transform massive, fragmented cybersecurity data into precise, context-aware actions in this never-ending, rapidly changing domain.
• Figure 2: Illustration of a typical data pipeline for cybersecurity sketched for this paper’s context. The process begins at the lower left, where an information retrieval agent gathers cyber entities and assets from internal sources and the web, consolidating them into an enriched database of entities. High-quality, manually-curated “gold standard” annotations are provided by human experts, while automated agents generate broader “silver standard” annotations using distant supervision techniques. Reasoning agents further augment these annotations with explanatory rationales to enhance interpretability. The resulting set of annotated and explained entities is transformed into machine-usable vector representations by a vector embedding agent and stored in a vector database. This integrated approach balances expert knowledge with scalable automation and reasoning, supporting the pipeline’s ability to adapt and maintain precision in the complex, evolving cybersecurity landscape.
• Figure 3: This figure illustrates how a GenAI pipeline in cybersecurity can bring together structured knowledge graphs—which map relationships among entities like CVEs, packages, and attack techniques—and expert-annotated examples to tune semantic embedding models (LLMs, potentially with GCNs). These embedding models, enriched with both human insight and graph structure, are then used alongside real-world context and public knowledge bases to support robust Retrieval-Augmented Generation (RAG). Agentic RAG, leveraging LLM agents, can further improve precision and context selection in the face of noisy, conflicting, or fast-changing security information.
• Figure 4: This figure summarizes the practical guidance on adapting large language models (LLMs) for real-world cybersecurity. The left side highlights scenarios when to continual learn LLMs: dealing with very large or messy data sources, filtering out noise or misinformation, supporting (implicit or explicit) advanced reasoning, resolving conflicting or inconsistent annotations, maintaining consistent answers across many queries, and keeping pace with fast-changing knowledge bases. The right side presents best practices for continual learning we suggest from our experience: using unsupervised learning (same objective as used for pretraining of LLMs) to absorb new information efficiently, adopting lightweight fine-tuning methods such as LoRA, tailoring learning for specific tasks instead of relying on a single general-purpose model, and focusing on training LLMs to understand the reasoning behind expert annotations—not just the labels themselves.