KryptoPilot: An Open-World Knowledge-Augmented LLM Agent for Automated Cryptographic Exploitation

TL;DR

KryptoPilot addresses the knowledge granularity bottleneck in LLM-based cryptographic exploitation by introducing open-world knowledge augmentation, a persistent workspace for structured memory, and a governance framework that stabilizes long-horizon reasoning. The Deep Research pipeline continuously retrieves task-relevant primary sources, while a Model Router allocates computational resources across subtasks, preserving high-quality context through downgrades. Across InterCode-CTF, NYU-CTF, and six live competitions, KryptoPilot achieves a 100% solve rate on simpler crypto tasks, 56–60% on harder cryptographic challenges, and 26/33 solves in real-world events, with several earliest solves, confirming practical effectiveness and cross-domain transferability. The results demonstrate that precise, executable knowledge and disciplined governance are essential for scaling LLM-based agents to real-world cryptographic exploitation and beyond.

Abstract

Capture-the-Flag (CTF) competitions play a central role in modern cybersecurity as a platform for training practitioners and evaluating offensive and defensive techniques derived from real-world vulnerabilities. Despite recent advances in large language models (LLMs), existing LLM-based agents remain ineffective on high-difficulty cryptographic CTF challenges, which require precise cryptanalytic knowledge, stable long-horizon reasoning, and disciplined interaction with specialized toolchains. Through a systematic exploratory study, we show that insufficient knowledge granularity, rather than model reasoning capacity, is a primary factor limiting successful cryptographic exploitation: coarse or abstracted external knowledge often fails to support correct attack modeling and implementation. Motivated by this observation, we propose KryptoPilot, an open-world knowledge-augmented LLM agent for automated cryptographic exploitation. KryptoPilot integrates dynamic open-world knowledge acquisition via a Deep Research pipeline, a persistent workspace for structured knowledge reuse, and a governance subsystem that stabilizes reasoning through behavioral constraints and cost-aware model routing. This design enables precise knowledge alignment while maintaining efficient reasoning across heterogeneous subtasks. We evaluate KryptoPilot on two established CTF benchmarks and in six real-world CTF competitions. KryptoPilot achieves a complete solve rate on InterCode-CTF, solves between 56 and 60 percent of cryptographic challenges on the NYU-CTF benchmark, and successfully solves 26 out of 33 cryptographic challenges in live competitions, including multiple earliest-solved and uniquely-solved instances. These results demonstrate the necessity of open-world, fine-grained knowledge augmentation and governed reasoning for scaling LLM-based agents to real-world cryptographic exploitation.

Figures (4)

• Figure 1: Overview of the KryptoPilot architecture.
• Figure 2: The Deep Research (DR) pipeline for dynamic open-world knowledge acquisition.
• Figure 3: Overview of the representative cryptographic challenge and the corresponding solving workflow of KryptoPilot.
• Figure 4: Performance comparison on the InterCode-CTF dataset (RQ1 & RQ2).