Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DeepCodeSeek: Real-Time API Retrieval for Context-Aware Code Generation

Esakkivel Esakkiraja, Denis Akhiyarov, Aditya Shanmugham, Chitra Ganapathy

TL;DR

DeepCodeSeek addresses real-time API retrieval for context-aware code generation in enterprise environments by building a multi-stage retrieval pipeline that combines a knowledge graph for search-space pruning, JSDoc-based enriched indexing, and LLM-driven code expansion, followed by a post-training regimen for compact rerankers. It achieves 87.86% top-40 retrieval accuracy on clear-intent samples, outperforming BM25 and establishing strong downstream performance. A compact 0.6B reranker, trained with supervised fine-tuning and reinforcement learning, matches or exceeds an 8B baseline while delivering 2.5x lower latency, enabling production-ready real-time code completion. The work demonstrates that domain-specific, data-efficient retrievers can deliver high-quality context for enterprise code generation, with practical impact on Script Includes usage and AI-assisted Build Agent tasks.

Abstract

Current search techniques are limited to standard RAG query-document applications. In this paper, we propose a novel technique to expand the code and index for predicting the required APIs, directly enabling high-quality, end-to-end code generation for auto-completion and agentic AI applications. We address the problem of API leaks in current code-to-code benchmark datasets by introducing a new dataset built from real-world ServiceNow Script Includes that capture the challenge of unclear API usage intent in the code. Our evaluation metrics show that this method achieves 87.86% top-40 retrieval accuracy, allowing the critical context with APIs needed for successful downstream code generation. To enable real-time predictions, we develop a comprehensive post-training pipeline that optimizes a compact 0.6B reranker through synthetic dataset generation, supervised fine-tuning, and reinforcement learning. This approach enables our compact reranker to outperform a much larger 8B model while maintaining 2.5x reduced latency, effectively addressing the nuances of enterprise-specific code without the computational overhead of larger models.

DeepCodeSeek: Real-Time API Retrieval for Context-Aware Code Generation

TL;DR

DeepCodeSeek addresses real-time API retrieval for context-aware code generation in enterprise environments by building a multi-stage retrieval pipeline that combines a knowledge graph for search-space pruning, JSDoc-based enriched indexing, and LLM-driven code expansion, followed by a post-training regimen for compact rerankers. It achieves 87.86% top-40 retrieval accuracy on clear-intent samples, outperforming BM25 and establishing strong downstream performance. A compact 0.6B reranker, trained with supervised fine-tuning and reinforcement learning, matches or exceeds an 8B baseline while delivering 2.5x lower latency, enabling production-ready real-time code completion. The work demonstrates that domain-specific, data-efficient retrievers can deliver high-quality context for enterprise code generation, with practical impact on Script Includes usage and AI-assisted Build Agent tasks.

Abstract

Current search techniques are limited to standard RAG query-document applications. In this paper, we propose a novel technique to expand the code and index for predicting the required APIs, directly enabling high-quality, end-to-end code generation for auto-completion and agentic AI applications. We address the problem of API leaks in current code-to-code benchmark datasets by introducing a new dataset built from real-world ServiceNow Script Includes that capture the challenge of unclear API usage intent in the code. Our evaluation metrics show that this method achieves 87.86% top-40 retrieval accuracy, allowing the critical context with APIs needed for successful downstream code generation. To enable real-time predictions, we develop a comprehensive post-training pipeline that optimizes a compact 0.6B reranker through synthetic dataset generation, supervised fine-tuning, and reinforcement learning. This approach enables our compact reranker to outperform a much larger 8B model while maintaining 2.5x reduced latency, effectively addressing the nuances of enterprise-specific code without the computational overhead of larger models.

Paper Structure

This paper contains 65 sections, 13 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Neural Code Retrieval
  4. RAG with Filtering and Query Enhancement
  5. Reranking
  6. Proposed Method
  7. Dataset and Index Construction
  8. Dataset Construction
  9. Index Construction
  10. Knowledge Graph Construction
  11. Experimental Setup
  12. Evaluation Metrics
  13. Baselines and Implementation
  14. Pretraining Knowledge Check
  15. Experiments and Results
  16. ...and 50 more sections

Figures (13)

  • Figure 1: Overview of the proposed multi-stage retrieval pipeline.
  • Figure 2: Task anatomy: highlighting how code_before, and code_after are used to recover the ground-truth Script Include required for code_middle.
  • Figure 3: Qwen 0.6B: Top-5 accuracy after finetuning, Mean Reciprocal Rank (MRR) and Inference latency
  • Figure 4: A4 code trimming and context length
  • Figure 5: Performance of Code Context Ablation
  • ...and 8 more figures