Improving LLM Reasoning via Dependency-Aware Query Decomposition and Logic-Parallel Content Expansion

TL;DR

The paper tackles the latency-quality trade-off in real-time Web LLM reasoning by introducing Orion, a two-phase framework that decomposes a complex query into key point generation and DAG-guided content parallel expansion. Key point generation uses retrieval-augmented few-shot prompting and a vector database to produce logically structured key points and dependencies; content expansion then concurrently elaborates on these points guided by a dependency DAG, with context-aware caching to maintain global coherence. A cross-query scheduling mechanism, Pipsch, overlaps the compute-intensive and memory-intensive phases across queries to exploit heterogeneous GPU resources, enabling significant throughput and quality gains. Experiments on Vicuna and WizardLM with LLaMA2 and Qwen models show up to 4.33x token speedup, up to 3.42x latency reduction, and up to 18.75% improvement in reasoning quality, validating Orion's effectiveness for real-time Web applications. Overall, Orion advances practical LLM reasoning by combining structured query decomposition, DAG-guided parallel expansion, and cross-query scheduling to achieve efficiency at quality on real-world workloads.

Abstract

The integration of Large Language Models (LLMs) into real-time Web applications, such as AI-powered search and conversational agents, presents a fundamental Web infrastructure challenge: reconciling the demand for high-quality, complex reasoning with the stringent low-latency and high-throughput requirements of interactive services. Current LLM reasoning, hindered by computationally inefficient sequential generation and rigid reasoning strategies, creates a critical bottleneck for the Web services. Existing approaches typically optimize the LLM reasoning for either efficiency or quality but struggle to achieve both, and thus fail to meet the dual requirements of modern Web platforms. To overcome these limitations, we propose Orion, a novel and efficient reasoning framework that enables dependency-aware query decomposition and logic-parallel content expansion. Concretely, Orion decomposes a single query reasoning process into two synergistic phases: (1) \textit{key point generation}, which distills logically structured key points through retrieval-augmented few-shot prompting, and (2) \textit{content parallel expansion}, which concurrently elaborates on these points based on a dependency graph to ensure logical consistency. Furthermore, Orion introduces a pipeline scheduling mechanism that exploits the complementary computational characteristics of the two phases (generation imposes pressure on GPU computing and expansion stresses on GPU memory) across multiple queries, enabling cross-query parallelism and dramatically improving reasoning performance (\ie, efficiency and quality). Experiments on diverse benchmarks show that Orion not only delivers up to 4.33x higher token generation speed and 3.42x lower answer latency over the baselines but also improves reasoning quality by up to 18.75% through explicitly modeling inter-point dependencies.

Figures (9)

• Figure 1: (a) Orion generates four key points (i.e., build, solve x, solve y and check) for the query. Points $2$ and $3$, requiring Point $1$ as context, are independent of each other, and Point $4$ depends on the output of Points $2$ and $3$. (b) The pipeline execution between key point generation and content parallel expansion is driven by phase-specific computational traits.
• Figure 2: The reasoning efficiency and quality of different methods on three datasets.
• Figure 3: (a) Generation (stressing GPU computing) and extension (stressing GPU memory) phases for different queries can execute in parallel. (b) Shapes denote methods and colors represent reasoning performance across datasets.
• Figure 4: The workflow of key point generation and content parallel expansion for a single query
• Figure 5: Workflow expression and execution comparison of existing reasoning strategy and Orion. (a) Query-level workflow in current reasoning strategy. (b) Decomposition-based dataflow graph in Orion (Searching and Prefilling are omitted due to space constraints). (c) Execution graph after multi-query scheduling optimization in Orion.