PromptPrism: A Linguistically-Inspired Taxonomy for Prompts

TL;DR

The paper presents PromptPrism, a linguistically-inspired taxonomy for prompts to enable systematic analysis of LLM prompting. It formalizes prompts with three hierarchical levels and a formal definition P = {(r_i,c_i)}. The authors validate the framework through three applications—taxonomy-guided refinement, dataset profiling, and prompt sensitivity analysis—demonstrating improved model performance and richer prompt characterization across tasks and models. Overall, PromptPrism provides a foundation for refining, profiling, and analyzing prompts, with potential to standardize prompt engineering across diverse LLM applications.

Abstract

Prompts are the interface for eliciting the capabilities of large language models (LLMs). Understanding their structure and components is critical for analyzing LLM behavior and optimizing performance. However, the field lacks a comprehensive framework for systematic prompt analysis and understanding. We introduce PromptPrism, a linguistically-inspired taxonomy that enables prompt analysis across three hierarchical levels: functional structure, semantic component, and syntactic pattern. We show the practical utility of PromptPrism by applying it to three applications: (1) a taxonomy-guided prompt refinement approach that automatically improves prompt quality and enhances model performance across a range of tasks; (2) a multi-dimensional dataset profiling method that extracts and aggregates structural, semantic, and syntactic characteristics from prompt datasets, enabling comprehensive analysis of prompt distributions and patterns; (3) a controlled experimental framework for prompt sensitivity analysis by quantifying the impact of semantic reordering and delimiter modifications on LLM performance. Our experimental results validate the effectiveness of our taxonomy across these applications, demonstrating that PromptPrism provides a foundation for refining, profiling, and analyzing prompts.

Figures (8)

• Figure 1: PromptPrism Framework. The PromptPrism taxonomy decomposes an example prompt from Super Natural Instruction dataset (Left) into three hierarchical levels (Right). (1) Structural, corresponding to specified roles in the prompt (e.g. System, User, Assistant) ; (2) Semantic, detailing the content's semantic components (e.g. Instruction, Output Constraints); and (3) Syntactic, describing the syntactic structure of each semantic element (e.g. Delimiters, Special tokens).
• Figure 2: Applications of PromptPrism
• Figure 3: Performance comparison of taxonomy-guided prompt refinement across task types. Our taxonomy-guided approach demonstrates improved performance on text generation tasks (average 29% improvement than CoT) and matches or exceeds CoT baseline on classification tasks (average 0.13% improvement than CoT) . Complete performance metrics are presented in Table \ref{['tab:prompt-results-detail']}.
• Figure 4: Dataset Profile of apigen-80kliu2024apigen and smol-magpie-ultraallal2025smollm2 on four primary dimensions: (1) Structural Level (2) Semantic Level (3) Syntactic level and (4) Metadata.
• Figure 5: Syntactic Components Implementation