Prompt Less, Smile More: MTP with Semantic Engineering in Lieu of Prompt Engineering

TL;DR

The paper tackles the challenge of capturing developer intent in AI-Integrated software where LLMs rely on prompts. It introduces Semantic Engineering and SemTexts to enrich MTP’s semantics, bridging the gap with Prompt Engineering while reducing developer effort. By extending the Jac/MTP stack with SemTable and MT-IR*, and by evaluating on a diverse benchmark suite, SemTexts achieve comparable or better accuracy than Prompt Engineering in many tasks and outperform traditional docstrings due to tighter spatial alignment of semantics with code entities. The work demonstrates that lightweight semantic annotations can largely close the performance gap with manual prompts, enabling scalable and intent-aligned AI-assisted software development.

Abstract

AI-Integrated programming is emerging as a foundational paradigm for building intelligent systems with large language models (LLMs). Recent approaches such as Meaning Typed Programming (MTP) automate prompt generation by leveraging the semantics already present in code. However, many real-world applications depend on contextual cues, developer intent, and domain-specific reasoning that extend beyond what static code semantics alone can express. To address this limitation, we introduce Semantic Engineering, a lightweight method for enriching program semantics so that LLM-based systems can more accurately reflect developer intent without requiring full manual prompt design. We present Semantic Context Annotations (SemTexts), a language-level mechanism that allows developers to embed natural-language context directly into program constructs. Integrated into the Jac programming language, Semantic Engineering extends MTP to incorporate these enriched semantics during prompt generation. We further introduce a benchmark suite designed to reflect realistic AI-Integrated application scenarios. Our evaluation shows that Semantic Engineering substantially improves prompt fidelity, achieving performance comparable to Prompt Engineering while requiring significantly less developer effort.

Figures (9)

• Figure 1: The left side illustrates three ways to implement the same functionality using LLMs; Prompt Engineering, Meaning-Typed Programming (MTP), and Semantic Engineering. The plot on the right shows how these approaches differ in developer effort and performance: Prompt Engineering delivers high performance but requires heavy manual effort; MTP reduces effort through automated prompt generation but loses performance when developer intent is missing; Semantic Engineering restores missing intent and elevate MTP performance closer to Prompt Engineering with far less effort.
• Figure 2: Data structures for Plan and RepoState, and the generate_plan function that should returns a list of Plan steps.
• Figure 3: Prompt Engineering implementation showing the full concatenated prompt used by the generate_plan function.
• Figure 4: MTP implementation of the generate_plan function and the automatic prompt generation derived from its semantic structure.
• Figure 5: End-to-end compilation pipeline and runtime system for performing Semantic Engineering using SemTexts in MTP.