Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets

TL;DR

The paper introduces Magentic Marketplace, an open-source environment for studying two-sided agentic markets in which Assistant agents represent consumers and Service agents represent businesses, enabling end-to-end lifecycle experimentation from search to transaction. It implements a three-endpoint protocol and a rich action space to support discovery, negotiation, and payments, and evaluates frontier and open-source LLMs on synthetic restaurant and contractor data under lexical and ideal discovery. Key findings show that frontier models can approach welfare under ideal search but face degradation with larger consideration sets and exhibit strong first-proposal bias, while manipulation resistance varies across models, underscoring the need for robust market designs and human-in-the-loop safeguards. The work provides a practical, extensible platform for safely probing emergent market dynamics and guiding the design of fair, efficient agentic marketplaces with real-world relevance.

Abstract

As LLM agents advance, they are increasingly mediating economic decisions, ranging from product discovery to transactions, on behalf of users. Such applications promise benefits but also raise many questions about agent accountability and value for users. Addressing these questions requires understanding how agents behave in realistic market conditions. However, previous research has largely evaluated agents in constrained settings, such as single-task marketplaces (e.g., negotiation) or structured two-agent interactions. Real-world markets are fundamentally different: they require agents to handle diverse economic activities and coordinate within large, dynamic ecosystems where multiple agents with opaque behaviors may engage in open-ended dialogues. To bridge this gap, we investigate two-sided agentic marketplaces where Assistant agents represent consumers and Service agents represent competing businesses. To study these interactions safely, we develop Magentic-Marketplace -- a simulated environment where Assistants and Services can operate. This environment enables us to study key market dynamics: the utility agents achieve, behavioral biases, vulnerability to manipulation, and how search mechanisms shape market outcomes. Our experiments show that frontier models can approach optimal welfare -- but only under ideal search conditions. Performance degrades sharply with scale, and all models exhibit severe first-proposal bias, creating 10-30x advantages for response speed over quality. These findings reveal how behaviors emerge across market conditions, informing the design of fair and efficient agentic marketplaces.

Figures (9)

• Figure 1: Magentic Marketplace is an open-source environment where AI agents can discover, communicate, and transact with each other. The environment can be used for evaluating different market designs and agent behaviors.
• Figure 2: Overview of Magentic Marketplace's architecture: agents, endpoints, and action space.
• Figure 3: Detailed overview of the Magentic Marketplace. It comprises two types of agents: Assistant Agents (left) acting on behalf of customers, and Service Agents (right) acting on behalf of businesses. Both agent types interact with a central Market Environment through REST API endpoints, supporting agent registration (POST /register), service discovery (Catalog and Search), inter-agent communication and negotiation (Communication), and transaction execution (Transaction). Action Routers on both sides manage the flow of messages (send/receive) and protocol requests (GET /protocol, POST /action), enabling autonomous negotiation and commerce in a two-sided marketplace setting.
• Figure 4: Total consumer welfare achieved in various instantiations of the marketplace. Left shows agentic markets run with different LLMs using both a more realistic lexical search (blue) and a perfect discovery layer (yellow) that always returns ideal matches. The right shows three baselines for comparison, where each has access to a different subset of information and uses different decision criteria as described in Table \ref{['tab:sanity-check-conditions']}. The dashed horizontal line represents the optimal total consumer welfare that can be achieved in the marketplace. For each sub-figure, the models in the left are sorted by their welfare when they use perfect search.
• Figure 5: Experiments with consideration set size revealed a paradox of choice effect where surprisingly increased options (from search results) reduced welfare.