FREYR: A Framework for Recognizing and Executing Your Requests

TL;DR

FreYR introduces a modular framework for recognizing and executing user requests through tools by decomposing tool usage into four explicit LLM roles: $LLM_{intent}$, $LLM_{parameters}$, $LLM_{summary}$, and $LLM_{chat}$. This divide-and-conquer design enables flexible use of multiple tools and allows different LLMs to specialize per role, while leveraging open-source models to avoid third-party dependencies. Evaluated on the LLMaker video-game level design task against Ollama Tools, FreYR achieves higher completion rates and faster overall response times in most configurations, with token usage benefits that depend on model and scenario. The work demonstrates both improved performance and creativity in tool-enabled LLMs and provides open-source code for broader adoption and extension, suggesting practical impact for real-time content editing and domain-specific tooling.

Abstract

Large language models excel as conversational agents, but their capabilities can be further extended through tool usage, i.e.: executable code, to enhance response accuracy or address specialized domains. Current approaches to enable tool usage often rely on model-specific prompting or fine-tuning a model for function-calling instructions. Both approaches have notable limitations, including reduced adaptability to unseen tools and high resource requirements. This paper introduces FREYR, a streamlined framework that modularizes the tool usage process into separate steps. Through this decomposition, we show that FREYR achieves superior performance compared to conventional tool usage methods. We evaluate FREYR on a set of real-world test cases specific for video game design and compare it against traditional tool usage as provided by the Ollama API.

Figures (1)

• Figure 1: An overview of our Freyr framework on editing a video game level.