SignalLLM: A General-Purpose LLM Agent Framework for Automated Signal Processing

TL;DR

SignalLLM introduces a general-purpose, agentic LLM framework for automated signal processing by integrating task decomposition, adaptive planning with retrieval-augmented generation, and hybrid execution via reasoning, modeling, and optimization modules. The framework enables flexible strategy selection across SP modalities and data constraints, addressing limitations of prior LLM-based SP approaches. Empirical evaluation across radar detection, HAR, text signal coding, feature optimization, and modulated signal recognition demonstrates superior performance, particularly in few-shot and zero-shot settings, compared to traditional methods and existing agent-based baselines. The work highlights the potential of dynamic, memory-enabled, multi-paradigm SP pipelines that leverage LLM reasoning and external tools to automate complex SP workflows with reduced reliance on expert design. It also notes limitations in action-space coverage and suggests future enhancements with advanced RAG, memory, and reinforcement learning for broader applicability and real-time deployment.

Abstract

Modern signal processing (SP) pipelines, whether model-based or data-driven, often constrained by complex and fragmented workflow, rely heavily on expert knowledge and manual engineering, and struggle with adaptability and generalization under limited data. In contrast, Large Language Models (LLMs) offer strong reasoning capabilities, broad general-purpose knowledge, in-context learning, and cross-modal transfer abilities, positioning them as powerful tools for automating and generalizing SP workflows. Motivated by these potentials, we introduce SignalLLM, the first general-purpose LLM-based agent framework for general SP tasks. Unlike prior LLM-based SP approaches that are limited to narrow applications or tricky prompting, SignalLLM introduces a principled, modular architecture. It decomposes high-level SP goals into structured subtasks via in-context learning and domain-specific retrieval, followed by hierarchical planning through adaptive retrieval-augmented generation (RAG) and refinement; these subtasks are then executed through prompt-based reasoning, cross-modal reasoning, code synthesis, model invocation, or data-driven LLM-assisted modeling. Its generalizable design enables the flexible selection of problem solving strategies across different signal modalities, task types, and data conditions. We demonstrate the versatility and effectiveness of SignalLLM through five representative tasks in communication and sensing, such as radar target detection, human activity recognition, and text compression. Experimental results show superior performance over traditional and existing LLM-based methods, particularly in few-shot and zero-shot settings.

Figures (7)

• Figure 1: Overview of the SignalLLM framework. SignalLLM operates in two main stages. Stage 1 focuses on tailored planning, where user requests are processed through task decomposition, subtask planning, and solution refinement. Stage 2 is dedicated to execution, carrying out the plan using either the LLM-Assisted SP Reasoning Module for reasoning-based tasks or the LLM-Assisted SP Modeling Module for tasks requiring LLM-assisted model creation.
• Figure 2: The illustration of SP Task Decomposition Module. A user's natural language request is processed by a Web Searcher to acquire SP Domain Knowledge. which is then used to decompose the high-level goal into a structured subtask chain.
• Figure 3: The illustration of SP Task Solution Planning Module. This module integrates complexity-aware RAG mechanisms with agent-based planning to dynamically support LLMs in solving SP subtasks.
• Figure 4: The illustration of the proposed Tailored LLM-Assisted SP Reasoning Module and its three operational modes. (1) Prompt engineering to do SP tasks: Employs direct task execution through sophisticated prompt design. (2) Code generation to solve the SP tasks: Generates and refines Python or Matlab code via planning and self-reflection. (3) Cross-Modal Reasoning for SP Tasks: Integrates code-generated data with visual representations to solve complex multimodal tasks.
• Figure 5: The illustration of the proposed Tailored LLM-Assisted SP Modeling Module and its three approaches to data-driven modeling. (1) LLMs' Language Modeling Support in SP task: Directly utilizes the LLM's own language modeling capabilities for tasks such as semantic communication. (2) LLM as Optimization Support for SP Model: Employs the LLM to tune the hyperparameters of an external SP model. (3) LLMs' Parameter-Level Support for SP Model: Leverages the LLM's pre-trained parameters to initialize and fine-tune new SP models via knowledge transfer.