Neurosymbolic LoRA: Why and When to Tune Weights vs. Rewrite Prompts

TL;DR

This work tackles adapting large language models by uniting numerical fine-tuning and symbolic prompt manipulation. It introduces neurosymbolic LoRA, which uses a unified monitoring signal and a reward-based classifier to dynamically switch between LoRA updates for factual reconstruction and TextGrad edits for style and constraint alignment, with symbolic edits offloaded to an external LLM to maintain memory efficiency. Empirical results across multiple backbones and tasks show that neurosymbolic LoRA outperforms purely numerical or purely symbolic baselines, and its data-remaking capability yields high-quality training data for downstream fine-tuning, especially in data-scarce domains. The framework offers a memory-efficient, scalable blueprint for hybrid fine-tuning, with practical implications for edge deployment and rapid domain adaptation, and points to future work in expanding unified signals and broader numeric-symbolic pairings.

Abstract

Large language models (LLMs) can be adapted either through numerical updates that alter model parameters or symbolic manipulations that work on discrete prompts or logical constraints. While numerical fine-tuning excels at injecting new factual knowledge, symbolic updates offer flexible control of style and alignment without retraining. We introduce a neurosymbolic LoRA framework that dynamically combines these two complementary strategies. Specifically, we present a unified monitoring signal and a reward-based classifier to decide when to employ LoRA for deeper factual reconstruction and when to apply TextGrad for token-level edits. Our approach remains memory-efficient by offloading the symbolic transformations to an external LLM only when needed. Additionally, the refined prompts produced during symbolic editing serve as high-quality, reusable training data, an important benefit in data-scarce domains like mathematical reasoning. Extensive experiments across multiple LLM backbones show that neurosymbolic LoRA consistently outperforms purely numerical or purely symbolic baselines, demonstrating superior adaptability and improved performance. Our findings highlight the value of interleaving numerical and symbolic updates to unlock a new level of versatility in language model fine-tuning.

Figures (5)

• Figure 1: Overview of the neurosymbolic LoRA framework: we visually depict the workflow for our framework. In particular, there are two primary means to alternate between numerical and symbolic updates, either via a selection criteria that utilizes a training signal or using a reward-based classifier.
• Figure 2: A depiction of the trend for accuracy on GSM8k for the Llama-3.1-8B-Instruct model as the number of training epochs for the baselines and our hybrid framework NS LoRA with two primary selection criteria $C(x)_2$ and $C(x)_1$ corresponding to NS LoRA (2) and (3) respectively.
• Figure 3: Performance comparison of traditional LoRA and neurosymbolic LoRA during training of the Phi-3-mini-4k-Ins and DeepSeek-R1-Distill-Qwen-1.5B model on GSM8K.
• Figure 4: Impact of neurosymbolic LoRA on LLM response. Answer#1 is the response of the pre-trained LLM. Answer#2 is the response of the LLM post LoRA. Answer#3 is the response of the LLM post NS LoRA (3).
• Figure 5: Examples of new questions generated by TextGrad during symbolic update. The text in red were added to the original question.