Electronic Circuit Principles of Large Language Models

TL;DR

This work introduces the Electronic Circuit Principle (ECP), a circuit-inspired framework that maps inference-time learning and reasoning (ITL/ITR) onto electromotive force and a resistive network to predict and improve LLM performance. By deriving closed-form relationships for power, voltage, and resistance, the authors obtain strong predictions across 9 LLMs and 350 tasks (Spearman > 0.7) and demonstrate substantial improvements over traditional scaling laws. They show that optimizing prompting components (magnetic field) and reasoning strategies (reducing resistance) yields tangible performance gains, including surpassing a majority of human competitors in IOI/IMO tasks using relatively weaker LLMs. The work also provides extensive methodological detail and theoretical proofs supporting self-consistency, coverage, and resistor-optimization strategies, offering a rigorous, modular toolkit for future LLM design and evaluation. Overall, ECP offers a principled, quantitative lens to predict, interpret, and optimize modular prompting and reasoning approaches in large language models.

Abstract

Large language models (LLMs) such as DeepSeek-R1 have achieved remarkable performance across diverse reasoning tasks. To uncover the principles that govern their behaviour, we introduce the Electronic Circuit Principles (ECP), which maps inference-time learning (ITL) onto a semantic electromotive force and inference-time reasoning (ITR) onto a resistive network governed by Ohm's and Faraday's laws. This circuit-based modelling yields closed-form predictions of task performance and reveals how modular prompt components interact to shape accuracy. We validated ECP on 70,000 samples spanning 350 reasoning tasks and 9 advanced LLMs, observing a about 60% improvement in Pearson correlation relative to the conventional inference-time scaling law. Moreover, ECP explains the efficacy of 15 established prompting strategies and directs the development of new modular interventions that exceed the median score of the top 80% of participants in both the International Olympiad in Informatics and the International Mathematical Olympiad. By grounding LLM reasoning in electronic-circuit principles, ECP provides a rigorous framework for predicting performance and optimising modular components.

Figures (12)

• Figure 1: The Introduction of Inference-Time Learning (ITL) and Inference-Time Reasoning (ITR), and Comparison of Micro- and Macro-Circuits. a, Introduction to ITL, which pairs queries with solution in input context as references, and to ITR, which provides step-by-step reasoning in model-generated output. b, Electronic Circuit Principles on Macro-circuit analyses: Traditional micro-circuits capture basical linguistic functions but miss higher-order reasoning principles. The micro-circuit structure follows rai2024practical.
• Figure 2: Glossary of Terms and Schematic of Electronic Circuit Principles (ECP). Inference-time learning (ITL) is depicted as a dynamic semantic magnetic field whose strength rises when demonstrations accompany the prompt, adding potential that steers token selection. Inference-time reasoning (ITR) is modelled as a series circuit: each reasoning sub-task acts as a resistor, and their sum gives the total cognitive load. The LLMs' intrinsic capacity supplies the driving voltage, and empirical accuracy corresponds to the power delivered across the load. "Strength", "Resistance" and "Voltage" map to field intensity, sub-task difficulty and LLM capability, respectively, offering a compact framework to quantify and compare LLM behaviour.
• Figure 3: Verification of the ECP via zero‐shot reasoning. a, Log-linear relationship between the classical inference-time scaling law (log token length versus task accuracy) and empirical performance of 4 LLMs, GPT-3.5 Turbo, GPT-4o-mini, GPT-4o and Qwen2.5-7B-Instruct, reveals a significant negative correlation (Spearman $\rho \in [-0.5664, -0.4254]$,$P < 0.01$). b, Observed zero-shot accuracy plotted against ECP-predicted computational power for 9 LLMs shows a strong positive correlation (Spearman $\rho >0.71$, $P < 0.01$), confirming the ECP's predictive validity. c, Comparative evaluation of 4 semantic-similarity metrics used to estimate field strength: projection length, Manhattan ($L_1$) distance, Euclidean ($L_2$) distance and cosine similarity, identifies projection length as the most accurate predictor. More complete results are shown in Fig. \ref{['fig:itl-verification-append']} in Appendix.
• Figure 4: Interpreting for Inference‐Time Learning Strategies of ECP. a,ECP‐predicted theoretical power correlates positively with accuracy across three semantic encoders (BERT, RoBERTa, BGE) over 350 tasks. b, Meta‐recognition embeddings yield higher mean theoretical power (73.6 vs 66.9) and accuracy (54.8% vs 49.9%) than vanilla encoders, with stronger Spearman correlations (0.75 vs 0.61, $P<0.01$), which using RoBERTa as semantic encoders. c, Cumulative distributions (CCDF, CDF) and 95% confidence ellipses for three ITR configurations (Acc = 0.442, 0.376, 0.352) illustrate that larger induced semantic fields correspond to higher theoretical power and observed accuracy.
• Figure 5: Interpreting for Inference‐Time Learning Strategies of ECP. a, ITR reconfigures the model’s reasoning circuit: relative to Rationale, removing the rationale (w/o Rationale) increases effective resistance (3.97 $\Omega$ vs 4.40 $\Omega$), lowers theoretical power (62.1 vs 56.7) and improves accuracy (41.7% vs 33.6%), as shown by the contour shifts. b, Applying natural language (NL), tool usage (TU) or program-of-thought (PoT) to instantiate the "superconducting" phenomenon ($R_c = 0$) surpasses NL for better Power-Accuracy correlations, in line with findings on externalized reasoning. c, Circuit analogies for two sampling strategies. Self-consistency sampling comprises parallel reasoning paths merging through an aggregation resistance $R_s$, denoting consensus cost; coverage sampling sets ($R_s = 0$), allowing independent paths for greater diversity. d, Empirical trends follow a logarithmic relation with sample count. e, Adopting independent rather than mixture sample counts motivates replacing linear models with logarithmic ones, markedly improving predictions.