LITHE: Bridging Best-Effort Python and Real-Time C++ for Hot-Swapping Robotic Control Laws on Commodity Linux

TL;DR

LITHE (Linux Isolated Threading for Hierarchical Execution) bridges a critical gap between high-level AI and low-level real-time control to unlock continuous real-time evolution of embodied intelligence in safe, human-in-the-loop systems.

Abstract

Modern robotic systems rely on hierarchical control, where a high-level "Brain" (Python) directs a lower-level "Spine" (C++ real-time controller). Despite its necessity, this hierarchy makes it difficult for the Brain to completely rewrite the Spine's immutable control logic, consequently inhibiting fundamental adaptation for different tasks and environments. Conventional approaches require complex middleware, proprietary hardware, or sacrifice real-time performance. We present LITHE (Linux Isolated Threading for Hierarchical Execution), a lightweight software architecture that collapses the robot control hierarchy onto a commodity single-board computer (Raspberry Pi 4B with pi3hat), while maintaining safe frequency decoupling between the Brain and Spine. LITHE integrates strict CPU isolation (isolcpus), lock-free inter-process communication (IPC), and pipelined execution to meet high-frequency deadlines with minimal jitter. By adding multi-threaded dynamic linking, LITHE enables a Python-based Brain to dynamically evolve the logic of a 1kHz C++ Spine without interruption. We validate "functional real-time" system performance with worst-case execution time (WCET) < 100 $μ$s and maximum release jitter (MRJ) < 4 $μ$s under heavy load. We demonstrate a novel application where a large language model (LLM) supervisor performs online system identification to evolve a real-time controller on-the-fly, without interrupting the 1 kHz control loop. In essence, LITHE eliminates the "immutable compiled code" bottleneck for best-effort Brains to synthesize and inject completely new control laws into the real-time Spine. This bridges a critical gap between high-level AI and low-level real-time control to unlock continuous real-time evolution of embodied intelligence in safe, human-in-the-loop systems.

Figures (5)

• Figure 1: Hierarchical control from the Brain, across the Spine, onto different tasks and applications. LITHE enables high-level "best-effort" models and algorithms to evolve and hot-swap low-level real-time controllers without interrupting on-the-fly real-time operation. This empowers continuous, real-time evolution of embodied intelligence.
• Figure 2: LITHE system architecture for evolving real-time robot control on a Raspberry Pi. A top level “Python Brain” is isolated to CPU 2. It can run independently, with best-effort tasks such as deep learning, or in collaboration with an external agentic robot client. The Brain sends commands and trajectories via lock-free inter-process communication to a low-level “C++ Spine” which is isolated on CPU 1. The Spine maintains a functional real-time controller, such as PID or impedance control. Through our hot-swap mechanism, this controller can be hot-swapped on-the-fly, without interrupting real-time. It interacts with robot hardware through an isolated transport layer on CPU 3, based on the open-source pi3hat library for the pi3hat GPIO expansion board. For our simple experiment in Section \ref{['sec:experiment_supervised']}, we set up an external agent based on qwen2.5-coder-7b to collaborate with the Brain. Through LITHE, we then implement various hot-swapped controllers that interact with a 1-DOF robot comprised of moteus hardware and a 3D printed arm.
• Figure 3: Example timing diagram of the pipelined (double buffered) execution for Cycle $k$. Spine dispatches the command $u_k$ (computed in the previous cycle) and immediately begins computing the next control law $u_{k+1}$ while Transport handles the blocking CAN-FD bus transactions, before returning feedback $x_{k+1}$ for the next cycle. This parallel structure allows the control logic to execute during the hardware transmission delay.
• Figure 4: Jitter distributions of the Spine under heavy synthetic load on the isolated Brain. (a) Execution Times for Spine to dispatch command u[k] under heavy synthetic load (NumPy, Stress-ng). Range is [0 $\mu$s, 100 $\mu$s], with 2 $\mu$s bin size. The worst case execution time (WCET) of transport handoff under NumPy is 87.704 $\mu$s, and for Stress-ng is 98.299 $\mu$s. (b) Release Jitter Distributions under heavy synthetic load (NumPy, Stress-ng). Range is [0 $\mu$s, 4 $\mu$s], with 0.08 $\mu$s bin size. The maximum release jitter (MRJ) for NumPy is 2.997 $\mu$s, and for Stress-ng is 3.110 $\mu$s.
• Figure 5: Dynamic evolution of real-time control law for robot arm through an external decentralized robot agent. The agent is qwen2.5-coder-7b, a large language model (LLM) (a) The agent starts by thinking about our prompts of the system. It formulates probing controllers for system identification, and a final gravity compensation controller based on received telemetry (green arrows). These controllers are sent at approximately $t=2, 6, 13$s (blue arrows), and correspond with increased LITHE jitter due to subsequent compilation processes (red boxes). (b) Execution time for command handoff in each control cycle. Largest time measured was 57.648 $\mu$s (within WCET 98.299 $\mu$s, Figure \ref{['fig:jitter']}a). (c) Release Jitter for each control cycle. Maximum jitter measured was 0.703 $\mu$s (within MRJ 3.110 $\mu$s, Figure \ref{['fig:jitter']}b). After compilation is complete for each new controller, they are hot-swapped into the Spine at approximately $t=5, 9, 16$s (blue arrows) (d) The robot's tracking performance across four hot-swapped real-time controllers, at 1000Hz. No real-time deadlines were missed. Note the initial startup delay for $t<2.2$s to initialize LITHE. At $t=21.5$s, we simulate a brain crash, but LITHE continues to hold the load smoothly against gravity, preventing mechanical collapse.