Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tiny, On-Device Decision Makers with the MiniConv Library

Carlos Purves

TL;DR

This work tackles the challenge of deploying visual reinforcement learning on edge devices by proposing a split-policy architecture that moves early vision processing to the device using MiniConv, a library of small encoders that compile to OpenGL fragment shaders. The on-device encoder outputs a compact K-channel feature tensor that is sent to a remote policy head, reducing data transmission and end-to-end latency while preserving learning performance across multiple tasks and devices. Empirical results on NVIDIA Jetson Nano, Raspberry Pi 4B, and Pi Zero 2 W show MiniConv can achieve competitive final performance with substantial latency and server-scaling benefits, especially under bandwidth constraints. The approach offers a practical, open-source pathway for on-device feature extraction and edge-cloud RL collaboration, balancing computation and communication to suit diverse hardware and network conditions.

Abstract

Reinforcement learning (RL) has achieved strong results, but deploying visual policies on resource-constrained edge devices remains challenging due to computational cost and communication latency. Many deployments therefore offload policy inference to a remote server, incurring network round trips and requiring transmission of high-dimensional observations. We introduce a split-policy architecture in which a small on-device encoder, implemented as OpenGL fragment-shader passes for broad embedded GPU support, transforms each observation into a compact feature tensor that is transmitted to a remote policy head. In RL, this communication overhead manifests as closed-loop decision latency rather than only per-request inference latency. The proposed approach reduces transmitted data, lowers decision latency in bandwidth-limited settings, and reduces server-side compute per request, whilst achieving broadly comparable learning performance by final return (mean over the final 100 episodes) in single-run benchmarks, with modest trade-offs in mean return. We evaluate across an NVIDIA Jetson Nano, a Raspberry Pi 4B, and a Raspberry Pi Zero 2 W, reporting learning results, on-device execution behaviour under sustained load, and end-to-end decision latency and scalability measurements under bandwidth shaping. Code for training, deployment, and measurement is released as open source.

Tiny, On-Device Decision Makers with the MiniConv Library

TL;DR

This work tackles the challenge of deploying visual reinforcement learning on edge devices by proposing a split-policy architecture that moves early vision processing to the device using MiniConv, a library of small encoders that compile to OpenGL fragment shaders. The on-device encoder outputs a compact K-channel feature tensor that is sent to a remote policy head, reducing data transmission and end-to-end latency while preserving learning performance across multiple tasks and devices. Empirical results on NVIDIA Jetson Nano, Raspberry Pi 4B, and Pi Zero 2 W show MiniConv can achieve competitive final performance with substantial latency and server-scaling benefits, especially under bandwidth constraints. The approach offers a practical, open-source pathway for on-device feature extraction and edge-cloud RL collaboration, balancing computation and communication to suit diverse hardware and network conditions.

Abstract

Reinforcement learning (RL) has achieved strong results, but deploying visual policies on resource-constrained edge devices remains challenging due to computational cost and communication latency. Many deployments therefore offload policy inference to a remote server, incurring network round trips and requiring transmission of high-dimensional observations. We introduce a split-policy architecture in which a small on-device encoder, implemented as OpenGL fragment-shader passes for broad embedded GPU support, transforms each observation into a compact feature tensor that is transmitted to a remote policy head. In RL, this communication overhead manifests as closed-loop decision latency rather than only per-request inference latency. The proposed approach reduces transmitted data, lowers decision latency in bandwidth-limited settings, and reduces server-side compute per request, whilst achieving broadly comparable learning performance by final return (mean over the final 100 episodes) in single-run benchmarks, with modest trade-offs in mean return. We evaluate across an NVIDIA Jetson Nano, a Raspberry Pi 4B, and a Raspberry Pi Zero 2 W, reporting learning results, on-device execution behaviour under sustained load, and end-to-end decision latency and scalability measurements under bandwidth shaping. Code for training, deployment, and measurement is released as open source.
Paper Structure (10 sections, 1 equation, 5 figures, 6 tables)

This paper contains 10 sections, 1 equation, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Implementation
  4. Evaluation
  5. Learning
  6. Execution Performance
  7. End-to-End Decision Latency
  8. Server Scalability
  9. Discussion
  10. Conclusion

Figures (5)

  • Figure 1: OpenGL fragment shaders can implement convolution and pooling by sampling input textures and writing output textures.
  • Figure 2: Per-frame processing time across devices as the input image size varies (mean of 100 consecutive inferences; shaded region shows standard deviation).
  • Figure 3: Sustained inference performance over 5,000 consecutive frames.
  • Figure 4: Resource usage during sustained inference (Pi Zero 2 W: RAM utilisation out of 512MB; Jetson Nano: power usage and memory pressure during 5,000 consecutive 3000$\times$3000 frames).
  • Figure 5: A breakdown of the steps involved in each decision that contribute to decision latency.