Dynamic Tsetlin Machine Accelerators for On-Chip Training at the Edge using FPGAs

TL;DR

The paper tackles the challenge of on-edge training for privacy-sensitive IoT data, where backpropagation-heavy DNN training is impractical on resource-constrained hardware. It presents a Dynamic Tsetlin Machine (DTM) accelerator implemented on an FPGA that supports both Vanilla TM and Coalesced TM (CoTM) through runtime reconfiguration, partial clause computation, and LUT/BRAM-friendly arithmetic. Key contributions include a modular five-module architecture (Clause Matrix, Weight Matrix, Weight Update Matrix, TA Update Matrix, Argmax), a PRNG cluster for on-chip randomness, and mechanisms to skip TA updates when unused, enabling energy-efficient, adaptable edge training. Evaluation on MNIST, Fashion-MNIST, KMNIST, and KWS-6 demonstrates competitive GOP/s/W, flexible TM switching, and significant training-time reductions, paving the way for online recalibration and personalized edge learning. Future work envisions extending to Convolution and Composite TM to handle larger datasets and more complex tasks.

Abstract

The increased demand for data privacy and security in machine learning (ML) applications has put impetus on effective edge training on Internet-of-Things (IoT) nodes. Edge training aims to leverage speed, energy efficiency and adaptability within the resource constraints of the nodes. Deploying and training Deep Neural Networks (DNNs)-based models at the edge, although accurate, posit significant challenges from the back-propagation algorithm's complexity, bit precision trade-offs, and heterogeneity of DNN layers. This paper presents a Dynamic Tsetlin Machine (DTM) training accelerator as an alternative to DNN implementations. DTM utilizes logic-based on-chip inference with finite-state automata-driven learning within the same Field Programmable Gate Array (FPGA) package. Underpinned on the Vanilla and Coalesced Tsetlin Machine algorithms, the dynamic aspect of the accelerator design allows for a run-time reconfiguration targeting different datasets, model architectures, and model sizes without resynthesis. This makes the DTM suitable for targeting multivariate sensor-based edge tasks. Compared to DNNs, DTM trains with fewer multiply-accumulates, devoid of derivative computation. It is a data-centric ML algorithm that learns by aligning Tsetlin automata with input data to form logical propositions enabling efficient Look-up-Table (LUT) mapping and frugal Block RAM usage in FPGA training implementations. The proposed accelerator offers 2.54x more Giga operations per second per Watt (GOP/s per W) and uses 6x less power than the next-best comparable design.

Figures (15)

• Figure 1: Block diagram of the fundamental components of the Tsetlin Machine algorithms for inference - they build the Vanilla TM (pink) and the Coalesced TM (CoTM) (yellow). Component breakdown: a) shows the pre-processing or Booleanization process (adapted from Olga_reproduciable) to generate the inputs to TMs using an MNIST datapoint example; b) shows the learning element - the Tsetlin Automata (TA); c) shows how each TA relates to its respective Boolean literal to create the Clause Output. The pink and yellow blocks for the Vanilla and CoTM show Class sum computation for each class (d and e).
• Figure 2: Block diagram of learning process used in the Vanilla TM and CoTM referred to as feedback. Feedback is used to transition the learning elements of the model, i.e., the TAs and weights (weights for CoTM). These processes are presented visually here, but will be explored as algorithm blocks in the subsequent section.
• Figure 3: The log scale comparison of logic-based (clause compute) vs. integer-based (class sum compute) arithmetic operations in CoTM inference. The number of operations increase with clauses.
• Figure 4: The architecture of the proposed Dynamic Tsetlin Machine (DTM) training accelerator. The DTM accommodates both the Vanilla and CoTM algorithms. It consists of 5 core modules: 1-Clause Matrix, 2-Weight Matrix, 3-Weight Update Matrix, 4-TA State Update Matrix, 5-Argmax Output. Blocks 1, 2 and 5 are for Vanilla and CoTM inference while TM training needs both blocks 3 and 4. All buffers and Pseudo Random Number Generator (PRNG) blocks are used for both algorithms. 6 also shows how the clause computation is mapped into LUT6 FPGA elements.
• Figure 5: Visualizing the iterations required to compute full clause and weight matrices. These figures show the rounds of compute required for the Clause Matrix and Weight Matrix. The colors of (a) and (b) correspond to the colors of the matrix modules in Fig. \ref{['fig:CoTM_Arch']}.