InTreeger: An End-to-End Framework for Integer-Only Decision Tree Inference

TL;DR

InTreeger tackles the challenge of deploying tree-ensemble inferences on resource-constrained devices by delivering an end-to-end, architecture-agnostic pipeline that converts floating-point DT/RF inferences into integer-only, fixed-point operations implemented as standalone C code. The approach combines probability-to-integer conversion with FlInt-style integer comparisons, all orchestrated through an extended tl2cgen and Treelite intermediary, to produce portable binaries with no external libraries. Across ARM, x86, and RISC-V, InTreeger demonstrates substantial latency improvements and meaningful energy savings without sacrificing accuracy, and showcases a concrete RISC-V microcontroller deployment as a practical edge use-case. This work enables safe, efficient edge AI for DT ensembles, broadening deployment options for embedded and ultra-low-power devices while simplifying integration for domain experts, and it provides a public release for broad adoption.

Abstract

Integer quantization has emerged as a critical technique to facilitate deployment on resource-constrained devices. Although they do reduce the complexity of the learning models, their inference performance is often prone to quantization-induced errors. To this end, we introduce InTreeger: an end-to-end framework that takes a training dataset as input, and outputs an architecture-agnostic integer-only C implementation of tree-based machine learning model, without loss of precision. This framework enables anyone, even those without prior experience in machine learning, to generate a highly optimized integer-only classification model that can run on any hardware simply by providing an input dataset and target variable. We evaluated our generated implementations across three different architectures (ARM, x86, and RISC-V), resulting in significant improvements in inference latency. In addition, we show the energy efficiency compared to typical decision tree implementations that rely on floating-point arithmetic. The results underscore the advantages of integer-only inference, making it particularly suitable for energy- and area-constrained devices such as embedded systems and edge computing platforms, while also enabling the execution of decision trees on existing ultra-low power devices.

Figures (5)

• Figure 1: The overview of InTreeger. From input dataset to integer-only architecture-agnostic if-else tree.
• Figure 2: The differences between the observed probabilities for the standard floating-point implementation, and our integer-only implementation.
• Figure 3: Performance evaluation (elapsed cycles) of the inference for the two datasets over the selected architectures and application-level systems with a variable number of trees in each ensemble.
• Figure 4: A schematic overview of the setup that was used for conducting the power measurements. Extracted from joulescope_man.
• Figure 5: The power consumption profile of a floating-point and integer-only implementation of 14,500,000 inferences of a Random Forest model with 50 classifiers and a maximum depth of 7.