Highly Efficient and Effective LLMs with Multi-Boolean Architectures
Ba-Hien Tran, Van Minh Nguyen
TL;DR
This paper tackles the high cost of quantizing and binarizing LLMs by introducing mbok, a native Boolean framework that trains weights directly in the Boolean domain using multiple Boolean kernels. It combines a rank-1 Boolean reformulation (SVID) with successive kernel extraction and knowledge distillation to transfer and refine information from a full-precision teacher, while automatically allocating kernels under a fixed budget. Empirically, mbok achieves near fp16 performance with extremely low bitrates (e.g., 2–3 kernels) and outperforms state-of-the-art ultra low-bit quantization and binarization baselines across OPT and Llama family models, with substantial memory and latency benefits. The approach is poised to enable efficient deployment on conventional hardware and motivates the development of dedicated Boolean accelerators to maximize gains in training and inference efficiency.
Abstract
Weight binarization has emerged as a promising strategy to reduce the complexity of large language models (LLMs). Existing approaches fall into post-training binarization, which is simple but causes severe performance loss, and training-aware methods, which depend on full-precision latent weights, adding complexity and limiting efficiency. We propose a novel framework that represents LLMs with multi-kernel Boolean parameters and, for the first time, enables direct finetuning LMMs in the Boolean domain, eliminating the need for latent weights. This enhances representational capacity and dramatically reduces complexity during both finetuning and inference. Extensive experiments across diverse LLMs show our method outperforms recent ultra low-bit quantization and binarization techniques.