Configurable p-Neurons Using Modular p-Bits

TL;DR

The paper tackles the limitation of coupled p-bits by introducing modular p-bits that decouple stochastic and input paths, enabling configurable p-neurons with activations such as $p$-Tanh, $p$-Sigmoid, and $p$-ReLU. It presents spintronic (CMOS + sMTJ) designs, including a 2M stochastic unit and a 1M1R variant, demonstrating tunable probabilistic ranges and distributions; these are complemented by analytic insights, notably $\frac{V_{probabilistic}}{V_{DD}} = \frac{TMR}{2+TMR}$, linking probabilistic range to MTJ TMR. The authors also implement LUT-free digital p-neurons on FPGA using LFSR-based randomness with stochastic-unit sharing, achieving order-of-magnitude reductions in hardware resources and enabling a modular p-AND that follows Boltzmann-like statistics. Collectively, the work provides scalable, configurable hardware routes for Probabilistic Neural Networks, bridging spintronic and digital implementations through modular p-neurons. $V_{\mathit{probabilistic}} / V_{\mathit{DD}} = \frac{\mathit{TMR}}{2+\mathit{TMR}}$ captures a key tunability relation that governs probabilistic activation Range in high-TMR devices.

Abstract

Probabilistic bits (p-bits) have recently been employed in neural networks (NNs) as stochastic neurons with sigmoidal probabilistic activation functions. Nonetheless, there remain a wealth of other probabilistic activation functions that are yet to be explored. Here we re-engineer the p-bit by decoupling its stochastic signal path from its input data path, giving rise to a modular p-bit that enables the realization of probabilistic neurons (p-neurons) with a range of configurable probabilistic activation functions, including a probabilistic version of the widely used Logistic Sigmoid, Tanh and Rectified Linear Unit (ReLU) activation functions. We present spintronic (CMOS + sMTJ) designs that show wide and tunable probabilistic ranges of operation. Finally, we experimentally implement digital-CMOS versions on an FPGA, with stochastic unit sharing, and demonstrate an order of magnitude (10x) saving in required hardware resources compared to conventional digital p-bit implementations.

Figures (5)

• Figure 1: P-neurons via decoupled modular p-bits. (a) Original p-bit architecture where the stochastic and input paths are coupled. The original design is shown, where the two paths are coupled at the drain node. (b) Proposed architecture that decouples the stochastic path from the input path, decoupling the design of these two paths. A design example based on a dual sMTJ voltage divider cell is shown. (c) P-neurons with different probabilistic (time-average) activation functions by modular engineering of the stochastic and the activation units.
• Figure 2: Probabilistic Neurons: spintronic (CMOS + sMTJ) design examples. (a)--(c) response of p-neurons (blue dots: instantaneous response, large orange circles: time-averaged response) with p-Tanh, p-Sigmoid, and p-RELU activation functions, respectively. Non-bipolar data points are due to the limited slew-rate of the amplifier Mohammed_2025GeneralizedOTAOTAMWSCASScalableOTAs. (d) and (e) transistor-level circuit designs for implementing p-neurons with p-Tanh and p-Sigmoid activation functions, respectively. The stochastic unit is a dual sMTJ voltage divider (2M cell). For p-Sigmoid, the activation unit was re-engineered to obtain and optimize unipolar activation of the p-neuron through the reduction of the $W/L$ of $M_6$ by a factor of three. (f) transistor-level circuit design for implementing a p-neuron with a p-RELU activation function, using a single sMTJ + single resistor (1M1R) cell as the stochastic unit, with $R_1$ = $0.35/G_0$ and $\alpha = 0.155$. For all designs $V_{\mathit{DD}}$ = 0.8.
• Figure 3: Engineering the stochastic unit: (a) A dual sMTJ stochastic unit - 2M cell (left), consisting of two sMTJs in series (each with a uniform conductance distribution). The probability distribution of $V_{Stochastic}$ of the 2M cell that is approximately normal (right). (b) A stochastic unit that consists of one sMTJ and one fixed resistor connected in series - 1M1R cell (left), with $R_1$ = $0.35/G_0$ and $\alpha = 0.155$. The probability distribution of $V_{Stochastic}$ of the 1M1R cell that is approximately uniform (right).
• Figure 4: Tunability of the p-neuron's probabilistic activation function. (a) For a stochastic unit of two sMTJs in series (2M cell), the characteristics of the $V_{Stochastic}$ probability distribution (mean and variance) can be tuned by $V_{DD}$. (b) Tuning the p-neuron's probabilistic range of the p-Tanh activation function. The p-Tanh tunability is controlled by a scaling factor $\beta$. (c) Analytical limit that describes the tunability of the probabilistic range ($V_{probabilistic})$ as a function of the sMTJ tunneling magnetoresistance (TMR). The other data points refer to other experimental realizations of p-bits.
• Figure 5: Digital p-neurons and stochastic unit sharing experiments. (a)--(d) Time-averaged response of digital p-neurons with (a) p-Tanh (b) p-Sigmoid (c) p-ReLU and (d) p-Linear activation functions, implemented on an FPGA. (e)--(h) The p-neuron digital designs employing 32-bit comparators, which compare the input string $I_{\mathit{IN}}$ with the random number generated by the stochastic unit. (i) Experimental setup for the FPGA implementation. (j) Shared stochastic unit. (k) The result of an FPGA experiment where one p-Sigmoid p-neuron and one p-Linear p-neuron receive their random numbers, with different probability distributions, from a single shared stochastic unit. (l) A modular probabilistic AND (p-AND) gate with 3 p-neurons that share a single RNG as their stochastic unit. The interconnections between the p-neurons are shown with weights on bidirectional arrows and biases on unidirectional arrows. (m) Histogram showing the probability distribution of visiting the correct p-AND gate states, and the expected Boltzmann distribution. (n) Savings in FPGA hardware resource utilization. (o) Comparison of the estimated transistor count in custom digital designs of modular p-Sigmoid p-neurons with original FPGA p-bits singh2024cmos.