Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Floating-floating point: a highly accurate number representation with flexible Counting ranges

Itamar Cohen, Gil Einziger

TL;DR

This paper introduces Floating-Floating-Point (F2P) - a floating point number that varies the partition between mantissa and exponent that leads to a large counting range combined with improved accuracy over a selected sub-range.

Abstract

Efficient number representation is essential for federated learning, natural language processing, and network measurement solutions. Due to timing, area, and power constraints, such applications use narrow bit-width (e.g., 8-bit) number systems. The widely used floating-point systems exhibit a trade-off between the counting range and accuracy. This paper introduces Floating-Floating-Point (F2P) - a floating point number that varies the partition between mantissa and exponent. Such flexibility leads to a large counting range combined with improved accuracy over a selected sub-range. Our evaluation demonstrates that moving to F2P from the state-of-the-art improves network measurement accuracy and federated learning.

Floating-floating point: a highly accurate number representation with flexible Counting ranges

TL;DR

This paper introduces Floating-Floating-Point (F2P) - a floating point number that varies the partition between mantissa and exponent that leads to a large counting range combined with improved accuracy over a selected sub-range.

Abstract

Efficient number representation is essential for federated learning, natural language processing, and network measurement solutions. Due to timing, area, and power constraints, such applications use narrow bit-width (e.g., 8-bit) number systems. The widely used floating-point systems exhibit a trade-off between the counting range and accuracy. This paper introduces Floating-Floating-Point (F2P) - a floating point number that varies the partition between mantissa and exponent. Such flexibility leads to a large counting range combined with improved accuracy over a selected sub-range. Our evaluation demonstrates that moving to F2P from the state-of-the-art improves network measurement accuracy and federated learning.
Paper Structure (11 sections, 10 equations, 1 figure, 6 tables)

This paper contains 11 sections, 10 equations, 1 figure, 6 tables.

Table of Contents

  1. Introduction
  2. The F2P Number System
  3. Preliminaries
  4. Floating Point Number Representation
  5. High-level Description of F2P
  6. F2P's Exponent Encoding
  7. Representing Integers
  8. Evaluation
  9. Using F2P as Approximate Counters
  10. Using F2P for Neural Networks Quantization
  11. Conclusion

Figures (1)

  • Figure 1: The positive values that are represented by different 8-bit grids. Existing solutions provide either good resolution and a small counting range (INT8, 5M2E) or a large counting range with inferior resolution (2M5E). Our F2P flavors (pink and green) combine a relatively large range with a dense representation for a chosen sub-range.