Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Instability of backoff protocols with arbitrary arrival rates

Leslie Ann Goldberg, John Lapinskas

TL;DR

This work addresses the long-standing Aldous conjecture on the instability of queue-free backoff protocols for any positive arrival rate. It introduces a domination technique that bounds the joint distribution of messages across the backoff bins by reducing to independent Poisson variables via a two-stream externally-jammed construction, random-unsticking, and time-reversal couplings. The authors prove instability for all backoff protocols whose send sequence $\mathbf{p}$ is not LCED (largely constant with exponential decay), thereby extending prior results and covering essentially all practical backoff schemes except a tightly defined LCED class. The approach yields strong instability results, including for monotone and low-mean waiting sequences, and identifies LCED as the principal remaining obstacle to a full resolution of the conjecture. This has implications for the design of backoff schemes in distributed systems and cloud services where such contention-resolution protocols are deployed.

Abstract

In contention resolution, multiple processors are trying to coordinate to send discrete messages through a shared channel with limited communication. If two processors send at the same time, the messages collide and are not transmitted successfully. Queue-free backoff protocols are an important special case - for example, Google Drive and AWS instruct their users to implement binary exponential backoff to handle busy periods. It is a long-standing conjecture of Aldous (IEEE Trans. Inf. Theory 1987) that no stable backoff protocols exist for any positive arrival rate of processors. This foundational question remains open; instability is only known in general when the arrival rate of processors is at least 0.42 (Goldberg et al. SICOMP 2004). We prove Aldous' conjecture for all backoff protocols outside of a tightly-constrained special case using a new domination technique to get around the main difficulty, which is the strong dependencies between messages.

Instability of backoff protocols with arbitrary arrival rates

TL;DR

This work addresses the long-standing Aldous conjecture on the instability of queue-free backoff protocols for any positive arrival rate. It introduces a domination technique that bounds the joint distribution of messages across the backoff bins by reducing to independent Poisson variables via a two-stream externally-jammed construction, random-unsticking, and time-reversal couplings. The authors prove instability for all backoff protocols whose send sequence is not LCED (largely constant with exponential decay), thereby extending prior results and covering essentially all practical backoff schemes except a tightly defined LCED class. The approach yields strong instability results, including for monotone and low-mean waiting sequences, and identifies LCED as the principal remaining obstacle to a full resolution of the conjecture. This has implications for the design of backoff schemes in distributed systems and cloud services where such contention-resolution protocols are deployed.

Abstract

In contention resolution, multiple processors are trying to coordinate to send discrete messages through a shared channel with limited communication. If two processors send at the same time, the messages collide and are not transmitted successfully. Queue-free backoff protocols are an important special case - for example, Google Drive and AWS instruct their users to implement binary exponential backoff to handle busy periods. It is a long-standing conjecture of Aldous (IEEE Trans. Inf. Theory 1987) that no stable backoff protocols exist for any positive arrival rate of processors. This foundational question remains open; instability is only known in general when the arrival rate of processors is at least 0.42 (Goldberg et al. SICOMP 2004). We prove Aldous' conjecture for all backoff protocols outside of a tightly-constrained special case using a new domination technique to get around the main difficulty, which is the strong dependencies between messages.
Paper Structure (28 sections, 34 theorems, 77 equations)

This paper contains 28 sections, 34 theorems, 77 equations.

Table of Contents

  1. Introduction
  2. Formal definitions
  3. Technical context
  4. Proof sketch
  5. Future work
  6. Preliminaries
  7. Preparation for our Technical Result
  8. Definition of externally-jammed process
  9. A two-stream externally-jammed process
  10. Ball vectors and noise
  11. Statement of our technical result
  12. Definitions depending on a suitable send sequence p
  13. The tau <= tau_init regime: Proving Lemma \ref{['lem:tau-init']}
  14. Properties of the block construction
  15. Noise
  16. ...and 13 more sections

Key Result

Theorem 2

For every $\lambda \in (0,1)$ and every monotonically non-increasing send sequence $\mathbf{p}= p_0,p_1,\ldots$, the backoff process with arrival rate $\lambda$ and send sequence $\mathbf{p}$ is unstable.

Theorems & Definitions (83)

  • Conjecture 1: Aldous's Conjecture
  • Theorem 2
  • Theorem 3
  • Definition 3
  • Theorem 4
  • Remark 5
  • Lemma 5
  • Theorem 6: kelly-macphee
  • Corollary 6
  • Definition 7
  • ...and 73 more