GLOSSARY

What is Jitter?

Jitter is deliberate randomness added to a delay, most often a retry delay, so that clients which failed at the same moment do not all retry at the same moment and rebuild the burst that caused the failure.

Try Pinlyx free Read the Data API docs

Free forever plan · No credit card required · Cancel anytime

Quick definition

In a single sentence: backoff decides how long to wait, jitter decides that not everyone waits the same amount.

What it means

Jitter is randomness you add on purpose. In distributed systems it almost always means randomising a delay: instead of every client waiting exactly two seconds, each waits somewhere between zero and four, or between one and three.

The word is borrowed from signal processing, where jitter is unwanted timing variation. In systems work it is the opposite: a small, deliberate imperfection that stops a population of independent clients from behaving as one very large client.

The problem, with numbers

Take five thousand clients hitting the same API. At time T the service degrades and every one of them receives an error. All five thousand start the same retry policy, which is textbook exponential backoff: wait one second, then two, then four.

At T plus one second, five thousand requests arrive inside the same second. They fail. At T plus three seconds, five thousand more. Then at T plus seven. The retry traffic is not a flood, it is a series of spikes, and every spike is as tall as the burst that broke the service in the first place. Backoff has reduced the average load and left the peak untouched, and peak is what saturates a server.

Now spread the same five thousand retries uniformly across an eight-second window. The expected peak is 625 requests per second rather than 5,000 in one second: an eight-fold reduction, from one line of code. That is the entire argument. Jitter costs nothing, changes no architecture, and removes the sharpest edge of a retry storm.

The pattern has a name outside retries too: the thundering herd, where many waiters wake simultaneously for one resource. Ethernet has randomised its binary exponential backoff since the original CSMA/CD design, for precisely this reason. The idea long predates HTTP.

The four strategies

The canonical formulation comes from an AWS Architecture Blog post by Marc Brooker, "Exponential Backoff and Jitter", which simulated the variants against a contended resource and found that adding jitter reduced both total work and time to completion compared with plain backoff. The table below works each one at the same point in the curve, so the shapes are directly comparable: base 500 ms, multiplier 2, fourth attempt, undecorated delay 4.000 seconds.

The practical distinction is the floor. Full jitter can return almost zero, which is fine when the failure was a blip and dangerous when the server is saturated, because a fraction of your clients will retry immediately. Equal jitter guarantees at least half the computed delay while still spreading the other half. Decorrelated jitter grows from the previous actual sleep rather than from the attempt number, so a long-lived client keeps widening its spread instead of resetting to the same ladder.

Retry-After is a floor, and jitter goes above it

When the server tells you how long to wait, obey it and then add randomness on top. Never inside.

wait = retry_after + random(0, min(retry_after, 5 seconds))

A client that jitters downward from a stated wait sends a request the server has already promised to refuse. On a metered API that refusal is not free: it still counts against the ceiling it was refused by. See Retry-After for the header itself and exponential backoff for the curve underneath it.

Everywhere else jitter belongs

Retries are the famous case. They are not the most common one.

Scheduled jobs. If every tenant's nightly sync is configured for 00:00, you have built a load test that runs once a day. This one is concrete on a metered API: the daily quota on the Pinlyx Data API resets at 00:00 UTC, which is exactly the moment every quota-aware client is most tempted to start its batch. Spread the start across the first ten minutes and the entire population stops competing for the same first minute.

Cache expiry. A thousand keys written in the same deploy with the same TTL expire in the same second, and every subsequent miss stampedes the origin at once. Jitter the TTL: ttl = base_ttl * (1 + random(-0.1, 0.1)) turns a cliff into a slope.

Reconnect loops. A WebSocket or SignalR server that restarts disconnects every client in the same instant, and an un-jittered reconnect policy brings them all back in the same instant too, against a process that has not finished warming up. This is one of the few cases where the reconnect storm can prevent the recovery it is trying to detect.

Token refresh. Every instance refreshing its credential at exactly expiry minus 60 seconds produces a synchronised burst against the auth service, and they stay synchronised forever because the next expiry is the same for all of them.

Health checks and polling. Same shape, lower stakes, and trivially fixed by randomising the first interval.

Where jitter is the wrong tool

Randomness is not free everywhere. Three cases where adding it is a mistake:

Inside a user-facing latency budget. Turning a predictable 200 ms into a random 0 to 400 ms makes the p99 worse and the experience feel unreliable, without spreading any load that mattered.
When ordering matters. Randomised delays reorder work. If two operations on the same record must land in sequence, jitter the batch, not the individual items.
As a substitute for a concurrency limit. Jitter spreads a burst over time; it does not cap how much work is in flight. If your client can open two hundred sockets, jitter merely staggers when it does so.

Related concepts

Exponential backoff: the curve jitter is applied to.
Retry-After: the server's floor, which jitter is added above.
Rate limit: the ceiling whose reset boundary is the most synchronising clock in the system.
Webhook replay: bulk re-delivery is another place a fleet can be triggered in unison.
Flood wait: the same dynamic on Telegram, where synchronised sending is what triggers the wait in the first place.

How Pinlyx handles it

Pacing with deliberate variation is built into the sending side of the product, not just the API client: outbound messaging enforces a minimum interval between sends per account and varies it rather than firing on a metronome, because a perfectly regular send cadence is itself a signal platforms look for. On the API side, every 429 carries a Retry-After so clients have a real floor to jitter above, and the two 429 codes are separated so nobody jitters around a wait that will not clear until midnight.

Cheat sheet · four strategies, one point on the curve

base 500 ms, multiplier 2, fourth attempt.

The undecorated delay at that point is 4.000 seconds. Every figure in the third column is that number, transformed.

Strategy	Formula	At attempt 4	Character
No jitter	sleep = min(cap, base x 2^n)	exactly 4.000 s	Perfectly synchronised. Every client that failed together retries together.
Full jitter	sleep = random(0, min(cap, base x 2^n))	0 to 4.000 s, mean 2.000 s	Flattest load curve. Can produce a near-instant retry, which is wrong when the server is saturated.
Equal jitter	temp = min(cap, base x 2^n); sleep = temp/2 + random(0, temp/2)	2.000 to 4.000 s, mean 3.000 s	Keeps a guaranteed floor while still spreading. The safe default under load.
Decorrelated jitter	sleep = min(cap, random(base, previous_sleep x 3))	0.5 to 6.000 s from a 2 s previous sleep	Grows from the last actual sleep rather than the attempt number. Spreads across time as well as across clients.

Cheat sheet · all four, in code

TypeScript and Python. Fresh randomness on every attempt.

const BASE = 500;      // ms
const CAP  = 60_000;   // ms

const noJitter   = (n: number) => Math.min(CAP, BASE * 2 ** n);

const fullJitter = (n: number) => Math.random() * noJitter(n);

const equalJitter = (n: number) => {
  const temp = noJitter(n);
  return temp / 2 + Math.random() * (temp / 2);
};

// Decorrelated grows from the LAST SLEEP, not from the attempt number.
const decorrelated = (previous: number) =>
  Math.min(CAP, BASE + Math.random() * (previous * 3 - BASE));

// Retry-After is a floor. Jitter goes above it, never below.
const aboveFloor = (retryAfterMs: number) =>
  retryAfterMs + Math.random() * Math.min(retryAfterMs, 5_000);

import random

BASE, CAP = 0.5, 60.0          # seconds

def no_jitter(n):     return min(CAP, BASE * 2 ** n)
def full_jitter(n):   return random.uniform(0, no_jitter(n))

def equal_jitter(n):
    temp = no_jitter(n)
    return temp / 2 + random.uniform(0, temp / 2)

def decorrelated(previous):
    return min(CAP, random.uniform(BASE, previous * 3))

# Spreading a scheduled job instead of a retry: the daily quota resets at
# 00:00 UTC, so do NOT start there with everybody else.
def batch_start_offset(spread_seconds=600):
    return random.uniform(0, spread_seconds)

Watch out for

Six ways jitter fails to do anything.

One random offset drawn at startup and reused for every attempt. The population is shifted, not spread, and the spikes are the same height.
Jitter applied below a server-provided Retry-After, which produces requests the server has already refused.
Jitter on the retry but not on the schedule, so the batch that generated the retries still starts at 00:00 UTC with everyone else.
Full jitter under real saturation, where a near-zero sleep sends part of the fleet straight back at a server that has not recovered.
Jitter used instead of a concurrency cap. It staggers when work starts, not how much runs at once.
Jitter on cache TTLs forgotten, so a fleet of keys written together expires together and every miss reaches the origin.

Jitter: FAQ

Which strategy, how much, and every place besides retries that needs it.

Equal jitter is the safe default: it keeps a guaranteed minimum wait while still spreading clients across a window. Use full jitter when the failure is a transient blip and speed of recovery matters more than politeness, because it can produce a near-instant retry. Use decorrelated jitter when clients are long-lived and you want the spread to keep widening over time rather than resetting with each attempt number.

The three named strategies come from an AWS Architecture Blog post by Marc Brooker, "Exponential Backoff and Jitter" (2015), which simulated them against a contended resource. Its headline finding was that adding jitter reduced both total work done and time to completion compared with plain exponential backoff, with full jitter and decorrelated jitter performing comparably. The underlying idea is much older: Ethernet has randomised its binary exponential backoff since the original CSMA/CD design, for exactly the same reason.

Yes, and this is the mistake that quietly nullifies the whole technique. If a client picks one random offset at startup and applies it to every retry, the population is still perfectly synchronised, just shifted; every client keeps its position in the queue and the spikes are the same height, only displaced. Draw fresh randomness for each attempt.

For jitter, yes. You are spreading load, not generating a secret, and a predictable jitter value gives an attacker nothing worth having. Use a cryptographically secure generator for tokens, nonces and idempotency keys; for a sleep duration the ordinary pseudo-random generator is fine and much cheaper.

No. Retry-After is a floor. Jitter goes on top of it, never inside it: wait = retry_after + random(0, min(retry_after, 5 s)). Retrying before the stated time produces a request the server has already told you it will refuse, and on a metered API that refusal still consumes a unit of quota.

Anywhere a population of clients shares a clock. Scheduled jobs, where every tenant running a sync at 00:00 UTC is a self-inflicted spike. Cache expiry, where a fleet of keys written together expires together and every miss stampedes the origin. Health-check and polling intervals. WebSocket and SignalR reconnect loops after a deploy, which is the worst case of all because every client disconnected in the same second. And token refresh, where every instance refreshing at exactly expiry minus 60 seconds produces a synchronised burst against the auth service.

Keep reading

Exponential backoff

The curve jitter is applied to, with a computed schedule.

Retry-After

The floor to jitter above, in both of its legal formats.

Rate limit

The reset boundary is the most synchronising clock you will meet.

Flood wait

What a perfectly regular send cadence earns you on Telegram.

Webhook replay

Bulk re-delivery is another way to trigger a fleet in unison.

Data API reference

Rate headers, reset times and the codes behind each 429.

Ready to ship

Same average load. A fraction of the peak.

Every 429 from the Pinlyx Data API carries a real Retry-After, so your client has an honest floor to spread above instead of a number it had to guess.

Get an API key Read the docs

Free forever plan · GDPR-ready · No credit card required