GLOSSARY

What is an Idempotency Key?

An idempotency key is a client-generated unique string sent with a write request so the server can recognise a retry of that exact request and return the original result instead of performing the work a second time.

Try Pinlyx free Read the Data API docs

Free forever plan · No credit card required · Cancel anytime

Quick definition

In a single sentence: it is how a client says "this is the same request I sent you thirty seconds ago, do not do it again".

What it means

An idempotency key is a unique string the client attaches to a write request. If the client never hears back and sends the request again with the same key, the server recognises the key, skips the work, and replays the response it produced the first time. One logical operation, one effect, no matter how many times the bytes travel.

The problem it solves is narrower and nastier than "networks are unreliable". When a request times out, the client learns exactly one thing: it did not get a response. It cannot distinguish between three very different worlds. The request never arrived. The request arrived, was fully applied, and the response was lost on the way back. The request arrived and was rejected. Two of those three want a retry; one of them wants silence. Without a key, the client has to guess, and the usual guess (retry) is the one that double-charges the card.

HTTP already gives you this property for most methods. RFC 9110 section 9.2.2 defines GET, HEAD, PUT, DELETE, OPTIONS and TRACE as idempotent: sending the request twice has the same intended effect on the server as sending it once. POST is deliberately excluded, because POST means "process this payload however you see fit", and creating a resource twice creates two resources. The idempotency key is the industry's retrofit: a header that lets a POST behave like a PUT without the client having to invent the resource identifier up front.

The header and the value

The header name is Idempotency-Key. It is specified in the IETF HTTP API working group draft "The Idempotency-Key HTTP Header Field", which standardises what Stripe, Adyen, PayPal and a long tail of payment and provisioning APIs had already converged on independently. It is a draft, not a published RFC, so treat it as a strong convention rather than a guarantee that any given vendor implements it identically.

The value should be:

Client-generated. The client is the only party that knows two attempts are the same operation.
Unique per logical operation, not per HTTP request. Generate it once, before the first attempt, and hold it for the life of the operation including every retry.
Opaque and unguessable. A UUIDv4 or a ULID is the normal choice. 128 bits of randomness makes accidental collision a non-issue.
Not derived from the payload. This is the mistake worth spelling out. Two identical charges of 10 dollars a minute apart are two real operations, and a key that is a hash of the body would collapse them into one and quietly lose the second payment.

What the server has to do

The client side is one header. All of the difficulty lives on the server, and the correct algorithm is more than "have I seen this key". Reading in order:

Compute a fingerprint of the request: method, path, and a hash of the raw body. This is what catches a client reusing a key with a changed payload.
Attempt an atomic insert of (principal, key, fingerprint, state = in_progress) into a table with a unique constraint on (principal, key). The uniqueness has to be enforced by the database, not by a read-then-write in application code, because two concurrent retries will both pass the read.
If the insert succeeded, run the work. On completion, store the status code and the response body against the key and set the state to completed.
If the insert conflicted and the stored state is in_progress, answer 409 Conflict. The first attempt is still running; a second run would be the exact duplication the key exists to prevent.
If it conflicted, the state is completed, and the fingerprint matches, replay the stored status and body. No work runs.
If it conflicted and the fingerprint does not match, refuse. The draft's guidance is 422; Stripe answers 400 with an idempotency error type. Either is fine. Silently running the new request is not.

Two details make the difference between a correct implementation and one that looks correct in testing.

Scope the key namespace to the authenticated principal. A key store with a global unique index on the key alone means one tenant can, by collision or by malice, receive another tenant's stored response body. Key the table on (api_key_id, idempotency_key) and the whole class of cross-tenant leak disappears.

Store the response, not a boolean. A store that only records "this key is done" leaves the retrying client with a 200 and no body, which means it still cannot learn the id of the thing it created. Persist the status code and the serialised body. That is the entire point: the retry has to be able to recover the result the first attempt lost.

How long to keep the key

The retention window has to be longer than the longest retry ladder that can reach the endpoint. This sounds obvious and is routinely wrong in production. If your client library backs off for up to 24 hours and your key store expires at six hours, the last three retries in that ladder find no key and re-run the work. The key store silently stops being a safety net exactly when the outage has gone on long enough to need one.

Stripe retains keys for 24 hours. A reasonable rule is to take your ladder's total span, double it, and round up. For a delivery pipeline like the Pinlyx Data API's watch deliveries, which retries across 1m, 5m, 30m, 2h, 6h, 12h and 24h for a total span of 44 hours 36 minutes, a receiver that de-duplicates on the event id should keep those ids for seven days, not for one.

Where the Data API needs one, and where it does not

The Pinlyx Data API is read-mostly: 103 of its 127 endpoints are GET, and RFC 9110 has already made those safe to repeat. Most of the 24 POST endpoints are POST for a boring reason, which is that they need a request body rather than because they create anything: a bulk account check accepts up to 50 handles at once, and the analysis tools take a parameter object. Repeating one of those costs budget and returns the same answer. It cannot corrupt anything.

The endpoints that genuinely create state solve the problem a different way, with natural-key idempotence rather than a header:

POST /data-api/v2/watch/endpoints registers a webhook destination and mints its signing secret. Registering a URL that is already registered on the account returns 422 rather than reissuing the secret, precisely because reissuing it would break the integration already using the old one. The URL is the natural key.
POST /data-api/v2/watch registers an account to watch. Posting the same account to the same endpoint again updates its thresholds instead of creating a second watch. The pair (account, destination) is the natural key.

That is the design lesson worth taking away. An idempotency key is the general-purpose answer for operations with no natural identity, such as "charge this card". When the operation does have a natural identity, an upsert on that identity is simpler, needs no key store, and cannot expire.

Related concepts

Exponential backoff: the retry strategy the key makes safe to use.
Webhook replay: the same problem on the receiving side, keyed on the event id.
Retry-After: the server's instruction about when the next attempt should happen.
Webhook: the delivery mechanism whose at-least-once guarantee makes receiver-side idempotence mandatory.

How Pinlyx handles it

Every Data API response carries X-Request-Id, echoed in meta.request_id, which identifies the attempt rather than the operation. Write endpoints are idempotent on their natural keys as described above, and outbound webhook deliveries carry a stable event id so a receiver can de-duplicate a re-delivery against its own store. The pattern is consistent: the server never asks the client to trust that a retry is safe, it makes the retry safe.

Cheat sheet · the same request, twice

One key, two attempts, one effect.

The first attempt times out on the client. The second is byte-identical apart from nothing at all: same key, same body. Notice that the replayed response is the original 201, and that only the request id changes.

Attempt 1 (response never arrives)

POST /v1/charges HTTP/1.1
Host: api.example.com
Authorization: Bearer psk_live_...
Idempotency-Key: 018f3b2c-9a41-7cc1-b0f3-6de2a1c94f77
Content-Type: application/json

{"amount": 2499, "currency": "usd", "customer": "cus_9F2C41"}

--- server did this, client never saw it ---

HTTP/1.1 201 Created
X-Request-Id: req_9f2c41a8b3d5

{"id": "ch_01HQ7M1VYK", "amount": 2499, "status": "succeeded"}

Attempt 2 (same key, replayed result)

POST /v1/charges HTTP/1.1
Host: api.example.com
Authorization: Bearer psk_live_...
Idempotency-Key: 018f3b2c-9a41-7cc1-b0f3-6de2a1c94f77
Content-Type: application/json

{"amount": 2499, "currency": "usd", "customer": "cus_9F2C41"}

HTTP/1.1 201 Created
Idempotent-Replayed: true
X-Request-Id: req_c71b09de4a22

{"id": "ch_01HQ7M1VYK", "amount": 2499, "status": "succeeded"}

The charge id is the same because no second charge exists. The request id differs because it identifies the HTTP attempt, not the operation. Any header that flags a replay is vendor-specific: the draft does not standardise one, so read your provider's docs rather than assuming.

The four cases a server must handle

Anything less than four branches is a bug waiting for traffic.

Situation	Stored state	Correct response
Key never seen before	Row inserted, state = in_progress	Work runs. Response stored against the key, then returned.
Key seen, original still running	Row exists, state = in_progress	409 Conflict. The client waits and retries; it must not start a second run.
Key seen, finished, same request fingerprint	Row exists, state = completed	The stored status code and body are replayed byte for byte. No work runs.
Key seen, finished, different fingerprint	Row exists, fingerprint mismatch	422 Unprocessable Content (the IETF draft) or 400 with an idempotency error (Stripe). Never silently run the new request.

Cheat sheet · the store and the guard

PostgreSQL table plus a Node handler.

-- The unique constraint is the lock. Do not emulate it in application code:
-- two concurrent retries will both pass a read-then-write check.
CREATE TABLE idempotency_keys (
  api_key_id    BIGINT      NOT NULL,
  key           TEXT        NOT NULL,
  fingerprint   TEXT        NOT NULL,   -- sha256(method + path + raw body)
  state         TEXT        NOT NULL,   -- 'in_progress' | 'completed'
  status_code   INT         NULL,
  response_body JSONB       NULL,
  created_at    TIMESTAMPTZ NOT NULL DEFAULT now(),
  PRIMARY KEY (api_key_id, key)
);
CREATE INDEX ON idempotency_keys (created_at);   -- for the 72h reaper

async function withIdempotency(req, res, run) {
  const key = req.header("Idempotency-Key");
  if (!key) return res.status(400).json({
    error: { code: "invalid_request", message: "Idempotency-Key is required." },
  });

  const fingerprint = sha256(req.method + req.path + req.rawBody);

  const claimed = await db.tryInsert({
    apiKeyId: req.auth.id, key, fingerprint, state: "in_progress",
  });

  if (!claimed) {
    const row = await db.get(req.auth.id, key);
    if (row.state === "in_progress") return res.status(409).json({
      error: { code: "conflict", message: "A request with this key is in flight." },
    });
    if (row.fingerprint !== fingerprint) return res.status(422).json({
      error: { code: "invalid_request",
               message: "This Idempotency-Key was used with a different payload." },
    });
    // Replay the original answer, byte for byte.
    return res.status(row.statusCode).json(row.responseBody);
  }

  try {
    const { status, body } = await run();
    await db.complete(req.auth.id, key, status, body);
    return res.status(status).json(body);
  } catch (err) {
    // Release the claim: a transient failure must not pin the key forever.
    await db.release(req.auth.id, key);
    throw err;
  }
}

The catch block is the part that gets left out. If a 500 leaves the row in in_progress, every later retry of that key gets 409 forever and the operation can never be completed or abandoned.

Watch out for

Six ways an idempotency key stops working.

A new key generated inside the retry loop. Every attempt looks like a new operation and the key does nothing at all.
The key derived from the request body, so two identical-but-real operations collapse into one.
No fingerprint check, so a client that reuses a key with a changed payload gets the old answer for a new request.
The key stored only on success, so a crash mid-flight leaves nothing to recognise the retry.
Retention shorter than the retry ladder, so the safety net expires before the last attempt arrives.
A global key namespace instead of one scoped per API key, which turns a collision into a cross-tenant data leak.

Idempotency keys: FAQ

The questions that come up the first time a retry double-charges someone.

No, and mixing them up causes real bugs. A request ID is generated by the server, is different on every attempt, and exists so support can find one log line. An idempotency key is generated by the client, stays the SAME across every retry of one logical operation, and exists so the server can recognise the retry. In the Pinlyx Data API the server-side identifier is X-Request-Id, echoed as meta.request_id, and it changes on a replayed request even though the body does not.

No. Two legitimately identical requests must both go through: charging a customer 10 dollars twice in one minute is a real business case, and a body hash would collapse the second one into the first and silently lose money. Generate a fresh UUIDv4 or ULID per logical operation, hold it in the client for the lifetime of that operation, and reuse it only when retrying that operation. The body hash has a different job: it is the fingerprint the server compares to catch a key being reused with a changed payload.

Longer than the longest retry ladder that can reach it. Stripe keeps keys for 24 hours. If your own client retries for up to 24 hours, a 24 hour window is exactly on the edge and the last retry may find the key expired and re-run the work. Pick a retention window with headroom: 24 hours of retries wants a 72 hour key store. Storage is cheap, double-charging is not.

No. RFC 9110 section 9.2.2 already defines GET, HEAD, PUT, DELETE, OPTIONS and TRACE as idempotent methods, meaning repeating them has the same intended effect as making them once. The key exists to retrofit that property onto POST. Of the 127 endpoints in the Pinlyx Data API, 103 are GET and need nothing; the 24 POST endpoints are mostly reads that need a request body, such as a bulk check of up to 50 handles.

That is the case the in_progress state exists for. Write the key row and commit it BEFORE doing the work, in the same transaction as the work where your database allows it. If the process dies mid-flight the row is left in_progress, later retries get 409 until a reaper expires the row or the original transaction rolls back. The failure mode you must avoid is storing the key only after success, which turns a crash into a silent duplicate on the next retry.

Nothing makes delivery exactly-once over an unreliable network; that is a theoretical result, not a vendor limitation. What you can build is at-least-once delivery plus idempotent processing, which is observationally identical to exactly-once from the outside. The key is the second half of that sentence.

Keep reading

Exponential backoff

The retry strategy an idempotency key exists to make safe.

Webhook replay

Receiver-side de-duplication when the same event arrives twice.

Retry-After

How the server tells you when the next attempt should happen.

Webhook

At-least-once delivery, and why receivers must be idempotent.

Data API reference

The 127 endpoints, their envelope and their error codes.

Data API

Seven platforms, one envelope, one key.

Ready to ship

Retries that cost nothing. Writes that happen once.

The Pinlyx Data API is read-mostly by design, its write endpoints are idempotent on their natural keys, and every response carries a request id you can quote.

Get an API key Read the docs

Free forever plan · GDPR-ready · No credit card required