Autopsy #01: Anatomy of a Double Payment in Production
Rapid double-click or 3G network blip charges the user twice despite a disabled={isSubmitting} React button.
The Incident & The False Culprit
On Friday at 14:10 UTC, a two-line PR replaced a native <form> submit with a custom onClick handler on the checkout button to support Apple Pay telemetry. QA verified the happy path on desktop Chrome over office fiber and approved the release.
Within 40 minutes of the deploy, finance alerts flagged duplicate settlements on mobile Safari and throttled Android devices.
- T+00:00Deploy `checkout-web@4.18.2` reaches 100% of edge PoPs.
- T+08:00Support receives the first ticket reporting two \$99.00 charges for a single `cart_id`.
- T+19:00Payment gateway logs show 0.42% of `POST /v1/charges` requests arriving in pairs spaced 80 ms to 3,400 ms apart, both returning `200 OK` with distinct `charge_id` values.
- T+42:00Refund webhooks lag 48 hours behind support volume; checkout traffic is rolled back to `4.18.1`.
The Red Herring — Why the First Reflex Failed
The on-call engineer's first Slack reflex was to add disabled={isSubmitting} on the <button> element and guard the handler with if (isSubmitting) return;. The patch passed every local Playwright test on localhost with zero duplicate requests.
In production, the patch failed to reduce the 0.42% duplicate rate for two reasons:
- Async render lag:
setIsSubmitting(true)schedules a React render for the next frame;button.disabledstaysfalsewhen a secondclickenters the call stack 12 ms later. - Premature timeout reset: A 3-second client timeout in
catchresetisSubmittingtofalseon slow 3G connections while the firstPOSTsocket remained open in flight.
Causal Chain Traversed
Step-by-step propagation across atomic concepts. Hover or click any step to inspect the underlying mechanism.
- 01DOM & EventsDOM Event Propagation (Capture, Target, Bubble)
Two native click events fire synchronously in the same frame before React commits disabled={true} to the DOM.
- 02State & ArchitectureDebounce, Throttle & rAF Coalescing
Uncoalesced click handlers invoke submitPayment() twice in consecutive tasks instead of locking on the leading edge.
- 03State & ArchitectureFinite State Machines for UI
Boolean flags (isLoading, hasError) permit re-entry after a timeout while the first request remains in flight.
- 04JS RuntimeAbortController and Cooperative Cancellation
Missing AbortSignal leaves orphaned POST streams active on the network socket after UI reset or retry.
- 05State & ArchitectureOutbox Queues, Idempotency Keys & Causal Ordering
Absent Idempotency-Key headers cause the payment gateway to treat retried TCP packets as two distinct charges.
The duplicate charge requires five consecutive layer failures. A synchronous DOM event outruns asynchronous React state, uncoalesced callbacks fire two network requests, boolean flags allow re-entry after a timeout, uncancelled sockets finish in the background, and a stateless server commits both payloads.
Failure Reproduction Lab
Step through the 5-stage timeline below to inspect how two clicks inside one 16 ms frame become two settled charges, then advance to Step 5 to verify how a synchronous finite state machine and Idempotency-Key collapse the race into a single charge.
Double-Submit Race: Unbounded Clicks vs FSM + Idempotency Key
Trace how two pointer clicks inside one 16 ms frame bypass React's disabled prop and settle as two distinct Stripe charges.
Two rapid pointerup/click events dispatch synchronously on the <button> before React schedules a commit. Both handlers read isSubmitting === false from the current closure.
button.addEventListener('click', onPay);
// Click #1 & #2 both enter handler- frameMs
- 0.4
- reactDisabledProp
- false
- inFlightRequests
- 0
- ledgerCharges
- 0
Layer-by-Layer Dissection
Synchronous DOM Dispatch Outruns Async React Commit
In DOM Event Propagation (Capture, Target, Bubble), Blink dispatches click events synchronously from the native input queue into the JavaScript call stack. When onClick invokes setIsSubmitting(true), React enqueues a state update for the next render pass while button.disabled remains false during the current frame.
A second tap arriving 12 ms later hits the same active DOM node. Because Debounce, Throttle & rAF Coalescing is absent on the leading edge, both closures read isSubmitting === false and enter submitPayment().
Uncoalesced Handlers Feed Re-Entrant Boolean State
Even when a leading-edge lock blocks sub-second double-clicks, cellular handoffs introduce a 3.2-second TLS or gateway stall. Under Finite State Machines for UI, a SUBMITTING state cannot transition back to IDLE on a client-side timer while the HTTP socket remains open.
Boolean flags (isSubmitting, hasTimedOut) violate that invariant. A catch block executes setIsSubmitting(false) at t = 3,000 ms, re-enabling the button while Request #1 is still traversing the carrier network.
Re-Enabled UI Leaves Orphaned Sockets on the Wire
When the user taps "Pay" a second time, Request #2 enters the network stack alongside Request #1. Without AbortController and Cooperative Cancellation, the client never signals intent to drop Request #1, and an aborted fetch() only closes the local socket after the server may have parsed the TCP payload.
- Client view: Marks Request #1 failed at
t = 3,000 msand dispatches Request #2. - Server view: Receives Request #1 at
t = 3,150 msand Request #2 att = 3,420 ms.
Stateless Retries Commit Twice Without Idempotency Keys
Both HTTP requests reach the payment endpoint as independent mutations. As documented in Outbox Queues, Idempotency Keys & Causal Ordering, HTTP POST is non-idempotent by specification; only an explicit Idempotency-Key header scoped to the checkout intent lets the gateway deduplicate Request #2 and replay Request #1's receipt.
Architectural Fix & Invariants
type PayState = 'idle' | 'submitting' | 'settled' | 'failed';
export function createPaymentSubmitter(intentUuid: string) {
let state: PayState = 'idle';
const controller = new AbortController();
return async function submit(payload: unknown): Promise<Response | null> {
if (state === 'submitting' || state === 'settled') return null;
state = 'submitting';
try {
const res = await fetch('/v1/charges', {
method: 'POST',
headers: { 'Content-Type': 'application/json', 'Idempotency-Key': intentUuid },
body: JSON.stringify(payload),
signal: controller.signal,
});
state = res.ok ? 'settled' : 'failed';
return res;
} catch (err) { state = 'failed'; throw err; }
};
}Architectural Invariants:
- Synchronous transition guard: Mutate
idle → submittingsynchronously before anyawaitor React render commit to block same-frame double-clicks. - Stable intent key: Generate
intentUuidonce per checkout form mount and reuse it across all retries so the gateway deduplicates re-submissions. - Terminal settled state: Treat
settledas final for a givenintentUuid; starting another payment requires a fresh mount and UUID. - Cooperative teardown: Call
controller.abort()on unmount so stale responses never mutate unmounted UI state.
Golden Rule: Never guard a financial mutation with React's asynchronous render state; lock synchronously in memory and deduplicate idempotently on the server.
Primary Sources
Last verified: 2026-11-16