Architecture Autopsies
Case #04Silent Data Corruption7 min read

Autopsy #04: The Zombie Bundle After a Production Deployment

Production Symptom

Active tabs throw ChunkLoadError or corrupt local IndexedDB state after a production deploy.

The Incident & The False Culprit

At 10:00 UTC, CI deployed a routine workspace schema migration (v2) along with new content-hashed JavaScript bundles and pruned old v1 chunks from the CDN bucket. Fresh incognito smoke tests passed immediately.

Within 12 minutes, RUM telemetry and customer support escalated a multi-tab failure affecting long-lived sessions.

  1. T+00:00`workspace-app@2026.11.16.2` rolls out; CDN purges `v1` lazy chunks.
  2. T+04:00`ChunkLoadError` rate spikes to 4.8% among users who opened a second tab after the deploy.
  3. T+12:00310 support tickets report missing offline drafts after users clicked the Back button from an external OAuth redirect.
  4. T+18:00IndexedDB telemetry confirms `v1` tabs writing deprecated record shapes into `v2` object stores.
REJECTED PATCH

The Red Herring — Why the First Reflex Failed

The first Slack reflex was to call self.skipWaiting() inside the Service Worker install handler and attach a window.location.reload() listener on ChunkLoadError. Locally, hard-reloading a single tab cleared the error.

In production, that patch worsened the outage. skipWaiting() hijacked active tabs mid-transaction, while back-button navigations still restored frozen v1 JavaScript heaps from bfcache without triggering a network fetch or Service Worker intercept, writing legacy records into the upgraded v2 database.

Causal Chain Traversed

Step-by-step propagation across atomic concepts. Hover or click any step to inspect the underlying mechanism.

  1. 01Network & Performance
    Service Worker Lifecycle

    An outdated Service Worker keeps serving stale HTML and chunk manifests to open tabs during deploy.

  2. 02Network & Performance
    HTTP Caching Directives

    Immutable asset headers and stale-while-revalidate HTML responses split the module graph across versions.

  3. 03JS Runtime
    ES Modules and Live Bindings

    Cached ESM modules retain live bindings to stale exports while new dynamic imports fail or mismatch.

  4. 04Network & Performance
    Back/Forward Cache (bfcache)

    Restoring a page from the back-forward cache resumes an old JS runtime after IndexedDB already upgraded.

  5. 05State & Architecture
    Client Storage Versioning & Zero-Downtime Migrations

    An older tab writes records using a deprecated schema after another tab bumped the IndexedDB version.

A deploy skew incident is a five-layer state divergence across worker registration, HTTP cache headers, the ECMAScript module map, browser history snapshots, and client-side storage schemas.

  1. The active Service Worker serves the cached v1 index.html manifest to existing tabs.
  2. CDN Cache-Control rules either 404 pruned chunks or serve mixed-version assets.
  3. The browser ESM module map preserves live bindings to v1 exports alongside v2 dynamic imports.
  4. bfcache restores a frozen v1 heap after another tab upgraded the origin's database.
  5. The zombie v1 runtime writes unmigrated records into a v2 IndexedDB object store.

Failure Reproduction Lab

Step through the five-stage deploy skew sequence in the simulator below. Observe how tabAVersion, tabBVersion, and idbVersion diverge when Tab B upgrades IndexedDB while Tab A remains pinned to v1 by the Service Worker and bfcache.

Interactive Simulator

Zombie Bundle: Deploy Skew & IndexedDB Corruption

Step through how a live deploy splits HTML, ESM chunks, bfcache snapshots, and IndexedDB schema versions across open tabs.

SW v1 (Active)
CDN v2
IndexedDB v1
Step 1 / 5: 1. Service Worker & HTML Skew

Deploy v2 lands on the CDN. Tab A still runs v1 under the active Service Worker while Tab B loads v2.

// sw-v1.js still serves cached index.html (referencing chunk-v1.js)
Internal State / Memory
tabAVersion
v1 (SW controlled)
tabBVersion
v2 (fresh load)
idbVersion
1

Layer-by-Layer Dissection

01 → 02

Service Worker Ownership Meets HTTP Cache Skew

Under the Service Worker Lifecycle, a newly deployed sw.js enters waiting until every client controlled by v1 unloads. While v1 controls Tab A, it intercepts navigation requests and returns cached index.html referencing v1 chunk hashes.

When traffic hits the CDN edge directly, HTTP Caching Directives compound the split:

  • index.html with stale-while-revalidate returns yesterday's manifest during the stale window.
  • Hashed chunks carry Cache-Control: public, max-age=31536000, immutable, so the browser skips revalidation.
  • Once the CDN prunes old build artifacts, v1 HTML requests a v1 lazy chunk and receives HTTP 404.
02 → 03

HTTP Split Corrupts Live ESM Bindings

When a dynamic import() resolves against a newer build while a shared runtime chunk resolves from cache, the module loader does not clone values. Under ES Modules Live Bindings, every importer holds a read-only reference to the exporter's live memory slot.

If an unhashed entrypoint survives in cache while a feature chunk loads from v2, v2 code reads v1 schema constants or throws SyntaxError on missing named exports during module linking.

03 → 04

Module Skew Survives Inside bfcache

When a user navigates to an external OAuth page, the browser snapshots the entire v1 JS heap and DOM into memory. Under bfcache Navigation, pressing Back restores that snapshot synchronously (pageshow with event.persisted === true).

No HTTP request fires and no Service Worker fetch event executes. Meanwhile, Tab B has already booted v2 from the network and executed onupgradeneeded on the shared origin database.

04 → 05

Zombie Runtime Corrupts Client Storage Migrations

IndexedDB is origin-scoped, not tab-scoped. When Tab B bumps the database version from 1 to 2, Client Storage Schema Migrations require every active tab to close its handle on versionchange before the upgrade proceeds.

A bfcache-restored tab that ignored versionchange still executes v1 serialization code. Opening a new readwrite transaction against the upgraded v2 store writes v1-shaped objects that violate v2 index key paths and corrupt shared origin state.

Architectural Fix & Invariants

TS
const APP_VERSION = '2026.11.16.2';

window.addEventListener('pageshow', (event) => {
  if (event.persisted && localStorage.getItem('active_build') !== APP_VERSION) {
    window.location.reload();
  }
});

const openReq = indexedDB.open('workspace_db', 2);
openReq.onsuccess = () => {
  const db = openReq.result;
  db.onversionchange = () => {
    db.close();
    window.location.reload();
  };
};

Architectural Invariants:

  • Immutable chunks, zero-stale HTML: Serve index.html with Cache-Control: no-cache and retain previous build chunks on the CDN for at least 24 hours after deploy.
  • No blind skipWaiting(): Activate a waiting Service Worker only when no active transactions are open, then reload all controlled clients atomically.
  • Mandatory versionchange teardown: Implement onversionchange on every IDBDatabase handle to call db.close() and halt writes from outdated bundles.
  • Revalidate on persisted restore: Compare the running APP_VERSION against the active origin version on every pageshow event where event.persisted === true.

Golden Rule: A frontend deploy is never atomic across open tabs and bfcache; fence every storage write behind a versioned onversionchange and pageshow check.

Primary Sources

Last verified: 2026-11-16