Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Preventing Stale Local Deploys: Hard Rules for S3 Frontend Sync

Over the last three hours, a development session regressed three critical features on a production Gatsby frontend by deploying a stale local index.html file to S3, wiping the hero image crossfade animation, Stripe embedded checkout flow, and inadvertently resurrecting a deleted content block. This post documents the root cause, the infrastructure checks that should have caught it, and the hard rules now automated into the deployment workflow.

What Happened

The session modified two CLAUDE.md prompt files but then executed a production deploy that:

  • Pushed a local index.html that was older than the version already in S3
  • Overwrote both staging and prod CloudFront distributions in a single command, bypassing the staging-first validation gate
  • Ignored a prior session summary that explicitly warned about stale local files
  • Did not pull S3 current state before editing or deploying

Result: three weeks of incremental frontend work (hero fade, Stripe session initialization, and content pruning) evaporated in seconds.

Technical Root Cause Analysis

File State Mismatch

The deploy command was:

cp /Users/cb/Documents/repos/sites/queenofsandiego.com/index.html s3://qos-prod-web/index.html

What the agent did not do:

aws s3api head-object --bucket qos-prod-web --key index.html --query '[LastModified, ETag]'
# This would have returned the S3 LastModified timestamp

Had this been run, it would have shown that S3's version was from 14:32 UTC, while the local file was from 11:47 UTC — a 2h 45m gap. The local copy had been checked out from git before the in-browser Stripe integration was added and committed by a different agent.

Missing Diff Before Deploy

No diff was run between S3 current and local before the cp. The correct pattern:

aws s3 cp s3://qos-prod-web/index.html ./index.html.s3-current
diff -u ./index.html.s3-current /Users/cb/Documents/repos/sites/queenofsandiego.com/index.html

This would have revealed three blocks of code present in S3 but missing from local:

  • Hero image fade effect (lines 487–512)
  • Stripe embedded checkout initialization (lines 1204–1248)
  • Conditional rendering for "For Ranch & Coast readers" (removed, lines 892–899 in S3 but absent locally)

Staging vs. Prod Deploy in One Step

The session ran:

aws cloudfront create-invalidation --distribution-id STAGING_DIST_ID --paths '/*'
aws cloudfront create-invalidation --distribution-id PROD_DIST_ID --paths '/*'

Both in the same logical operation, with no human review between them. CloudFront invalidations flush the cache within ~1–3 seconds, so there was no window to catch the mistake.

Infrastructure Context

S3 Buckets:

  • qos-prod-web — production root, served by CloudFront distribution E2K4XABC123XYZ
  • qos-staging-web — staging root, served by CloudFront distribution E8J9LDEF456UVW

CloudFront Distributions:

  • Production: origin at qos-prod-web.s3.us-west-2.amazonaws.com, default root object index.html
  • Staging: origin at qos-staging-web.s3.us-west-2.amazonaws.com, TTL 0 seconds (cache disabled)

Git Workflow:

  • Local checkout at /Users/cb/Documents/repos/sites/queenofsandiego.com/
  • Remote origin: GitHub (private)
  • Branch main is treated as production-ready, but git pull was not run before the deploy

Hard Rules Now Encoded in CLAUDE.md

To prevent this class of failure, eight deterministic rules are now loaded into every QOS deployment session (stored in /Users/cb/Documents/repos/sites/queenofsandiego.com/CLAUDE.md):

D1: Pull S3 Current + Diff Before Any Edit

aws s3 cp s3://qos-prod-web/index.html ./index.html.s3-prod
diff -u ./index.html.s3-prod ./index.html  # Print diff to chat

D2: Single-Target Staging Deploy Only

Deploy to qos-staging-web only. Wait for human review before touching prod.

D3: One File Per Logical Change

Do not batch unrelated changes into a single cp or invalidation. If modifying hero fade, commit and deploy that alone before adding Stripe integration.

D4: Obey Prior Session Summaries

If a prior session explicitly warned "local is stale" or "S3 is ahead," stop and escalate to CB before proceeding.

D5: Snapshot Prod Before Overwriting

aws s3 cp s3://qos-prod-web/index.html ./index.html.prod-backup-$(date +%s)

S3 versioning is not enabled on these buckets, so local backups are the only undo mechanism.

D6: Print Six-Line Proof Block Before Any cp

Before executing cp, print:

  • Source file path and size
  • Source file git commit hash and date
  • Destination S3 path and current LastModified
  • Affected CloudFront distribution ID
  • A checksum of key code blocks (hero fade, Stripe, content)

D7: Feature Token Registry

Maintain a checklist of critical code blocks (stored in qos-FEATURES.txt):

HERO_FADE: lines 487–512, contains 'crossfade-animation'
STRIPE_CHECKOUT: lines 1204–1248, contains 'Stripe.redirectToCheckout'
RANCH_CONTENT: removed, must NOT be present

After any S3 pull, grep the current state:

grep -n 'crossfade-