Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Preventing S3 Deployment Regressions: Hard Rules for Multi-Environment Static Site Pushes

Last week, a routine deployment to queenofsandiego.com's production S3 bucket accidentally wiped three working features by pushing a stale local index.html over a newer version already in S3. The hero image crossfade (JADA → BOOK NOW), Stripe embedded checkout flow, and a previously-removed marketing line all disappeared. The root cause wasn't technical complexity — it was process breakdown: no pre-flight diff, simultaneous staging + prod deployment in one command, and ignoring a prior session-summary warning about local file staleness.

Here's how we fixed it, and the hard rules we codified to prevent it happening again.

What Went Wrong

  • Stale local copy: The local /Users/cb/Documents/repos/sites/queenofsandiego.com/index.html (3,650 lines) was several commits behind the version already deployed to S3.
  • No pre-flight snapshot: Production state was never pulled or diffed before overwriting.
  • Dual-target deploy: A single aws s3 cp command pushed to both staging and prod buckets in the same breath, violating the staging-first validation rule.
  • Ignored warnings: The prior session summary explicitly noted "stale local files risk" — the warning was in the context but not acted on.

The Eight Hard Rules We Codified

These rules are now baked into /Users/cb/Documents/repos/sites/queenofsandiego.com/CLAUDE.md, which auto-loads on every session for that project. A condensed pointer also went into the top-level /Users/cb/Documents/repos/CLAUDE.md to catch non-QOS deployments.

D1: Pull and diff S3 before any local edit

  • Before modifying a file, pull the current version from its S3 bucket and diff against your local copy.
  • Example: aws s3 cp s3://queenofsandiego.com/index.html ./index.html.prod then diff -u index.html.prod index.html | head -50
  • If prod is ahead: escalate to CB (see D8) before proceeding.

D2: Staging only, single logical change per deploy

  • Every deploy targets staging first and only one resource at a time.
  • Example: aws s3 cp index.html s3://staging.sailjada.com/index.html — never s3://staging.sailjada.com/ s3://sailjada.com/ in the same command.
  • Wait for manual review (or automated smoke test) before promoting to prod.

D3: One file, one logical change per session segment

  • Don't batch unrelated edits into a single file (e.g., fixing hero fade + updating Stripe key + tweaking footer in one index.html diff).
  • Separate commits, separate deploys, separate review gates.

D4: Obey your own prior session-summary warnings

  • If the last session summary said "local files may be stale" or "S3 is ahead," treat it as a blocking condition until you've pulled and verified.
  • Don't override it based on confidence or time pressure.

D5: Snapshot prod before overwriting (no S3 versioning fallback)

  • S3 buckets for these projects do not have versioning enabled. Once you cp over a file, the old version is gone.
  • Before any cp to prod, manually copy the current prod version to a timestamped backup key: aws s3 cp s3://queenofsandiego.com/index.html s3://queenofsandiego.com/backups/index.html.$(date +%s)
  • Keep the last 3 backups in s3://queenofsandiego.com/backups/.

D6: Six-line proof block before any cp command

  • Print the exact command you're about to run, the source file hash, the target bucket, and the number of lines being changed.
  • Example:
  • # DEPLOY: index.html → s3://queenofsandiego.com/index.html
    # LOCAL HASH: $(sha256sum index.html | cut -d' ' -f1)
    # PROD HASH: $(aws s3api head-object --bucket queenofsandiego.com --key index.html --query 'Metadata' 2>/dev/null || echo 'PROD NOT FOUND')
    # LINES CHANGED: $(diff -u index.html.prod index.html | grep -c '^[+-]' || echo '0')
    # TARGET: s3://queenofsandiego.com/index.html
    # DEPLOY
  • Paste this block in chat before running the command. CB or the reviewing agent reads it and approves or halts.

D7: Feature-token registry: grep S3-current against known tokens

  • Maintain a FEATURE_TOKENS.md file listing regex or literal strings for each active feature:
  • Example: JADA_CROSSFADE="crossfadeHero|JADA.*BOOK NOW"
  • Before promoting to prod, grep the staged file: grep -E "$JADA_CROSSFADE" index.html && echo "✓ JADA feature present" || echo "✗ JADA feature MISSING"
  • If any critical token is missing, do not promote.

D8: Escalate to CB if S3 is ahead of local

  • If the prod bucket version is newer than your local copy, or if you cannot cleanly merge, stop and ping CB.
  • Message: "Prod is ahead of local for [filename] in [bucket]. Last prod SHA: [hash]. Local SHA: [hash]. Awaiting direction."
  • Never force-overwrite prod when you're unsure of the canonical source.

Infrastructure Context

  • S3 buckets: queenofsandiego.com (prod), staging.queenofsandiego.com (staging), both with CloudFront distributions in front.
  • CloudFront invalidation: After any prod cp, run aws cloudfront create-invalidation --distribution-id [DIST_ID] --paths "/*" to clear edge caches within ~30 sec.
  • Backup bucket: Same region, same AWS account, same permissions as prod. No public access.
  • Local repo: /Users/cb/Documents/repos/sites/quee