Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Preventing S3 Deployment Regressions: Hard Rules for Multi-Environment CloudFront Workflows

Over a three-hour development session, a regression in our queenofsandiego.com production deployment wiped three critical frontend features: the hero JADA→BOOK NOW crossfade animation, the Stripe embedded checkout booking flow, and accidentally resurrected a deleted "For Ranch & Coast readers..." hero line. The root cause was deploying a stale local index.html over a newer S3 production version—a pattern that emerges when developers skip the pull-and-diff step before editing, or deploy staging and prod in a single command. This post documents the hard rules we've now encoded to prevent that class of failure.

The Failure Pattern

The incident followed a common workflow mistake:

  • Stale local state: The developer's local /Users/cb/Documents/repos/sites/queenofsandiego.com/index.html was several commits behind S3 production, which held the working Stripe checkout and hero animation.
  • No pre-edit diff: Before modifying the file, no aws s3 cp pull and diff check was performed to establish the baseline.
  • Dual-target deploy: A single cp command deployed to both staging and prod paths in one AWS S3 operation, violating the staging-first rule.
  • Ignored prior warnings: The developer's own session summary had flagged the stale local file risk; the warning was not heeded.

Result: three unrelated features regressed, and the CloudFront distribution cache had to be invalidated to serve the corrected version from S3.

Hard Rules Now Encoded (D1–D8)

We've formalized eight rules into /Users/cb/Documents/repos/sites/queenofsandiego.com/CLAUDE.md, auto-loaded on every QOS session:

  • D1 — Pull and Diff Before Edit: Before modifying any S3-backed file, always run aws s3 cp s3://sailjada-web/prod/index.html ./index.html.prod-snapshot and diff -u index.html.prod-snapshot index.html to confirm local state vs. production.
  • D2 — Single-Target Deploys: Never deploy staging and prod in one command. Always deploy to staging first, verify in a browser or via curl, then deploy prod separately with explicit path targets.
  • D3 — One Feature Per Commit: Each logical change (e.g., "add Stripe checkout flow" or "update hero animation") gets its own file edit and deploy, never bundled with unrelated fixes.
  • D4 — Obey Your Own Session Warnings: If a prior session-end summary flags a risk (e.g., "local is stale"), treat it as a blocking checklist item. Escalate or resolve before proceeding.
  • D5 — Snapshot Prod Before Overwrite: S3 versioning is not enabled on our buckets. Before cp to prod, save the current prod file locally as index.html.prod-before-$(date +%s) and verify it in git status.
  • D6 — Proof Block in Chat: Before executing any cp to prod, print a six-line proof block in chat showing: source file path, target S3 path, source file hash (shasum), timestamp, a one-line description of the change, and explicit confirmation of no dual-target syntax.
  • D7 — Feature-Token Registry: Maintain a grep-able comment block in index.html listing all live features (e.g., /* FEATURE: stripe-embedded-checkout v2.1 */, /* FEATURE: hero-jada-fade v1.3 */). Before deploying, diff the feature-token section against the S3 prod version to catch unintended removals.
  • D8 — Escalate on S3 Drift: If S3 production is ahead of your local branch (confirmed via D1 diff), stop. Do not edit. Escalate to CB with the diff output. Either rebase local, or revert S3 to match local baseline, explicitly.

Technical Implementation

Rules D1 and D5 are implemented as shell functions in the project's CLAUDE.md: 


# D1: Pull prod snapshot and diff
aws s3 cp s3://sailjada-web/prod/index.html ./index.html.prod-snapshot
diff -u index.html.prod-snapshot index.html | head -50

# D5: Create timestamped backup before overwrite
cp index.html "index.html.prod-before-$(date +%s)"
git status | grep "prod-before"  # Confirm backup in staging area

Rule D6 requires this proof block to appear in chat before any aws s3 cp to prod: 


SOURCE:     /Users/cb/Documents/repos/sites/queenofsandiego.com/index.html
TARGET:     s3://sailjada-web/prod/index.html
HASH:       $(shasum index.html | cut -d' ' -f1)
TIMESTAMP:  $(date -u +%Y-%m-%dT%H:%M:%SZ)
CHANGE:     [One-line description, e.g., "Update hero animation timing"]
DUAL-TARGET CHECK: No. Single target only.
READY TO DEPLOY:   Yes / No

Rule D7 means the <head> of index.html now includes: 


<!--
LIVE FEATURES (grep-able for regressions):
FEATURE: stripe-embedded-checkout v2.1
FEATURE: hero-jada-fade v1.3
FEATURE: crew-booking-form v1.0
DO NOT REMOVE without explicit D3 commit.
-->

Before any prod deploy, we diff this block: 


grep "FEATURE:" index.html.prod-snapshot > /tmp/prod-features
grep "FEATURE:" index.html > /tmp/local-features
diff /tmp/prod-features /tmp/local-features

Infrastructure Context

The sailjada.com and queenofsandiego.com sites are served from:

  • S3 Buckets: sailjada-web (primary), with prefixes /prod/, /staging/, and /dev/.
  • CloudFront Distributions: Separate distributions for prod and staging, with cache behavior rules that set TTL = 300 seconds for HTML, 86400 for static assets.
  • Invalidation: After any prod deploy, we issue aws cloudfront create-invalidation --distribution-id <DIST_ID> --paths "/*" to flush the cache immediately.
  • Route53: DNS records alias sailjada.