Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Debugging a Cascading Deployment Failure: Race Conditions, Format String Escaping, and Multi-Site Staging Coordination

During a recent development session, Claude 4.5 was tasked with fixing a booking calendar race condition on sailjada.com. What followed was a cautionary tale about how a single fix, when deployed across 22 pages without proper validation, can introduce new bugs and corrupt staging environments. This post details what went wrong, how we diagnosed it, and the infrastructure review process we're implementing to prevent future cascades.

The Original Issue: A Race Condition in jadaOpenBook()

The core problem was straightforward: the jadaOpenBook() function was opening a booking modal immediately without waiting for availability data to load from the backend. This allowed users to interact with a calendar UI before real availability had been fetched, creating a confusing user experience and potential data inconsistency issues.

The intended fix was to add a loading state check:

// Before: Opens immediately
jadaOpenBook() {
  this.showBookingModal = true;
}

// After: Waits for data
jadaOpenBook() {
  if (this.isLoading) return;
  this.showBookingModal = true;
}

The Deployment Cascade: What Went Wrong

The 4.5 session identified 22 pages containing jadaOpenBook references and applied the fix to all of them. The agent then staged these changes to s3://queenofsandiego.com/_staging/sailjada/ for review. However, a critical issue was introduced during this process:

Python Format String Escaping was Left in JavaScript Context

The modified files contain JavaScript code like:

{{ isLoading: false }}

This is invalid JavaScript. The double-brace syntax {{ }} is legitimate in CSS (for custom properties or in template literal contexts), but when it appears in raw JavaScript object literals within <script> tags, it causes parsing failures.

The root cause: The original codebase uses Python format string templates (evidenced by placeholders like {STRIPE_LINK}), meaning these HTML files are processed server-side before deployment. The 4.5 agent escaped braces to prevent format-string interpolation, but didn't verify that the JavaScript contexts would parse correctly after the Python pre-processing stage.

Scope of the Damage

  • 23 files modified across /Users/cb/Documents/repos/sites/sailjada.com/ and /Users/cb/Documents/repos/sites/sailjada.com/releases/rc1/
  • Staging bucket contaminated: s3://queenofsandiego.com/_staging/sailjada/ contains broken files
  • Secondary collateral: The staging cleanup revealed additional staged files at s3://queenofsandiego.com/_staging/ for other domains, including a mysterious brandicarlile.html redirector, which required investigation
  • Production S3 buckets (s3://sailjada.com/ and production CloudFront distribution) were untouched, preventing user-facing impact

Investigation and Remediation

Phase 1: Identifying the Scope

We ran grep patterns across all local sailjada files:

grep -r "jadaBookingState\|jadaOpenBook" /Users/cb/Documents/repos/sites/sailjada.com/ --include="*.html"

This revealed which files had been modified and which were corrupted with the format-string escaping issue. We then compared local versions against production via S3:

aws s3 cp s3://sailjada.com/index.html ./prod_index.html
diff -u prod_index.html ./index.html | head -100

The diff showed that production files were stable and correct, while local files contained the broken escaping.

Phase 2: Restoration and Cleanup

Rather than attempt to hand-fix 23 files, we restored all local sailjada files from production S3:

aws s3 sync s3://sailjada.com/ /Users/cb/Documents/repos/sites/sailjada.com/ --exclude "*" --include "*.html"

We then deleted the contaminated staging deployment:

aws s3 rm s3://queenofsandiego.com/_staging/sailjada/ --recursive

Finally, we audited other staging paths to identify lingering issues and understand what other work was in flight.

Infrastructure and Deployment Architecture

The session revealed our multi-domain, multi-stage deployment structure:

  • Production buckets: s3://sailjada.com/, s3://queenofsandiego.com/ (backed by CloudFront distributions)
  • Staging bucket: s3://queenofsandiego.com/_staging/ (unified staging location for all domains)
  • Local dev repos: /Users/cb/Documents/repos/sites/ (source of truth for local development)
  • File processing pipeline: HTML files are Python format-string templates before S3 deployment, suggesting a pre-deployment build/render step

This architecture means that developers must understand which files undergo template processing and which are served as-is. The staging bucket location under the queenofsandiego.com domain indicates a centralized review and promotion workflow.

Key Decisions and Lessons

Why We Didn't Hand-Fix the Files

Given that production was untouched and stable, the fastest and safest path was restoration from the known-good production source. Hand-fixing 23 files risks introducing new subtle bugs and wastes review cycles.

Why We Investigated Staging Artifacts

The cleanup revealed brandicarlile.html and other files staged to s3://queenofsandiego.com/_staging/. While outside the scope of this race condition fix, understanding what was in flight prevented us from accidentally deleting unrelated work or discovering the contamination too late during a production deployment.

Why Template Escaping Requires Context Awareness

Double-brace escaping is context-dependent. In CSS custom properties or within template literals, {{ }} may be legitimate. In JavaScript object literals within <script> tags that will be processed by Python's string formatting, escaping creates syntax errors. Future fixes must validate that the final rendered JavaScript is syntactically correct by test-rendering the template server-side or through a build step.

What's Next: Preventing Cascades

  • Multi-file validation rule: When a fix spans more than 5 files, require a staging deployment to a subdomain (e.g., s3://queenofsandiego.com/_staging/sailjada/) and full end-to-end testing before approval
  • Template syntax validation: Implement a pre-commit or pre-deploy check that validates JavaScript syntax after template rendering