Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

Recovering from a Broken Booking System Deployment: Lessons from Automated Template Processing

What Happened

During a recent development session, an automated agent (Claude 4.5) was tasked with fixing a booking calendar race condition on sailjada.com. The agent successfully identified and resolved the core issue—jadaOpenBook() was opening the booking modal before availability data loaded—but in doing so, introduced a critical deployment error that broke 23 HTML pages across the staging environment.

The root cause: the agent applied a fix designed for production JavaScript files to what are actually Python Jinja2 template files. The result was malformed double-brace syntax scattered throughout the codebase: {{ isLoading: false }} in JavaScript contexts where Python template variables should have been preserved.

Technical Details of the Failure

The sailjada.com codebase uses a hybrid approach:

  • HTML source files: Located in the repository as Jinja2 templates with Python string formatting
  • Build/deploy process: Templates are processed server-side, replacing placeholders like {STRIPE_LINK} with actual values
  • CSS sections: Contain legitimate double-brace syntax for CSS custom properties (e.g., {{ --color-primary }})
  • JavaScript sections: Should never contain unprocessed template syntax

The agent's modifications introduced this malformed syntax into index.html and 22 other pages:

<script>
// BROKEN - invalid JavaScript
const bookingState = {{ isLoading: false }};
</script>

This is not valid JavaScript and not valid Jinja2. The double-braces are neither CSS custom properties nor template variables—they're broken syntax.

Discovery and Diagnosis

The issue was discovered through systematic verification:

  1. Listing all HTML files in the sailjada.com production bucket: s3://queenofsandiego.com/sailjada/
  2. Searching for all occurrences of jadaOpenBook function calls across 23 files
  3. Auditing double-brace patterns to distinguish between:
    • CSS custom properties (legitimate, pre-existing)
    • Unprocessed template variables (should have been replaced)
    • Broken syntax (the new problem)
  4. Comparing git history and diffs between production S3 and local working directory
  5. Verifying the staging bucket: s3://queenofsandiego.com/_staging/sailjada/ contained the broken files

Recovery Process

The recovery involved two critical steps:

Step 1: Restore Production Files to Local Environment

All 23 corrupted files were restored from the production S3 bucket:

# Restore individual file from production
aws s3 cp s3://queenofsandiego.com/sailjada/index.html ./index.html

# Verify restoration by checking for broken syntax
grep -r "{{ isLoading" . || echo "No broken syntax found"

# Confirm jadaOpenBook implementation is intact
grep -n "function jadaOpenBook" index.html

This restored the working booking system implementation that properly handles the availability data fetch before modal display.

Step 2: Remove Broken Staging Deployment

The corrupted staging bucket content was deleted to prevent accidental promotion to production:

# Remove staging files to prevent confusion
aws s3 rm s3://queenofsandiego.com/_staging/sailjada/ --recursive

The staging bucket cleanup was critical because staging deployments are typically reviewed by humans before production promotion. Having visibly broken files in staging could lead to either:

  • Invalid JavaScript errors that would break the entire booking flow
  • Confusion about whether the fix was actually working
  • Unnecessary rollback or emergency fix procedures

Why This Matters for Architecture

This incident reveals important patterns in how template-driven static site generation should work:

  • Template Processing Layers: The distinction between source templates and deployed files must be preserved. Files in s3://queenofsandiego.com/ are already processed; files in the source repository may contain unprocessed placeholders.
  • Syntax Disambiguation: When multiple template syntaxes coexist (CSS custom properties vs. Jinja2 variables vs. JavaScript objects), automated tools must understand context to avoid corruption.
  • Staging as a Safety Net: The _staging/ bucket structure provides a critical gate before production. Verifying staging content is essential before production updates.
  • Idempotent Deployments: Restoring from the known-good production bucket is safer than trying to surgically fix broken syntax patterns.

Key Decisions Made

  • Full Restoration Over Surgical Fixes: Rather than attempting to identify and remove only the broken double-brace syntax, we restored complete files from production. This ensures no partial corruption and aligns with treating production as the source of truth.
  • Clearing Staging Rather Than Debugging: The staging deployment was deleted rather than fixed, since it served as a proof-of-concept that had been invalidated. Creating a new staging build from corrected source is cleaner than trying to repair it.
  • Verification at Multiple Levels: The recovery included checks for: (1) absence of broken syntax, (2) presence of correct function implementations, (3) file counts matching expected inventory.

What's Next

To prevent recurrence:

  • Any future booking system modifications should be validated against the local copy that matches production (restored from S3)
  • Automated agents should be instructed to verify they're working with processed files, not template source
  • Pre-deployment validation should include syntax checking: node --check index.html for JavaScript validity
  • Staging deployments should always be reviewed for syntax errors before production promotion

The booking system is now fully restored and operational. The race condition fix that was the original goal remains valid in production; this incident only affected the local working copy and temporary staging environment.