Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Diagnosing and Remediating Multi-Site Deployment Pipeline: OAuth Token Rotation, CloudFront Cache Invalidation, and Daemon Health Verification

This session involved systematic troubleshooting across three distinct infrastructure domains: orchestrator daemon health on AWS Lightsail, Google Analytics API authentication failures, and a multi-site static content deployment pipeline. The work exposed authentication drift, cache coherency issues, and provided actionable patterns for future deployments.

What Was Done

  • Verified jada-agent orchestrator daemon health on Lightsail instance 34.239.233.28 via temporary SSH credentials from Lightsail API
  • Diagnosed and documented a broken Google OAuth token affecting port_sheet_sync.py (failing every 30 minutes with HTTP 400)
  • Refactored site directory structure: renamed /repos/sites/86dfrom.com → /repos/sites/86from.com to correct semantic naming
  • Created new SEO landing page at /repos/sites/86from.com/site/what-does-86d-mean
  • Deployed corrected booking widget JavaScript to staging S3 bucket after fixing template syntax errors (double-brace Jinja2 conflicts)
  • Invalidated CloudFront distribution caches for both production and staging environments
  • Collected comprehensive GA4 analytics for 86from.com property across 7-day window

Technical Details: Daemon Health Assessment

The jada-agent service is fundamentally healthy but revealed two operational patterns worth documenting:

Service State (as of 2026-05-13 UTC)

  • jada-agent.service is active and running, uptime 3 days (since May 10)
  • Instance uptime: 11 days; load average 0.00 indicates the daemon is between task cycles
  • CPU utilization: ~0.65% average across last 2 hours (normal for 60-second polling interval)
  • Memory footprint: 144MB of 914MB available — well within safe operating bounds
  • Disk usage: 6.2GB of 39GB (17%) — no pressure on storage
  • AWS status checks: 0 failures in last 2 hours

Session Activity Pattern

The daemon executed three sessions on 2026-05-13:

  • Session 1 (00:00 UTC): Reached Claude turn limit (30) with exit code 1. This is logged as an error but is not a crash; the daemon continues normally.
  • Session 2 (00:02 UTC): Completed successfully, processed blockers for e-signature and crew page functionality, created a needs-you task for manual review
  • Session 3 (00:05 UTC): Hit max turns again, exit code 1. No additional tasks queued afterward.

The pattern of hitting max turns in complex reasoning tasks is expected behavior at the current task scope. Sessions that complete (like Session 2) demonstrate the daemon is functional and picking up work from the queue correctly.

Critical Finding: Google OAuth Token Expiration in port_sheet_sync

The port_sheet_sync.py script has been failing every 30 minutes with:

[port-sheet] token error: HTTP Error 400: Bad Request

Root Cause: The Google OAuth refresh token stored for the port sheet sync operation is expired or revoked. This is a common failure mode when:

  • OAuth tokens are not rotated on a predictable schedule
  • Credentials are invalidated on the OAuth provider side (account security event, consent revocation)
  • The service account or user account lost necessary scopes

Remediation Path: The token will need to be re-authenticated. The auth_ga.py utility in /repos/tools/ provides a pattern for OAuth token refresh. Given that the dangerouscentaur@gmail.com account has valid GA4 credentials already stored (verified during this session), the port sheet sync token should be re-created using the same authentication flow, storing the new refresh token securely in the secrets directory.

Infrastructure: Multi-Site Deployment Pipeline

Directory Restructuring

Renamed /Users/cb/Documents/repos/sites/86dfrom.com → /Users/cb/Documents/repos/sites/86from.com. This corrects the semantic meaning: "86 from X" (a phrase meaning to exclude) rather than "86 d from" (malformed). All subsequent deployments reference the corrected path.

Content Updates

  • Created /repos/sites/86from.com/site/what-does-86d-mean as a new SEO landing page targeting keyword variations
  • Fixed booking widget JavaScript in /repos/sites/sailjada.com/index.html — replaced Jinja2 template syntax conflicts (double braces {{ / }}) with single braces to prevent conflicts with JavaScript template literal parsing
  • Updated /repos/sites/queenofsandiego.com/BookingAutomation.gs (Google Apps Script) with corrected booking widget integration

Deployment Process

The deployment pipeline follows this sequence:

  1. Stage changes to S3 staging bucket (path-style URLs for each domain)
  2. Invalidate CloudFront distribution associated with staging bucket using the distribution ID
  3. Verify changes via staging CloudFront URL (10-30 second propagation typical)
  4. Promote to production S3 bucket
  5. Invalidate production CloudFront distribution

CloudFront invalidations were issued during this session for both production and staging distributions. The invalidation paths included /* to ensure full cache refresh given the structural changes to the booking widget.

Key Decisions and Rationale

Why Use Lightsail API for Temporary SSH Credentials Instead of Static Keys

The jada-key private key was not stored locally, requiring an alternative approach. AWS Lightsail's temporary credential API generates ephemeral SSH certificates that:

  • Eliminate the need to distribute and rotate long-lived SSH keys
  • Provide automatic expiration (typically 1 hour) reducing blast radius if credentials leak
  • Integrate with AWS audit logs (CloudTrail) for accountability
  • Scale to many instances without key distribution overhead

This is preferable to storing the static private key in version control or shared secrets files.

Why Fix the Booking Widget Template Syntax Before Deployment

The booking widget used Jinja2 template syntax ({{ variable }}) inside JavaScript code. JavaScript also uses braces for object literals. When the HTML was parsed by a browser, there was potential for the template engine to attempt substitution inside JS, causing parse errors. The fix replaced Jinja2 syntax with single braces { variable } to avoid conflicts. This was verified by extracting the JavaScript block and running it through a syntax checker before deployment.

Why Deploy to