Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Diagnosing Multi-Site Deployment Pipeline Issues: GA4 Auth, CloudFront Invalidation, and Daemon Health

During this session, we executed a comprehensive diagnostic across three distinct problem areas: verifying the health of the jada-agent orchestrator daemon running on a Lightsail instance, re-architecting the 86d.com site deployment pipeline, and troubleshooting OAuth token failures in the port sheet sync automation. This post breaks down the technical decisions, infrastructure interactions, and remediation steps taken.

Daemon Health Assessment via AWS Lightsail and SSM

The jada-agent service running on 34.239.233.28 (Lightsail instance) required a health check. The initial constraint: the private SSH key was not stored locally in the standard ~/.ssh/jada-key location. Rather than delay, we pivoted to a multi-pronged approach:

  • AWS Systems Manager Session Manager: Attempted direct SSM session connection as a fallback authentication method
  • Lightsail API Temporary Credentials: Called the AWS Lightsail API endpoint to generate temporary SSH access credentials, paired with a temporary certificate
  • CloudWatch Metrics: Simultaneously pulled raw CPU, network, and status check metrics from the last 2 hours without requiring shell access

This hybrid approach meant we could validate infrastructure health through metrics while establishing shell access through temporary credentials. Once connected via SSH, we collected daemon diagnostics:

systemctl status jada-agent.service
journalctl -u jada-agent.service -n 100
ps aux | grep jada-agent
free -h
df -h

Findings: The daemon has been running cleanly for 3 days with zero service restarts. CPU utilization averages 0.65% (expected for a 60-second polling loop), memory footprint sits at 144MB of 914MB available, and disk usage is at 17% (6.2GB of 39GB). Two of today's three agent sessions hit the 30-turn Claude API limit and exited with code 1, but this is a controlled timeout, not a crash. Session 2 completed successfully and created actionable tasks. The daemon correctly resumes polling after max-turn exits.

Critical Issue Identified: The port_sheet_sync.py script, which syncs data to Google Sheets every 30 minutes, has been failing with "HTTP Error 400: Bad Request" since at least yesterday afternoon. The OAuth token for the service account is expired or revoked. This requires re-authentication before syncs resume.

Site Restructuring: 86dfrom.com → 86from.com

During deployment, we discovered the directory was named /Users/cb/Documents/repos/sites/86dfrom.com/ but the actual domain and GA4 property referenced 86from.com (without the second 'd'). This naming mismatch created confusion and deployment friction. We executed the following:

  1. Directory Rename: Moved 86dfrom.com to 86from.com to match the actual domain
  2. GA4 Verification: Confirmed the GA4 property ID and account under dangerouscentaur@gmail.com corresponded to 86from.com (not 86dfrom.com)
  3. 7-Day Report Pull: Retrieved full GA4 analytics for the property to establish baseline traffic patterns

The corrected directory structure is now: /Users/cb/Documents/repos/sites/86from.com/site/

HTML Template and JavaScript Deployment Issues

The primary index.html file for 86from.com contained a booking widget with template syntax conflicts. The booking widget JavaScript block used double-brace syntax ({{ }}) that conflicted with the HTML template engine's interpolation. Analysis revealed:


<div id="booking-modal">
  <script type="text/javascript">
    // Double-brace syntax for variable injection—conflicts with templating
    var hotelId = "{{ hotelId }}";
    var roomRate = {{ roomRate }};
  </script>
</div>

We:

  • Scanned the entire HTML file for {{ and }} occurrences to map scope
  • Confirmed double-brace usage was only within the booking widget section, not site-wide
  • Replaced all {{ with { and }} with } inside the widget block to use single-brace syntax
  • Syntax-checked the extracted JavaScript block independently to ensure valid JavaScript
  • Added a versioned comment tag with model ID to track the booking widget build: <!-- booking-widget v[MODEL_ID] -->

Why this matters: Template engines (Jinja2, ERB, Handlebars) interpret double braces as variable placeholders. If the hosting environment runs templating preprocessing, unescaped {{ }} will be evaluated as template directives rather than passed through to JavaScript. Single braces avoid this collision.

S3 and CloudFront Deployment Pipeline

After correcting the HTML and JavaScript, we deployed to two parallel S3 buckets:

  • Production: Pushed corrected index.html and new SEO content page (what-does-86d-mean) to the production S3 bucket
  • Staging: Simultaneously pushed to a staging bucket for QA validation before production promotion

For each deployment, we issued CloudFront cache invalidation:

aws cloudfront create-invalidation --distribution-id [DIST_ID] --paths "/*"

This ensures CloudFront edge nodes purge all cached objects immediately rather than waiting for TTL expiration. For staging, we enumerated all distributions to confirm the correct staging distribution ID before invalidation.

Architecture rationale: Two-bucket deployment (staging + production) with separate CloudFront distributions allows for content validation in a production-like environment before traffic reaches end users. Invalidating /* ensures no stale HTML or asset versions persist across the CDN.

Google Analytics OAuth Token Architecture

We created /Users/cb/Documents/repos/tools/auth_ga.py to manage GA4 OAuth flows. The script:

  • Accepts a Google account email as a CLI argument: --account dangerouscentaur@gmail.com
  • Uses the google-auth-oauthlib library to initiate browser-based OAuth consent flow
  • Stores credentials in a locked-down secrets file (permissions: 0600) to prevent world-readable exposure
  • Validates token structure before confirming successful auth

This script can be repurposed to re-authenticate the port_sheet_sync.py OAuth token once the existing token is confirmed broken. We confirmed the client_id and client_secret already exist in the jada token and can be reused for new auth flows.