Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Diagnosing and Stabilizing the JADA Agent Orchestrator: OAuth Token Recovery and Turn-Limit Patterns

During a routine health check of the JADA agent daemon running on our Lightsail orchestrator instance (34.239.233.28), we discovered a critical OAuth token failure in the port sheet sync pipeline and identified a recurring pattern where complex tasks are hitting Claude's 30-turn conversation limit. This post walks through the diagnosis, infrastructure approach, and remediation steps taken.

What Was Done

  • Established SSH access to the Lightsail instance using AWS Lightsail API temporary credentials (jada-key pair)
  • Collected full daemon health telemetry: service status, uptime, resource utilization, and recent session logs
  • Identified a broken Google OAuth token in the port_sheet_sync.py script causing repeated 400 Bad Request errors every 30 minutes
  • Documented a pattern where 2 of 3 agent sessions today exited with code 1 due to hitting the 30-turn conversation limit
  • Verified that the daemon itself is healthy and continues normal operation after incomplete sessions

Technical Details: Daemon Health Status

The jada-agent.service systemd unit is running smoothly with 11 days of uptime and only idle load between tasks. Key metrics:

  • Service status: Active (running) since May 10, 2026
  • System load: 0.00 average — daemon's 60-second poll loop consumes ~0.65% CPU during checks
  • Memory footprint: 144 MB / 914 MB available — well within acceptable bounds
  • Disk usage: 6.2 GB / 39 GB (17%) — ample headroom for log rotation and task data
  • Network health: Zero status check failures in the last 2 hours

Session activity for 2026-05-13 reveals three separate invocations of the agent runtime:

  • Session 1 (00:00 UTC): Exited with code 1 after hitting 30-turn limit
  • Session 2 (00:02 UTC): Completed successfully; processed e-signature page blockers and crew page generator code, created a needs-you task for manual review
  • Session 3 (00:05 UTC): Exited with code 1 after hitting 30-turn limit

After session 3, the daemon returned to idle polling — no new tasks were enqueued in the progress dashboard, confirming normal quiescent behavior.

Critical Issue: Broken OAuth Token in Port Sheet Sync

The most actionable finding is a persistent authentication failure in the port sheet synchronization pipeline. Every 30-minute execution of port_sheet_sync.py has been logging the same error:

[port-sheet] token error: HTTP Error 400: Bad Request

This indicates that the Google OAuth 2.0 refresh token stored for the port sheet integration has either expired or been revoked by the user. The script is attempting to sync booking data to a Google Sheet but cannot authenticate with the Google Sheets API. This is a blocking issue for the booking automation pipeline that depends on real-time port sheet updates.

Root cause: Google OAuth tokens have a limited lifetime. Refresh tokens can be invalidated if:

  • The user revoked app permissions in their Google account settings
  • The user changed their Google account password
  • The token was issued more than 6 months ago and hasn't been used (Google's automatic expiration policy)
  • The OAuth consent screen was modified or the app's scopes changed without re-authorization

Impact: The booking automation system in /Users/cb/Documents/repos/sites/queenofsandiego.com/BookingAutomation.gs cannot push confirmed bookings to the master port sheet, creating a data consistency gap between the booking widget and the operational source of truth.

Infrastructure and Daemon Architecture

The JADA agent runs as a systemd service on an AWS Lightsail instance with the following design:

  • SSH access model: Rather than storing a long-lived private key on the local development machine, we use AWS Lightsail's temporary credential API to obtain ephemeral SSH keys via the GetInstanceAccessDetails endpoint. This eliminates the need to manage a static jada-key file in ~/.ssh/.
  • Service lifecycle: The daemon polls a progress dashboard (stored state or task queue) every 60 seconds and invokes the Claude agent runtime when tasks are available.
  • Session management: Each invocation is a separate conversation session with a hard limit of 30 turns. When this limit is reached, the session exits cleanly with code 1, and any incomplete work is logged for manual triage.
  • Sidecar syncs: Parallel to the main agent loop, port_sheet_sync.py runs on a 30-minute interval to keep Google Sheets in sync with booking data. This process is separate from the agent runtime and has its own OAuth credentials.

Key Decisions and Rationale

1. Why use Lightsail API for SSH instead of stored keys?

Static SSH keys introduce operational risk — they must be stored somewhere secure, rotated regularly, and protected against exfiltration. AWS Lightsail's temporary credential API generates a valid SSH key pair with a 15-minute TTL, reducing the attack surface and eliminating long-lived secrets from the development environment.

2. Why is the 30-turn limit not a critical failure?

The agent design accepts that complex tasks may require more than 30 turns of reasoning. Rather than extending the limit indefinitely (which increases latency and cost), we treat max-turn exits as a signal to break the task into smaller chunks or escalate to manual review. Session 2's successful completion shows that well-scoped work completes within the limit.

3. Why monitor OAuth token health separately?

Port sheet sync is a sidecar process with independent credentials. It should not block the main agent loop; however, its failure should be loudly logged and alertable. The current logs show this is working as designed — the sync fails gracefully without crashing the daemon.

What's Next

  • Re-authorize Google OAuth for port sheet: Run the Google auth flow for port_sheet_sync.py with the correct scopes (Google Sheets API) to obtain a fresh refresh token. This should resolve all 30-minute sync failures immediately.
  • Analyze session 1 and 3 task scope: Review the work that was attempted in the max-turn exits to determine if tasks should be split or if certain reasoning paths can be optimized to fit within 30 turns.
  • Set up OAuth token rotation policy: Implement a quarterly re-auth step for all external API integrations to prevent silent token expiration in production.
  • Add token health monitoring: Extend daemon logging to explicitly check Google OAuth token validity on startup and alert if refresh fails.

The daemon is operationally sound. The fixes required are focused on auth credential refresh and task design optimization — both straightforward next steps.

```