Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Fixing Google OAuth Token Refresh and Scope Issues in the JADA Booking System

What Was Done

The JADA booking system experienced a critical OAuth token refresh failure affecting Gmail operations. A 400-level error on the legacy gmail-full token combined with scope inconsistencies across multiple token files broke email outreach, crew communication, and administrative workflows. We rebuilt the re-authentication flow with proper scope management, implemented email-based OAuth callback handling, and moved the system to unified token storage with deterministic scope enforcement.

The Problem: Token Fragmentation and Scope Creep

The JADA infrastructure had accumulated multiple token files over time:

  • ~/.jada-secrets/tokens/gmail-full.json — legacy token with inconsistent scopes
  • ~/.jada-secrets/tokens/unified-token.json — intended unified token (incomplete scopes)
  • Service account tokens scattered across multiple Lambda environments

When reauth_jada_all.py attempted to refresh the legacy gmail-full token, the OAuth provider rejected it with a 400 Bad Request after 10 seconds. The root cause: the token was issued with outdated scopes that no longer matched the current OAuth app configuration in Google Cloud Console. The scopes were frozen at an old timestamp and no longer aligned with what we needed:

  • gmail.modify — for reading/modifying Gmail messages
  • gmail.compose — for composing and sending (added later, missing from legacy token)
  • calendar — for JADA Internal GCal read access

Technical Diagnosis

We traced the issue through several steps:

  1. Scope audit: Inspected both token files using jq '.scopes[]' ~/.jada-secrets/tokens/*.json and found the legacy token lacked gmail.compose, while the unified token had incomplete metadata.
  2. Token refresh simulation: Ran python3 tools/jada_google.py --health from Lightsail (the only location with OAuth client credentials cached). The Mac environment could not reach api.stripe.com or complete OAuth flows due to network policy — a known constraint documented in [[estate-infra-quickmap.md]].
  3. OAuth app production status: Verified that the OAuth app was still in Testing mode. Google's Testing mode limits token lifetime to 7 days, forcing frequent re-auth. Per [[jada-google-auth-robust.md]], the fix was to publish the app to Production status in Google Cloud Console.

Solution Architecture: Email-Based OAuth Callback

Instead of relying on a browser session (which created friction when CB was away), we built an email-driven OAuth callback mechanism: 

User (CB) triggers re-auth on Lightsail:
  $ cd ~/icloud-repos/tools
  $ python3 reauth_jada_all.py

Server starts local HTTP listener on ephemeral port (e.g., 53212)
  ↓
User visits Google OAuth consent screen (Opera browser opens automatically)
  ↓
OAuth code redirects to http://localhost:53212/callback?code=...
  ↓
Local server exchanges code for tokens, persists to ~/.jada-secrets/tokens/
  ↓
Listener watches incoming emails to jadasailing@gmail.com for OAuth redirect
  ↓
finish_auth.py + watch_auth_email.py sync tokens across infrastructure

This dual-path approach (local callback + email confirmation) handled both same-machine auth and remote re-auth scenarios without requiring the user to manually copy/paste tokens.

Code Changes and Infrastructure

File: tools/reauth_jada_all.py (rebuilt, ~200 lines)

  • OAuth flow: Uses google-auth-oauthlib.flow.InstalledAppFlow to request scopes [gmail.modify, gmail.compose, calendar] with proper timeout handling (15-second socket/connect timeouts to handle transient SSL errors).
  • Local listener: Starts a small Flask server on localhost:ephemeral_port, extracts the OAuth code from the redirect, and immediately exchanges it for refresh/access tokens.
  • Token storage: Writes unified token to ~/.jada-secrets/tokens/unified-token.json with explicit scopes array and expiry metadata.
  • Scope enforcement: Before token refresh, the script validates that required scopes exist in the token; if missing, it raises an error and halts (preventing silent failures).

New file: tools/watch_auth_email.py

Monitors jadasailing@gmail.com for the OAuth redirect callback. When the user consents and Google redirects, the local callback server sends an SMS to CB's JADA line with the magic URL. This email watcher polls incoming messages every 10 seconds for up to 3 hours, searching for the OAuth redirect URL in the email body, then extracts and logs it for manual inspection if needed.

New file: tools/finish_auth.py

Final sync step: Reads the unified token from ~/.jada-secrets/tokens/unified-token.json and distributes it to:

  • Lambda environment variables in AWS (for shipcaptaincrew function)
  • Lightsail EC2 instance /root/.jada-secrets/tokens/ (canonical home)
  • Mac iCloud jada-ops bridge (read-only, via symlink)

Per [[environments-and-auth.md]], the Lightsail box is the canonical credential holder. The Mac cannot initiate OAuth flows due to network restrictions, so it syncs via the iCloud bridge.

Google Cloud Console Changes

OAuth 2.0 Application Status:

  • Published the JADA Google OAuth app from Testing → Production status in Google Cloud Console.
  • Reason: Testing mode enforces a 7-day token lifetime, causing forced re-auth every week. Production mode allows refresh tokens to persist indefinitely (until revoked).

Authorized Redirect URIs:

  • Added http://localhost:*/callback to handle ephemeral port binding in the local re-auth flow.
  • Kept https://shipcaptaincrew.app/auth/callback for any web-based re-auth in future.

Why This Approach

  • Unified token storage: One authoritative token file (unified-token.json) eliminates sync inconsistencies and version mismatches.
  • Scope freezing: Token file stores the exact scopes granted at auth time. On each refresh attempt, we validate scopes match requirements; if not, we fail loudly rather than silently using a degraded token.
  • Email callback fallback: The email watcher decouples the local callback from requiring CB to stay at the machine. He can step away, check Slack, and the system captures the redirect URL via Gmail polling.
  • Deterministic re-auth: The script is idempotent — running it twice in a row succeeds, with the second run being a no-op. No session state pollution.
  • Lightsail-first credential flow: Per [[cb-infra-authorization.md]], Lightsail is the only location with OAuth client credentials (whitelisted by IP at 34.239.233.28 in Namecheap API). The Mac never holds or rotates credentials; it syncs read-only copies via iCloud.

What's Next

  • Nightly health check: Integrate token refresh health into the JADA test system cron ([[jada-test-system.md]]) so token degradation is caught before it breaks production workflows.
  • Staged scope rotation: As new Gmail/Calendar features are needed, add scopes to the OAuth app, trigger a re-auth, and verify the new token's scopes in the ledger ([[save-human-name-with-ids.md]]).
  • Service account decoupling: Lambda functions should use service accounts (with Workload Identity Federation) instead of sharing OAuth refresh tokens. This isolates credentials per function and allows granular IAM revocation.

Key Commands (No Secrets)

# Trigger full re-auth (run on Lightsail only)
cd ~/icloud-repos/tools
python3 reauth_jada_all.py

# Check token health
python3 jada_google.py --health

# Inspect token scopes (safe command, no secrets exposed)
jq '.scopes[]' ~/.jada-secrets/tokens/unified-token.json

# Watch for OAuth callback email (run in background)
python3 watch_auth_email.py 53212 180  # port, timeout_seconds

# Finalize and distribute tokens to all infrastructure
python3 finish_auth.py
```