Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Diagnosing and Staging a Critical Deposit Flow Outage: Apps Script Access Control and Endpoint Recovery

What Was Done

During a June 2026 operational session, a total stoppage was identified across the deposit/reservation funnel: all 11 event pages on sailjada.com and queenofsandiego.com were returning 403 Forbidden and 404 Not Found errors when users attempted to reserve slots or submit deposits. Root cause analysis revealed that Google Apps Script deployments backing the "Reserve" widget had either lost public access or been deleted entirely. This post covers the diagnosis methodology, staging of the fix, and infrastructure patterns that prevent similar silent failures in the future.

The Silent Failure Pattern

The outage was particularly dangerous because it was invisible to monitoring: users simply saw dead buttons, and the booking funnel metrics showed zero attempts rather than failures. Without explicit endpoint health checks, this could have persisted indefinitely. The deposit system architecture depends on client-side JavaScript calling Google Apps Script /exec URLs to write reservations to a backend database.

Two distinct Apps Script projects were in use:

  • Primary booking endpoint: Project ID 1dDpSK8JZda7XUpKIGlyyAX19KLL4JqFjYVtpcunB5ZE3-NMX_9v0lQJ5, deployment .../44Pme8wCA/exec — returning 403 Forbidden
  • Worship-specific endpoint: Deployment .../AFsLWaO3/exec — returning 404 Not Found

Diagnosis: HTTP Status Verification and Endpoint Enumeration

The investigation began with direct endpoint testing:

# Test primary booking endpoint
curl -I https://script.google.com/macros/s/AKfycbw.../44Pme8wCA/exec
# HTTP/1.1 403 Forbidden

# Test worship endpoint
curl -I https://script.google.com/macros/s/AKfycbw.../AFsLWaO3/exec
# HTTP/1.1 404 Not Found

A 403 indicates the deployment exists but lacks public access. A 404 suggests the deployment was deleted or the URL is malformed. To distinguish between these cases and locate the Apps Script projects, the following search strategy was employed:

  • Searched CloudDocs ops directory for any reference to Apps Script project names or deployment IDs
  • Searched GitHub repositories in ~/Documents/repos/ for .clasp.json configuration files, which contain project IDs and point to source repositories
  • Inspected appsscript.json manifests in those repositories to confirm project display names and current deployment state
  • Cross-referenced deployment IDs against script.google.com console history

This multi-vector search confirmed both projects were legitimate, recently maintained, and still owned by the account — the issue was purely access control and deployment state.

Root Cause: Access Control and Redeployment

Apps Script deployments have two critical access settings:

  • Who has access: Options include "Only me," "Specific people," or "Anyone"
  • Execute as: Options include "Me" (the script owner) or the user calling the endpoint

The primary endpoint was set to "Only me," blocking all external calls. The worship endpoint appeared to have been deleted or redeployed into a different slot, orphaning the old URL.

The fix requires accessing script.google.com, navigating to the project, opening Deploy → Manage deployments, and updating the active deployment to:

  • Who has access: "Anyone"
  • Execute as: "Me" (so backend operations run with script owner permissions)

Why "Anyone" with "Execute as Me"? This pattern is safe for internal service endpoints because the script owner's identity is baked into the execution context, not the caller's. The caller is unauthenticated but the backend has fixed permissions. This avoids the complexity of OAuth token exchange in browser-based reservation flows.

Infrastructure: Event Pages and Endpoint Wiring

Event pages are hosted across two S3 buckets behind CloudFront distributions:

  • sailjada.com: Hosted in S3, synced via source repository at ~/Documents/repos/sites/sailjada/, deployed to CloudFront distribution for CDN caching
  • queenofsandiego.com: Hosted in S3 bucket queenofsandiego-web, synced via ~/Documents/repos/sites/queenofsandiego.com/, deployed to CloudFront

Each event page contains a "Reserve" button that triggers a JavaScript function calling the Apps Script endpoint. Page HTML includes inline references to the full /exec URLs — no environment variables or DNS aliases mediate the call. This direct reference pattern eliminates abstraction layers but requires careful endpoint management.

During the outage investigation, a dry-run was performed to confirm that redeploying the Apps Script would not require page edits:

# Dry-run endpoint rewrite across all event pages
# Verifies that if the deployment ID remains unchanged,
# pages need no edits when re-sharing access.

grep -r "script.google.com/macros/s" ~/Documents/repos/sites/ | head -20
# Confirms all references use the same deployment URLs

This confirmed that the fix is not a code or infrastructure change — it is purely a deployment access control toggle in the Google console.

Staging and Verification

The fix was documented in ~/Library/Mobile Documents/com~apple~CloudDocs/jada-ops/DEPOSIT-OUTAGE-FIX-2026-06-02.md with exact steps and the Apps Script project ID. The file was emailed to the responsible party as part of a broader operational briefing, flagged as the highest-priority action due to its total-stoppage impact on the deposit funnel.

Post-fix verification requires re-running the endpoint tests:

curl -I https://script.google.com/macros/s/AKfycbw.../44Pme8wCA/exec
# Expected: HTTP/1.1 200 OK

Once confirmed, no S3 updates, CloudFront invalidations, or source repo commits are needed — the pages continue to call the same URLs, which now respond with 200.

Key Decisions and Lessons

  • Direct endpoint testing as the first diagnostic step: Before searching logs or configs, verify the actual HTTP response. This eliminated hours of searching.
  • Apps Script deployment ID vs. /exec URL clarity: The project ID and deployment ID are different; only the full /exec URL matters for client calls. The staging doc explicitly distinguishes these to prevent confusion.
  • Silent failures demand explicit health checks: A booking funnel showing zero attempts is a red flag that should trigger automated alerting. Consider adding a lightweight CloudWatch synthetic that periodically calls key endpoints.
  • Access control as infrastructure: Even in