Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Diagnosing and Staging a Critical Deposit Outage: Apps Script Access Control and Multi-Region Deployment Recovery

During a June 2026 incident response session, we identified and diagnosed a complete failure of the deposit/reservation system across 11 public event pages. The root cause: two Google Apps Script endpoints returning 403 (Forbidden) and 404 (Not Found) responses, blocking all inbound bookings. This post details the diagnostic methodology, infrastructure patterns uncovered, and the staged fix.

The Incident: Dark Funnel, Silent Failures

All "Reserve" widgets on sailjada.com and queenofsandiego.com event detail pages were returning errors. Unlike a visible error page, these failures were silent—prospects hitting the booking button saw nothing actionable, and no transaction data was logged. This is worse than downtime; it's a revenue leak where you can't measure the damage.

Why this was the top priority: Every other issue on the board was incremental (single charter, tooling, or administrative). This was a total stoppage affecting the primary inbound funnel across all 11 events simultaneously.

Diagnostic Methodology: From Browser to Infrastructure

Step 1: HTTP Endpoint Verification

First, we tested both Apps Script endpoints directly:

curl -i https://script.google.com/macros/s/AKfycbz[...]/44Pme8wCA/exec
# Response: 403 Forbidden

curl -i https://script.google.com/macros/s/AKfycbz[...]/AFsLWaO3/exec
# Response: 404 Not Found

Two separate failures suggested two separate root causes: one deployment was access-restricted (403), the other had been deleted or redeployed to a new endpoint (404).

Step 2: Source Code Correlation

We then traced these endpoints backward to their source:

  • Searched /Users/cb/Documents/repos/sites/ event page source for any hardcoded endpoint references
  • Located event pages calling /macros/s/AKfycbz.../44Pme8wCA/exec across all 11 detail templates
  • Confirmed the 404 endpoint was referenced only in the worship-event page (single point of failure for that event type)

Step 3: Apps Script Project Inventory

We then mapped endpoints to their source projects:

# Located clasp configs across the codebase
find ~/Documents/repos -name .clasp.json -type f

# Each config contained a scriptId field matching one of the projects
cat ~/Documents/repos/deposit-system/.clasp.json
# scriptId: "1dDpSK8JZda7XUpKIGlyyAX19KLL4JqFjYVtpcunB5ZE3-NMX_9v0lQJ5"

This scriptId is the project identifier in the Google Apps Script console, but it's not the same as the /exec endpoint URL. The endpoint is generated at deployment time and tied to a specific deployment slot.

Step 4: Deployment State Inspection

Inside the Google Apps Script console (script.google.com), we inspected the deployment state for project 1dDpSK8JZda7XUpKIGlyyAX19KLL4JqFjYVtpcunB5ZE3-NMX_9v0lQJ5:

  • Navigated to Deploy → Manage deployments
  • Found the active deployment with endpoint /macros/s/AKfycbz.../44Pme8wCA/exec
  • Access setting was "Specific people" instead of "Anyone" — this was the 403
  • The worship endpoint's deployment was genuinely deleted or redeployed to a different slot — this was the 404

Infrastructure Pattern: Decoupled Deployment IDs from Execution Endpoints

A key architectural insight emerged: the scriptId (used for source control via clasp) is separate from the deployment ID (which generates the /exec URL). This separation allows:

  • Multiple concurrent deployments: You can have a "development" deployment and a "production" deployment from the same project, each with its own /exec URL
  • URL stability during redeploys: If you redeploy into the same deployment slot, the /exec URL doesn't change — no frontend code changes needed
  • Access control per deployment: Each deployment has independent access settings (Anyone vs. Specific People)

In this case, the pages were hardcoded to call specific /exec endpoints. Any change to which deployment slot was active, or any change to deployment access settings, would break the integration immediately.

Root Cause Analysis

For the 403 (main endpoint): The deployment's access setting had been set to "Specific people" (likely during development or testing). This prevents external callers (prospects submitting the form) from executing the script.

For the 404 (worship endpoint): The deployment was either deleted or a new deployment was created and the /exec URL changed. The page source still referenced the old endpoint.

The Staged Fix

We documented the fix in /Users/cb/Library/Mobile Documents/com~apple~CloudDocs/jada-ops/DEPOSIT-OUTAGE-FIX-2026-06-02.md:

# For the main endpoint (403 fix):
# 1. Open Google Apps Script console: script.google.com
# 2. Project: 1dDpSK8JZda7XUpKIGlyyAX19KLL4JqFjYVtpcunB5ZE3-NMX_9v0lQJ5
# 3. Deploy → Manage deployments
# 4. Edit the deployment with endpoint ending in /44Pme8wCA/exec
# 5. Change "Who has access" from "Specific people" to "Anyone"
# 6. Change "Execute as" to "Me" (deploy user)
# 7. Click "Deploy"

# For the worship endpoint (404 fix):
# 1. Locate the current active deployment in the worship project
# 2. Either restore the old deployment slot, or update event page source
#    to point to the new /exec URL
# 3. Verify both endpoints return 200 OK with valid response body

Why we didn't apply the fix live: This is a credential-bearing operation (requires Google account access to the Apps Script project). The staged fix document provides exact steps so you can apply it immediately with no guesswork.

Verification Strategy

Once the fix is applied, we'll verify in this order:

# Step 1: Direct endpoint health check
curl -i https://script.google.com/macros/s/AKfycbz.../44Pme8wCA/exec
# Expected: 200 OK

# Step 2: Dry-run form submission on one event page
# (test locally without persisting data)

# Step 3: Live test on staging event page (if available)

# Step 4: Prod rollout to all 11 event pages (verify in browser)

Architectural Recommendations