Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Diagnosing and Staging a Deposit-Outage Fix: Apps Script Access Control and Multi-Region Email Delivery

Executive Summary

During a June 2026 incident review, we discovered that the primary deposit/reservation widget across 11 sailjada and queenofsandiego event pages had gone silent — all returning HTTP 403 (Forbidden) or 404 (Not Found) from Google Apps Script endpoints. This post documents the diagnostic process, root-cause isolation, and the staged fix that restores booking funnel visibility without requiring code changes or S3 redeployment.

What Was Done

We identified a complete stoppage in the public-facing reservation system affecting every event page's "Reserve" button. The incident silently broke the booking funnel — users saw dead buttons with no error feedback, meaning lost revenue was invisible until explicitly diagnosed. The fix required no code changes, only a single authorization shift in the Google Cloud console.

Diagnostic Process: From Symptom to Root Cause

Step 1: Live Endpoint Health Check

We began by testing both known Apps Script endpoints directly:


# Primary endpoint (10-page fleet)
curl -I https://script.google.com/macros/s/AKfycbw...44Pme8wCA/exec
# Response: HTTP 403 Forbidden

# Secondary endpoint (worship page)
curl -I https://script.google.com/macros/s/AKfycbw...AFsLWaO3/exec
# Response: HTTP 404 Not Found

The 403 indicated the deployment still existed but access was restricted. The 404 suggested the second deployment had been deleted or never redeployed after a code update.

Step 2: Mapping Event Pages to Endpoints

We audited all event-page HTML files in the S3 bucket to confirm which endpoint each page called:


# Searched S3 for endpoint references
s3:// queenofsandiego.com/events/*/index.html

Found 10 pages hitting the primary endpoint (all returning 403) and 1 worship-specific page hitting the secondary endpoint (returning 404). This 100% coverage of the visible "Reserve" widget meant every inbound booking attempt was silently failing.

Step 3: Apps Script Project Identification

We located the project ID by reading stored Google Apps Script deployment configs and the appsscript.json manifest:

  • Project ID: 1dDpSK8JZda7XUpKIGlyyAX19KLL4JqFjYVtpcunB5ZE3-NMX_9v0lQJ5
  • Primary Deployment ID: AKfycbw...44Pme8wCA
  • Secondary Deployment ID: AKfycbw...AFsLWaO3

The distinction between project ID and deployment ID is critical: the project is the source container; the deployment is a published executable instance. An Apps Script project can have multiple deployments (for staged rollouts or A/B testing), each with its own `/exec` URL and access control list.

Root Cause: Access Control Misconfiguration

By inspecting the deployment metadata in the Google Cloud console, we determined:

  • Primary deployment: Access set to a restricted list (likely a specific user or service account), triggering 403 for public requests
  • Secondary deployment: Entirely missing from the active deployment list, suggesting it was deleted after a refactor or during a failed redeploy

Neither deployment was set to "Anyone" (public), which is required for a stateless booking widget that serves anonymous users from the web.

Technical Details: Apps Script Deployment Lifecycle

Deployments vs. Projects

Google Apps Script uses two separate concepts:

  • Project: The source code container and development environment. Only one project per source tree.
  • Deployment: A published, executable instance of the project code. A project can have unlimited deployments, each with its own URL and access policy.

When you edit code and redeploy, you can either:

  • Create a new deployment (new `/exec` URL, requires page code updates)
  • Update an existing deployment (same `/exec` URL, code updates transparent to callers)

The event pages are hardcoded to specific deployment URLs, so we had to fix the existing deployments rather than create new ones.

Access Control Model

Apps Script deployments support three access levels:

  • Me (Execute as self): Only the authenticated project owner can call the endpoint.
  • Specific users/service accounts: A whitelist of Google identities.
  • Anyone (Execute as Me): Any user (authenticated or anonymous) can call the endpoint. The script runs as the project owner, with their permissions.

For a public booking widget, "Anyone" is required because the caller is a web visitor with no Google credentials.

The Staged Fix

The fix requires a single manual action in the Google Apps Script console (no code changes, no redeployment):

  1. Navigate to script.google.com and open the project by ID
  2. Click Deploy → Manage deployments
  3. For each deployment:
    • Click the pencil (edit) icon
    • Change "Who has access" to "Anyone"
    • Ensure "Execute as" is set to "Me" (the project owner)
    • Click Deploy
  4. Verify that both `/exec` URLs now return HTTP 200 (not 403 or 404)

This change is live immediately — no S3 cache invalidation, no CloudFront distribution flush, no code push required. The event pages will begin receiving booking submissions within seconds.

Why This Happened: Risk Analysis

The most likely cause:

  • Accidental restriction: During a code update or security audit, an admin may have tightened the access control without realizing it would break the public widget.
  • Service account leak: The secondary deployment may have been created for a temporary service-to-service integration that was later deleted without checking for public dependencies.
  • Insufficient test coverage: No alerting or synthetic test was in place to detect when public endpoints became unreachable.

Infrastructure Context

The booking system is architected as:

  • Frontend: 11 static HTML event pages in S3 bucket queenofsandiego.com (or sailjada)
  • Compute: Google Apps Script deployments (serverless, no VPC/EC2 needed)
  • Database: Google Sheets (implied by Apps Script usage; acts as the booking led