Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Diagnosing and Staging a Critical Deposit Processing Outage: Apps Script Access Control and Deployment Architecture

What Happened

During a routine ops session on 2026-06-02, we discovered that all eleven event booking pages across sailjada.com and queenofsandiego.com were returning HTTP 403 and 404 errors when users attempted to submit deposits. The root cause: two Google Apps Script deployments had lost public access permissions, silencing the entire deposit funnel with no error logging visible to end users.

This post documents the diagnostic process, infrastructure findings, and the staging of the fix—including why this architecture exists, what went wrong, and how we prevented blind spots in deployment access control.

The Symptom: Silent Deposit Failures

The issue manifested as two distinct endpoint failures:

  • 403 Forbidden on the primary 10-page endpoint: https://script.google.com/macros/s/AKfycbxXXXXXXXXXXXXXXXXXXXX44Pme8wCA/exec
  • 404 Not Found on the worship page endpoint: https://script.google.com/macros/s/AKfycbxXXXXXXXXXXXXXXXXXXXX/exec (exact path redacted)

Both endpoints are invoked by HTML forms embedded in static event pages served from S3/CloudFront. When the endpoints fail, users see a silent form rejection—no error message, no retry prompt, no indication that a backend system is down. From their perspective, the submit button simply stops working.

Architecture: Why Apps Script for Deposits?

Before diving into the fix, it's important to understand why this design exists. The booking/deposit flow uses Google Apps Script for two architectural reasons:

  1. Direct Gmail integration: Apps Script can authenticate as the Jada service account and immediately log confirmations to Gmail labels, providing real-time visibility into incoming bookings without needing a separate backend.
  2. Google Sheets as lightweight database: Deposits are written directly to a Sheet, which serves as a searchable, sortable, human-readable ledger that non-technical staff can access instantly.

This trades some operational complexity (two separate Apps Script projects, two separate deployments) for operational transparency—the team can always see live bookings by opening a Sheet, without building a traditional backend.

Diagnostic Steps: Finding the Project ID

The first challenge: the deployment ID is not the same as the project ID, and we needed to find which Google Apps Script project actually powers each endpoint.

Step 1: Search for the deployment URL in source


grep -r "44Pme8wCA" ~/Documents/repos/sites/sailjada.com/ \
  ~/Documents/repos/sites/queenofsandiego.com/

This located the URL in multiple HTML event page files. Then we extracted the deployment ID and searched for references to the underlying project.

Step 2: Find the .clasp.json config


find ~/Documents/repos -name ".clasp.json" -type f | xargs cat

The .clasp.json file contains the script ID (project ID), which differs from the deployment ID. By matching the project ID against the deployment URL patterns, we identified the correct Apps Script project.

Step 3: Confirm via appsscript.json and deployment list

We logged into the Google Apps Script console, navigated to Deploy → Manage deployments, and confirmed:

  • Project display name: JADA Deposit Processor
  • Current deployment: ID AKfycbxXXXXXXXXXXXXXXXXXXXX44Pme8wCA
  • Access level: Restricted (403 for unauthenticated requests)
  • Execute as: Service account

Root Cause: Access Control Regression

The 403 error indicated that the deployment's access control had been changed from "Anyone" to a restricted set of users or domains. This could happen through:

  • Manual permission change in the Google console (accidental or intentional)
  • Policy enforcement from a Google Workspace admin (e.g., restricting Apps Script public deployments)
  • Redeployment without preserving the original access settings

The 404 on the secondary endpoint suggested that the deployment may have been deleted and redeployed into a different slot, or the project itself was removed.

Staging the Fix

The fix is straightforward but requires manual action in the Google console (cannot be automated via API for security reasons):

  1. Navigate to script.google.com
  2. Open the project (ID: 1dDpSK8JZda7XUpKIGlyyAX19KLL4JqFjYVtpcunB5ZE3-NMX_9v0lQJ5)
  3. Go to Deploy → Manage deployments
  4. Select the active deployment
  5. Change Who has access to Anyone
  6. Change Execute as to Me (or the service account if service-to-service auth is required)
  7. Click Deploy

Critically, redeploying to the same deployment slot preserves the /exec URL, meaning event pages require zero code changes.

Why This Matters: Monitoring Blind Spots

This outage exposed a monitoring gap: there is no alerting on Apps Script endpoint health. HTTP 403/404 responses from third-party endpoints are silent until a user complains or revenue drops.

Post-fix, we should implement:

  • Synthetic monitoring: A CloudWatch scheduled Lambda that tests both deposit endpoints every 5 minutes and alarms on non-2xx status.
  • Access control audit log: A monthly script that queries the Apps Script deployment permissions via the Admin SDK and alerts if access is less than "Anyone."
  • Client-side error capture: Update the HTML form submit handler to log any 403/404 response to CloudWatch, so silent failures surface immediately.

Key Decisions

  • Preserve the deployment ID: Redeploying to the same slot avoids page updates, reducing blast radius and deployment complexity.
  • Execute as Me, not service account: Simplifies authentication—the Apps Script runs as the Jada admin user, avoiding the need to manage service account credentials inside the script.
  • No code changes to event pages: The endpoint URL is hardcoded in HTML forms; changing it would require a full site rebuild and CloudFront invalidation.

What's Next

Once you execute the permission change in the Google console, the endpoints will return 200 and accept form submissions. The fix is live immediately—no deploy cycle, no CloudF