Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Automating Charter Document Generation and S3 Publishing: Building a Scalable Manifest System for JADA Operations

Overview: From Manual Spreadsheets to Automated Document Pipeline

This session involved building an end-to-end automation pipeline for JADA's charter operations, specifically targeting the generation and publication of crew manifests and trip sheets. Previously, these documents were manually created in spreadsheets and stored inconsistently. The goal was to create a system that:

  • Generates properly formatted HTML manifests from charter calendar data
  • Publishes documents to S3 with correct content-type headers
  • Invalidates CloudFront cache to ensure live updates
  • Maintains document durability by storing templates locally
  • Integrates with the existing shipcaptaincrew Lambda infrastructure

Technical Architecture

The solution leverages JADA's existing AWS infrastructure, specifically:

  • S3 Bucket: queenofsandiego-prod — stores published crew manifests and trip sheets
  • CloudFront Distribution: Fronts the S3 bucket, serving documents at https://shipcaptaincrew.queenofsandiego.com/
  • Lambda Function: /repos/sites/queenofsandiego.com/tools/shipcaptaincrew/lambda_function.py — handles crew page rendering and document serving
  • Document Prefix Structure: docs/ for general crew documents, crew-page/docs/ for charter-specific manifests

The charter document lifecycle follows this flow:

Calendar API → Charter Readiness Report → HTML Generation → S3 Upload → CloudFront Invalidation → Live URLs

Key Implementation Details

Document Generation and Templates

Created two primary Python scripts to handle charter operations:

  • /jada-ops/charter_provisioner.py — Main provisioning engine that processes charter data and generates formatted HTML manifests. This script reads from the JADA calendar API, extracts passenger lists and trip details, and renders them into consistent HTML templates.
  • /jada-ops/send_charter_emails.py — Companion script for distributing generated documents to crew via email integration.

Charter documents are stored in two locations for resilience and ease of access:

  • Local Repository: /jada-ops/{charter_name}/ — stores manifest and trip sheet files locally for version control and durability
  • S3 Production: s3://queenofsandiego-prod/crew-page/docs/{event_id}/ — published manifests served via CloudFront

HTML Manifest Format and Content Structure

Manifests include structured passenger information, safety briefing sections, and vessel-specific details. Example structure:

<h1>Quinn Male Charter Manifest</h1>
<table>
  <tr><th>Passenger Name</th><th>Cabin Assignment</th></tr>
  <tr><td>Passenger 1</td><td>Forward</td></tr>
</table>

This ensures that manifests render correctly in both the Lambda-served crew page and as standalone PDFs when printed from the browser.

S3 Publishing and Cache Invalidation Strategy

Upload Process

Documents are uploaded to S3 with explicit content-type headers to ensure proper browser rendering:

aws s3 cp quinn-male-manifest.html s3://queenofsandiego-prod/crew-page/docs/{event_id}/ \
  --content-type "text/html; charset=utf-8" \
  --region us-west-2

The event_id is extracted from the JADA calendar API response and used as the S3 key prefix, allowing documents to be associated with specific charters programmatically.

CloudFront Cache Invalidation

After uploading, CloudFront's cache must be invalidated to ensure users receive the latest version. The distribution serves these document paths, and invalidations target the specific paths modified:

aws cloudfront create-invalidation \
  --distribution-id {DISTRIBUTION_ID} \
  --paths "/crew-page/docs/{event_id}/*" \
  --region us-west-2

Why this matters: CloudFront caches HTML documents aggressively. Without invalidation, users would see stale manifests for up to 24 hours. Explicit invalidation ensures live updates within seconds.

Integration with Existing Lambda Infrastructure

The shipcaptaincrew Lambda function serves crew pages dynamically. Key integration points:

  • Document Discovery: The Lambda's build_event_pages function scans S3 prefixes to locate available documents for each charter
  • Document Serving: The handle_get_doc function retrieves documents from S3 and serves them with proper headers
  • URL Generation: Crew pages dynamically generate download links for manifests: /crew-page/{event_id}/doc/{document_id}

The Lambda function was updated to support multiple document prefixes (docs/ and crew-page/docs/), allowing gradual migration of documents without breaking existing links.

Data Flow and Integration Points

The complete charter readiness workflow:

  1. Calendar Fetch: Query JADA calendar API for weekend charters using OAuth token refresh
  2. Readiness Report: Generate comprehensive markdown report listing all upcoming charters (saved to /jada-ops/weekend-charters-readiness-2026-05-29.md)
  3. Per-Charter Processing: For each charter (Quinn Male, Jonathan, etc.):
    • Extract passenger list and trip details from calendar event
    • Generate HTML manifest and trip sheet
    • Save locally to /jada-ops/{charter_name}/
    • Upload to S3 with correct content-type
    • Invalidate CloudFront cache
  4. Email Distribution: Send manifests to crew via send_charter_emails.py

Key Architecture Decisions

  • Dual Storage (Local + S3): Local files in the repository provide version history and a fallback if S3 becomes unavailable. S3 serves as the source of truth for live documents.
  • Event ID as S3 Key: Using the calendar event_id as the S3 prefix allows the Lambda function to associate documents with specific charters without additional