Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Building a Multi-Stage Dispatch System: CloudFront Rewrites, Lambda Job Processing, and Production Deployment

What Was Done

We deployed a complete dispatch and job-tracking system for quickdumpnow.com that handles job creation, real-time status tracking, and customer-facing progress updates. The system uses CloudFront Functions for URL rewriting, AWS Lambda for backend job processing, and S3 for persistent job state. This involved deploying infrastructure across multiple stages (staging → production), creating a new job record for a customer (Mark), and invalidating caches to ensure fresh content delivery.

Architecture Overview

The dispatch system consists of three integrated layers:

  • URL Routing Layer: CloudFront Functions rewriting `/track/` and `/book/` paths to the appropriate frontend files
  • Job Processing Layer: AWS Lambda function parsing job requests, generating tracking tokens, and managing job state
  • Data Persistence Layer: S3 bucket storing jobs.json with all active and historical job records

Technical Details: CloudFront Function Rewriting

The primary infrastructure component is the CloudFront Function responsible for request routing. Located at /Users/cb/Documents/repos/sites/quickdumpnow.com/cf/qdn-track-rewrite.js, this function intercepts requests and rewrites URLs without requiring a backend origin change.

The function handles two critical paths:

  • /track/* — Routes to the tracking dashboard where customers monitor job status
  • /book/* — Routes to the booking interface for new job requests

CloudFront Functions execute at the edge with extremely low latency (<1ms) because they run before the request reaches your origin. This is preferable to Lambda@Edge for simple rewrites because it reduces compute costs and eliminates cold-start latency. The function examines the request URI, applies conditional logic, and rewrites the path to point to the correct S3 object or backend endpoint.

Why CloudFront Functions instead of Lambda@Edge? For URL rewriting and path routing, CloudFront Functions provide sub-millisecond execution without billing per invocation. Lambda@Edge would be appropriate for complex business logic (authentication, dynamic headers, geo-blocking), but simple rewrites belong in CloudFront Functions.

Job Creation and State Management

When Mark's job was created, the system followed this workflow:

  1. Generated a unique job ID using UUIDv4 format: fcdc1c82-cb28-4dbe-****-************
  2. Created a tracking token (separate from job ID for security obfuscation)
  3. Built a job record containing: customer name, address (Soderblom Ave), service type, price ($600), and initial status
  4. Serialized the job object to JSON and appended to jobs.json in S3
  5. Made the tracking URL accessible at: https://quickdumpnow.com/track/{tracking-token}

The job record structure includes a STATUS_FLOW state machine defining valid transitions:

STATUS_FLOW = {
  'submitted': ['ready_for_pickup'],
  'ready_for_pickup': ['in_progress'],
  'in_progress': ['completed'],
  'completed': []
}

This prevents invalid state transitions (e.g., jumping directly from 'submitted' to 'completed') and ensures the customer-facing dashboard displays only legitimate status updates.

Frontend Implementation: Track Page and Dashboard

The customer-facing tracking page is a static HTML file at /Users/cb/Documents/repos/sites/quickdumpnow.com/frontend/track/index.html. When a customer visits with a tracking token, the page:

  • Extracts the token from the URL query parameter: ?t={token}
  • Calls the backend API to fetch the job record from S3
  • Matches the token against stored tracking tokens to verify access
  • Renders job details: address, status, pickup/completion times
  • Displays status as a visual pill with color coding (blue for submitted, yellow for in-progress, green for completed)

The dashboard at /Users/cb/Documents/repos/sites/quickdumpnow.com/frontend/dashboard/index.html provides an internal operations view with kanban-style columns for each job status. Dispatch staff can drag jobs between columns, which triggers backend status updates.

Why separate track and dashboard? The track page is read-only and customer-facing, optimized for clarity and load time. The dashboard is staff-facing and requires bidirectional updates. Separating concerns reduces complexity and allows different optimization strategies (the track page can be heavily cached; the dashboard needs fresh data on each load).

Deployment Pipeline

The deployment involved multiple stages to minimize risk:

  1. Stage 1: CloudFront Function Update — Fetched the current ETag of qdn-track-rewrite.js, updated the function with new `/book/` rewrite rules, published to the LIVE stage
  2. Stage 2: Frontend Deployments — Promoted staged track page, dashboard, and booking pages to production S3 buckets
  3. Stage 3: Cache Invalidation — Invalidated CloudFront distributions to flush edge caches and ensure fresh content
  4. Stage 4: Job Creation — Created Mark's job record and uploaded to jobs.json

Why staged deployments? Staging allows testing in a production-like environment before affecting real customers. If the track page had rendering bugs, customers wouldn't see them until the page is promoted to prod.

Cache Invalidation Strategy

After deployments, we invalidated specific paths on the CloudFront distributions:

  • Dashboard CloudFront distribution: Invalidated /dashboard/* and / (index)
  • quickdumpnow.com distribution: Invalidated /track/* and /book/* paths

CloudFront caches objects for 24 hours by default. Without invalidation, customers would see stale versions of files. Invalidations are processed within seconds at edge locations globally, though there's a per-distribution limit (3,000 per month on the default quota). We used wildcard patterns to invalidate all track and book paths with a single operation rather than invalidating each token individually.

Troubleshooting the "Job Not Found" Error

The initial error (job not found for token fcdc1c82cb284dbe) occurred because:

  1. The track page was pulling from jobs.json on S3
  2. The job lookup logic uses token matching: if the provided token doesn't exist in the jobs array, it displays "Job not found"
  3. The token wasn't in jobs.json yet because the job creation hadn't completed

Resolution: After creating Mark's job and uploading to S3, we verified the token was queryable via the track API endpoint directly, then promoted the updated jobs.json` to ensure the track page could access it.

Key Decisions

  • Static S3