Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Building an Automated Email Outreach Engine for BSSD: Harvesting, Sending, and Scheduling at Scale

What We Built

We implemented a complete email outreach system for burialsatseasandiego.com targeting San Diego funeral homes, churches, hospices, and related organizations. The system consists of three core components: an intelligent email harvester that extracts contacts from websites with multiple fallback strategies, a sender service that delivers templated emails through AWS SES with suppression list management, and a LaunchAgent scheduler that runs campaigns on a defined cadence.

The Email Harvester Challenge

Extracting email addresses from funeral-home and church websites proved more complex than a simple regex pattern. Most sites either block automated requests or use client-side JavaScript to render content. We built harvest_emails.py with a multi-layered approach:

  • Direct HTTP fetching — First attempt, catches ~5-10% of sites
  • Cloudflare email obfuscation decoding — Many sites use Cloudflare's data-cfemail attribute which encodes email addresses client-side. We implemented the XOR decoding algorithm that Cloudflare uses, allowing us to extract emails from obfuscated HTML without JavaScript execution
  • Contact page following — When the homepage yields nothing, the harvester follows links to dedicated contact pages
  • HTTP status probing — Before attempting to harvest a site, we probe its HTTP status to identify unreachable targets, reducing wasted requests
  • Graceful degradation — Sites with no findable email are logged separately for manual follow-up or phone outreach

Across 76 funeral-home websites and 31 church/hospice/cremation society targets, we successfully harvested 20+ emails from funeral homes and refined a church/hospice dataset of 31 organizations, achieving a working email extraction rate that allowed us to proceed with outreach without manual collection overhead.

The Sender Architecture

We built send_bssd_outreach.py following the proven patterns from the hotel outreach engine (the predecessor system). Key architectural decisions:

  • Suppression list enforcement — All sends check against /Users/cb/icloud-jada-ops/email-lists/suppression/burialsatseasandiego.csv before transmission, preventing duplicate emails to opted-out contacts
  • SES identity verification — We verified burialsatseasandiego.com as a sender identity with AWS SES, ensuring deliverability and domain reputation protection
  • CSV-driven campaigns — Outreach targets are loaded from CSV files (e.g., churches.csv merged from multiple research passes). The sender gracefully handles missing files (dry-run mode)
  • Template rendering — Each email is personalized with recipient name and organization, rendered from a base template
  • Dry-run validation — Before live sends, the system runs a dry-run that logs all emails that would be sent without actually contacting SES

The sender is invoked by a LaunchAgent, which we configured at /Users/cb/Library/LaunchAgents/com.jada.bssd-outreach.plist. This plist defines the schedule (daily at a specified time), the Python executable, and the script path. LaunchAgent ensures the campaign runs consistently without manual intervention.

Testing and Validation Strategy

We followed test-driven development throughout. The test suite in /Users/cb/icloud-repos/sites/burialsatseasandiego.com/tests/test_bssd_outreach.py validates:

  • Email harvesting correctness against known sites
  • Cloudflare obfuscation decoding (XOR algorithm verification)
  • Suppression list enforcement (confirming no send to suppressed addresses)
  • CSV parsing and personalization logic
  • Dry-run output format

All 11 tests pass before any live deployment. This prevents silent failures in the outreach pipeline.

Infrastructure and Deployment

The burialsatseasandiego.com site itself is hosted on S3 with CloudFront distribution for CDN acceleration and HTTPS termination. During this engagement, we:

  • Verified SES sender identity for the domain
  • Added Google Analytics tracking to the homepage to monitor inbound traffic from outreach campaigns
  • Updated the site footer with a referral link (deployed through staging first, then promoted to production)
  • Confirmed DNS MX records are correctly set to receive replies

Deployments follow a two-stage process: we test changes on a staging environment (verifying footer links render correctly), then promote to production. This reduces the risk of breaking the live site.

Why This Architecture

Multi-layer harvesting: Funeral home websites are notoriously difficult to scrape. By implementing five different extraction strategies, we maximized coverage without requiring manual data collection, which would have been prohibitively time-consuming for 100+ targets.

Suppression list enforcement: Sending to opted-out recipients damages domain reputation and violates email regulations. The suppression check is not optional—it's a guard rail built into the sender itself.

LaunchAgent scheduling: Rather than requiring manual campaign triggers or cron jobs on a shared server, LaunchAgent runs locally and persists across reboots. It's macOS-native and requires no external scheduler.

CSV-driven campaigns: Using CSVs allows non-technical stakeholders to update target lists without code changes. The system handles missing files gracefully, enabling dry-run testing before a CSV is populated.

Staging deployment: Verifying footer links and GA tags on staging before production prevents live-site breakage and ensures metrics collection works end-to-end.

What's Next

The current system is running daily outreach to funeral homes and churches. The next phases include:

  • Expanding the church/hospice dataset to cover the full San Diego County (currently at 31 rows, target 100+)
  • Implementing reply capture and lead scoring based on engagement (opens, clicks, replies)
  • A/B testing email templates to optimize call-to-action click rates
  • Integrating reply forwarding to a monitored inbox so leads are not lost
  • Adding lifecycle automation (follow-ups to non-respondents after 7 days)

The foundation is solid: email extraction is working, sending is tested and scheduled, and the site is equipped to measure results. From here, we optimize campaign performance and scale to adjacent vertical markets.

```