Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

Diagnosing Dual Email Delivery Failures: Gmail SMTP Relay Misconfiguration vs. Exchange Online Rejection

During a recent support investigation, we encountered a thread containing two distinct email delivery failures originating from the same sender but failing at different points in the delivery pipeline. This post documents the diagnostic approach and root causes, which will be valuable for anyone operating multi-channel email infrastructure with both Gmail aliases and AWS SES.

The Problem

Three emails sent from admin@queenofsandiego.com (via jadasailing@gmail.com) were not delivering to their intended recipients. The bounce messages were vague enough to obscure the actual failure points. Initial investigation required distinguishing between client-side authentication failures, DNS/authentication record issues, and downstream recipient server rejections.

Diagnostic Approach

We followed a layered investigation strategy:

  • Layer 1: Credential & Configuration Audit — Located environment files and verified AWS SES credentials across regions and AWS profiles
  • Layer 2: DNS & Authentication Infrastructure — Validated all SES domain verification records (SPF, DKIM, DMARC, custom MAIL FROM)
  • Layer 3: SES Account Health — Checked suppression lists, complaint rates, and account-level configuration sets
  • Layer 4: Application-Level Email Logic — Traced email sending code paths (both Lambda functions and daemon scripts) to understand sender configuration per campaign
  • Layer 5: Exchange Online Server-Side — Investigated recipient domain MX records and DKIM validation

Finding 1: Gmail "Send mail as" SMTP Relay — Stale Credentials

The first failure was client-side, at Gmail's relay layer. When using Gmail's "Send mail as" feature with an external SMTP server, Gmail stores SMTP credentials (username and password) in an encrypted form. These credentials point to AWS SES's SMTP endpoint.

Root Cause: The SMTP credentials configured in Gmail for the admin@queenofsandiego.com alias had expired or been rotated. AWS SES SMTP credentials are derived from IAM access keys and have a specific generation algorithm (SigV4). If the underlying IAM key is rotated or a new SMTP credential set is generated in the SES console, the old credentials become invalid immediately.

Evidence: The bounce message referenced "misconfigured or out of date" SMTP settings — this language is Gmail-specific and indicates the relay authentication failed before the message ever reached SES infrastructure.

Resolution Path: To fix this, regenerate SES SMTP credentials:

AWS Console → SES Console → SMTP Settings → Create SMTP Credentials
(Credentials are account-scoped and region-scoped; verify you're in the correct region)

Then update the Gmail "Send mail as" alias configuration:

Gmail Settings → Accounts and Import → Send mail as
→ Edit admin@queenofsandiego.com
→ Update SMTP Server: email-smtp.[REGION].amazonaws.com (e.g., us-east-1)
→ Port: 587 (TLS) or 465 (SSL)
→ Username: [new SES SMTP username]
→ Password: [newly generated SES SMTP password]
→ Test connection

Finding 2: Exchange Online DKIM Validation Rejection

The second failure occurred at the recipient's mail server (Exchange Online), after the email successfully left our infrastructure. This was a more subtle issue because the email reached SES and was accepted, but rejected downstream.

DNS Infrastructure Verification: We validated all records required for SES sending:

  • SPF Record — Present on queenofsandiego.com, includes SES IP range authorization
  • DKIM CNAME Records — All three DKIM tokens (CNAME records) were correctly in place pointing to SES token endpoints
  • DMARC Record — Configured with appropriate policy (p=quarantine or p=reject depending on business requirements)
  • Custom MAIL FROM Domain — Configured at the SES identity level to use a subdomain (typically bounce.queenofsandiego.com)

Root Cause: Despite our DNS records being correct, Exchange Online's DKIM validation was failing. This typically indicates one of two scenarios:

  • DNS propagation delay (TTL hadn't expired on recipient's cached records)
  • Recipient's mail server cached a DKIM verification failure from an earlier send attempt, before all records were in place
  • Edge case: SES region mismatch — if emails were being sent from a Lambda function in us-west-2 but DKIM tokens were verified in us-east-1

Investigation Commands Used:

dig queenofsandiego.com TXT              # Check SPF and DMARC
dig [TOKEN1]._domainkey.queenofsandiego.com CNAME  # Validate DKIM token 1
aws ses get-identity-dkim-attributes --identity-name admin@queenofsandiego.com  # SES perspective
aws ses get-identity-mail-from-domain-attributes --identity-name admin@queenofsandiego.com

Infrastructure Context: SES Multi-Region Architecture

Our investigation uncovered a multi-region email infrastructure:

  • Primary Region (us-east-1): Verified SES identity for admin@queenofsandiego.com, suppression lists, and complaint tracking via SNS → CloudWatch
  • Secondary Region (us-west-2): Lambda functions for campaign sending (jada-blast-sender, jada-blast-scheduler), asset storage in S3, and separate suppression list
  • Email Sending Paths:
    • Gmail relay → SES SMTP (usually us-east-1)
    • Lambda functions → SES API (region-dependent, tied to Lambda VPC or explicit region config)
    • Daemon scripts on EC2 Lightsail → SES API (region specified in boto3 client)

Critical Finding: Lambda functions in us-west-2 were sending emails without an explicit SES identity verified in that region. This caused emails to fall back to a default sending identity or fail silently. Suppression lists are region-specific in SES — an address bouncing in us-east-1 will not automatically suppress in us-west-2.

Key Decisions Going Forward

  • Standardize on Single Region: Consolidate SES sending to us-east-1 for all email campaigns. Cross-region sending adds operational complexity with minimal benefit for our use case.
  • Verify Credentials in Automation: Add pre-send validation in Lambda and daemon scripts to check SES identity verification status and DKIM/SPF alignment before attempting bulk sends.
  • Suppress List Synchronization: If multi-region is necessary, implement a Lambda function that syncs suppression data from all regions to a canonical DynamoDB table to prevent re-sending to bounced addresses.
  • Gmail Relay