Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Bifurcating Claude API Usage: Subscription vs. Cost-Optimized Routing

The Problem: Unsustainable API Token Spend

Running production systems against Claude API had become financially untenable. Monthly token costs were running ~$1,500, with no clear path to optimization without sacrificing reliability on critical paths. The core issue: every request, regardless of complexity or error tolerance, was being routed through the same cost tier.

The ideal state would be:

  • Interactive developer work (Claude Code, real-time problem-solving) → Claude subscription (predictable, high-quality)
  • Decomposed, fault-tolerant background tasks → cheaper API tier on EC2 (Haiku model, high error tolerance)
  • No conflicts between authentication methods when switching contexts

What Was Built

This session implemented a shell-environment-based routing layer that bifurcates Claude API usage by context without touching settings files or requiring manual token management.

Technical Architecture

Authentication Routing Mechanism

The solution leverages a simple environmental principle: Claude Code's CLI respects shell environment variable precedence.

  • Interactive shell (`claude` command): Removed ANTHROPIC_API_KEY from ~/.zshrc exports. The keychain-stored OAuth token (stored by Claude Code under credential ID Claude Code-credentials, account cb) becomes the default auth method. This routes requests to claude.ai subscription billing.
  • Programmatic/farm-out work: The API key remains in ~/Documents/repos/repos.env. Scripts that require cheap API routing explicitly source this file: source ~/Documents/repos/repos.env. This keeps the key out of the interactive environment while remaining available for subprocess invocation.

Why this approach: Setting `forceLoginMethod` in `~/.claude/settings.json` appeared promising but is actually enterprise/managed-only and silently ignored in personal Claude setups. The real lever is pure shell environment state. When `ANTHROPIC_API_KEY` is absent, Claude Code's CLI falls back to the stored OAuth credential automatically.

EC2 Farm-Out Infrastructure

The cheap-tier Claude instance runs on a Lightsail box provisioned for background task automation:

  • Host: ubuntu@34.239.233.28 (Lightsail instance, internal IP ip-172-26-6-34)
  • SSH key: ~/.ssh/LightsailDefaultKey-us-west-2.pem (passwordless, verified with batch-mode probe)
  • Claude service: `/usr/bin/claude` installed and active via jada-agent.service systemd unit
  • Authentication on box: API key and model parameters are injected per-invocation by the daemon, not stored in the login shell — requiring any farm-out wrapper to pass them explicitly

The EC2 instance runs a systemd service that manages Claude daemon lifecycle. Secrets (API key for cheap tier) are stored in an `EnvironmentFile` referenced by the service unit, keeping them out of shell history and process listings.

Concrete Implementation Details

Shell Configuration Changes

In ~/.zshrc:

# REMOVED: export ANTHROPIC_API_KEY=...
# This allows keychain OAuth to take precedence for interactive work

# Added: burst-valve function for rate-limiting cheap Claude calls
burst-valve() {
  # Rate-limit logic for farm-out requests
  # Prevents token exhaustion on low-cost tier
}

Why remove the global export: Any exported environment variable takes precedence over keychain credentials in Claude Code's resolution order. Removing it forces the fallback chain to authenticate via stored OAuth.

Script-Side Farm-Out Pattern

For Python tools like ~/Documents/repos/tools/gmb_lead_responder.py and ~/Documents/repos/tools/carole_digest.py:

#!/usr/bin/env python3
import os
import subprocess

# Load cheap-tier credentials only when needed
def load_cheap_api_config():
    with open(os.path.expanduser('~/Documents/repos/repos.env'), 'r') as f:
        for line in f:
            if line.startswith('ANTHROPIC_API_KEY='):
                return line.split('=', 1)[1].strip()
    return None

def invoke_cheap_claude(prompt, model='claude-3-5-haiku-20241022'):
    key = load_cheap_api_config()
    env = os.environ.copy()
    env['ANTHROPIC_API_KEY'] = key
    
    result = subprocess.run(
        ['claude', 'chat', '-m', model, prompt],
        env=env,
        capture_output=True,
        text=True
    )
    return result.stdout

Key design decision: Scripts explicitly source/load the API key only when executing Claude calls. This preserves the interactive shell environment (no API key) while enabling programmatic access where needed.

Atomization of Tasks

Before routing to cheap Claude, work must be sufficiently decomposed:

  • GMB lead responder: Pre-filtered by subscription Claude into structured JSON, then cheap Claude formats responses for Gmail
  • Carole digest: IMAP ingestion and filtering done by subscription Claude, email HTML templating delegated to cheap tier
  • Error recovery: Failures on cheap tier trigger a retry flag but do not block—digest generation degrades gracefully

This keeps high-quality work on subscription (reading customer intent, decision logic) while cheap tier handles templating, formatting, and idempotent transformations.

Infrastructure Verification

All connectivity was verified before implementation:

# Batch-mode SSH probe (no interactive prompt)
ssh -o BatchMode=yes -i ~/.ssh/LightsailDefaultKey-us-west-2.pem \
    ubuntu@34.239.233.28 'systemctl status jada-agent.service'

# Confirmed: jada-agent.service is active (running)
# Confirmed: /usr/bin/claude executable present and functional

Cost Projections

Token spend is projected to drop from ~$1,500/month to ~$75–150/month:

  • Subscription usage: ~$20/month (interactive Claude Code work remains subscription-billed)
  • Cheap API tier (Haiku model): ~$55–130/month (background tasks, high error tolerance)
  • Savings: ~90% reduction while maintaining quality on critical paths

What's Next

Three kanban cards were created to track remaining work:

  • Card 1: Full integration test of GMB responder with cheap-tier routing
  • Card 2: Carole digest production deployment and monitoring setup
  • Card 3: Billing reconciliation and actual cost tracking against projections

The architecture is now in place to support additional background tasks