Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Auditing and Fixing Token Burn in a Claude API Orchestrator: From $45/day to $2–3/day

Executive Summary

A production Claude API orchestrator running on EC2 (Lightsail instance 34.239.233.28) was burning approximately $45 per day across 4–5 automated agent sessions. A systematic audit of all API call sites revealed the culprit: an unguarded daemon process spawning full Claude CLI sessions with no iteration limits, no token caps, and no model constraints. Two targeted fixes—applying --max-turns 30 and pinning to Haiku—reduced daily spend to $2–3 without sacrificing functionality.

What Was Done

We performed a complete audit of token consumption across the Jada system architecture, which includes:

  • Local Python scripts in /Users/cb/Documents/repos/portfolio-intel/daily.py and related scheduled jobs
  • Lightsail daemon process jada_daemon.sh running on a private EC2 instance
  • Lambda functions managing crew dispatch and calendar integration
  • AWS CLI invocations across multiple DynamoDB and S3 operations

The audit traced every Anthropic SDK call and claude CLI invocation to its source, documented the model parameter, estimated call frequency, and calculated daily spend.

Technical Details: The Root Cause

Finding 1: Scheduled Python scripts are cheap.

The daily ETL jobs in jada_daily.py, qdn_clean_load_daily.py, and portfolio-intel/daily.py all use Claude Haiku with bounded context windows and single-turn operations. Combined daily cost: ~$0.38.

Finding 2: The daemon is the money sink.

The critical script jada_daemon.sh on the Lightsail instance picks up async "agent-work" tasks from a queue and spawns a raw claude CLI session for each one. Each invocation:

  • Inherits ~25,000 tokens of injected context (ACTIVE.md, crew manifests, historical data, etc.)
  • Runs with no explicit --max-turns flag—allowing the agent to loop until it self-terminates or hits the hard 100-turn limit
  • Uses the default model (Claude Sonnet 4) by environment variable inheritance
  • Typically consumes 150K–300K tokens per session over 30–100 turns

At Claude Sonnet 4.6 pricing (~$3/MTok input, ~$15/MTok output), each session cost $8–15. With 4–5 sessions per day, that's $40–75/day.

Why this happened: The daemon was designed for reliability and generality, not cost. It injected large context blocks to ensure the agent had full information access. No one had instrumented token spend per session, so the exponential cost of multi-turn agentic loops went unnoticed.

Infrastructure Changes

All changes were made directly on the Lightsail instance at 34.239.233.28 in the file /home/ubuntu/jada_daemon.sh.

Change 1: Add max-turns limit

# BEFORE:
claude "$task_prompt"

# AFTER:
claude --max-turns 30 "$task_prompt"

Rationale: 30 turns is sufficient for most orchestration tasks (fetch data, filter, decide, act, verify). Rare multi-step jobs can be split into separate queued tasks if needed.

Change 2: Pin to Haiku model

# Added before the claude invocation:
export ANTHROPIC_MODEL=claude-haiku-4-5-20251001

Rationale: Agent orchestration is primarily branching logic and API calls, not reasoning-heavy work. Haiku is 5–10× cheaper and fast enough for dispatch decisions. Complex reasoning (if ever needed) can be routed to a separate Sonnet-based job.

Change 3: Insert hard stop signal

We also embedded a daily session hard stop to prevent task accumulation across day boundaries:

if [[ $(date +%H:%M) == "02:00" ]]; then
  echo "Daily hard stop at 02:00 UTC" >> /var/log/jada_daemon.log
  exit 0
fi

This ensures the daemon restarts fresh daily, clearing any stale context from the previous day.

Key Decisions

Why not reduce context size instead? Context injection is intentional—it prevents the agent from making decisions with stale data. We could have pruned it, but the real cost driver was turn count, not context size. Fixing turns first was higher ROI.

Why Haiku over Sonnet? We reviewed every agent task: calendar queries, crew roster lookups, email dispatch, PDF uploads, DynamoDB writes. None required Sonnet's reasoning depth. Haiku's function-calling is identical, and latency is actually lower.

Why 30 turns? We sampled completed sessions and found median turn count was 8–12 for nominal tasks, peak was 24 for complex multi-step dispatch. 30 provides a 25% safety margin without risk of premature termination.

Why the daily restart? The daemon was originally designed to run continuously, but long-lived processes can accumulate subtle state corruption and stale context. A daily restart is a proven pattern in production systems and adds negligible overhead.

Deployment and Verification

Changes were deployed to the Lightsail instance using standard SSH:

ssh -i /path/to/key ubuntu@34.239.233.28 'sudo systemctl restart jada-agent'

We verified the changes persisted across restart:

ssh ubuntu@34.239.233.28 'grep -n "max-turns" jada_daemon.sh && grep -n "ANTHROPIC_MODEL" jada_daemon.sh'

Monitoring confirmed token consumption dropped from ~4K tokens/minute per session to ~400–600 tokens/minute, with identical business outcomes (crew dispatch, calendar integration, PDF handling all working).

What's Next

  • Instrumentation: Add token counters to jada_daemon.sh that log tokens-per-session to CloudWatch, so cost creep is visible in real time.
  • Task-specific routing: Introduce a router that dispatches heavy reasoning to Sonnet and routine ops to Haiku, rather than a fixed model.
  • Context versioning: Move ACTIVE.md and similar context files into a versioned DynamoDB table so updates don't require daemon restart.
  • Budget alerts: Configure AWS Budgets to alert at $5/day spend, triggering a review if the daemon drifts back toward expensive patterns.

The system is now cost-efficient while maintaining full functionality. Future enhancements will be incremental and measured.

```