Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

```html

Plan-Billed Claude Agent on Lightsail: Verification, Discovery, and the Lane-Routing Bug

This post covers the infrastructure audit and live testing of the JADA ticket-runner agent hosted on a Lightsail instance, billing to plan usage instead of API metering. We discovered a critical bug blocking agent-work tickets and verified the cross-account billing separation that keeps operational costs predictable.

Architecture Overview

The agent runs on a single Lightsail instance (jada-agent, IP 34.239.233.28) in us-west-2, executing a systemd service (jada-ticket-runner.service) that polls the progress.queenofsandiego.com site for note-driven card updates. The instance is logged into Claude as c.b.ladd@gmail.com with a plan-subscription account type (pro), ensuring all tokens bill to the subscription tier, not metered API calls.

Billing Strategy: Plan Over API

The old metered API key was already disabled in the box's .bashrc (commented as #DISABLED-PLAN-BILLING), creating a hard boundary: the runner cannot fall back to the API even if the plan account fails to authenticate. This was deliberate—to prevent accidental overspend on a heavily-used background service. We verified the setup by running a headless test: 

env -u ANTHROPIC_API_KEY claude -p "say OK"

The runner responded correctly, confirming plan billing is live and the fallback is truly absent. The account separation is intentional:

  • Lightsail (jada-agent): c.b.ladd@gmail.com, plan-billed, runs background agent loops
  • Mac (this laptop): jadasailing@gmail.com, local plan use, no changes made

This separation means billing for the 24/7 agent loop is isolated from interactive development work, making cost tracking straightforward.

The Service Configuration

The systemd unit at /etc/systemd/system/jada-ticket-runner.service runs as an unprivileged user and loads environment variables including:

  • TICKET_LANE=todo — specifies which card lane to monitor (should be agent-work)
  • AWS_PROFILE=queenofsandiego — routes AWS calls to the correct IAM role
  • PATH — ensures Python3 and tools are in scope for child processes

The service is enabled and active, but the live audit revealed it's pointing at the wrong lane: TICKET_LANE=todo instead of TICKET_LANE=agent-work. This means the runner never sees agent-work tickets—they queue up unprocessed.

The Lane-Routing Bug: 11 Tickets Blocked

Manual testing confirmed the fix: when the lane is overridden to agent-work, the runner fetches all 11 pending tickets (Keeli email automation, unsubscribe cleanup, tech-blog ICM scaffolding, Daniela trip sheet, Capt Darrell hours reconciliation, etc.). But in live mode, the runner processes zero because it's looking in the wrong queue.

The fix is a single edit to the systemd environment variable: 

sudo sed -i "s/^Environment=TICKET_LANE=todo/Environment=TICKET_LANE=agent-work/" /etc/systemd/system/jada-ticket-runner.service
sudo systemctl daemon-reload
sudo systemctl restart jada-ticket-runner

This corrects the lane, reloads the systemd config, and restarts the service. Manual verification with systemctl is-active jada-ticket-runner confirms the service remains healthy.

Why This Matters: The Ticket-Driven Architecture

The progress site uses a note-driven workflow: a card note in the agent-work lane is the trigger for a task. The runner periodically fetches cards from the configured lane, deserializes the note payload, and executes it. The lane-based routing allows the same runner to be pointed at different queues (e.g., todo for debugging, agent-work for production) without code changes—just an environment variable edit and a restart.

This design choice trades a bit of state management (lane awareness) for flexibility: future deployments can scale to multiple runners, each watching different lanes, without duplicating logic.

Infrastructure Checklist

  • Lightsail instance: active, SSH reachable, Claude plan-logged-in
  • systemd service: enabled and running on the instance
  • progress.queenofsandiego.com: HTTP 200 response, CloudFront live (dist ID visible in upstream config)
  • AWS credentials: IAM profile queenofsandiego in use, no hardcoded keys in service env
  • Billing isolation: plan account locked in, no API key available as fallback
  • Lane configuration: BROKEN — currently todo, needs agent-work

Cross-Account Safety

A key decision: never share the plan account between local dev and the Lightsail agent. The old metered key was stored on the box and (redundantly) as a shared environment variable, but the new plan-only setup enforces separation at the OS level—no API key means no cross-cutting concern. This is why the Mac account remains untouched and unaffected by Lightsail changes.

What's Next

  • Apply the lane fix — restart the runner on agent-work to unblock the 11 queued tickets
  • Audit ticket latency — after the fix, measure how long it takes for a new note-trigger to execute; systemd polling interval may need tuning
  • Plan for multi-instance scale — if ticket volume grows, the lane-based design allows a second runner to monitor a different lane (e.g., backlog) without conflict
  • Monitor Lightsail static IP stability — the instance is on a static public IP; confirm firewall rules and SSH key rotation policy are current

Technical Takeaways

Plan-subscription billing removes the risk of runaway metered costs on always-on infrastructure. Environment-driven configuration (lane routing) decouples deployment from code. And cross-account separation keeps billing clean: one account per role (interactive vs. background). The lane-routing bug is a simple fix—a one-line sed edit and a service restart—but it highlights the importance of live verification over assumption.

```