Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

I'll read the key files from the session to extract specific technical details for the blog post. Reading the session context files to ground the post in real implementation details. ```html

Automating Git Snapshots and Code Review: Building Staff-Level Engineering Practices

This post documents the infrastructure and automation built to address a fundamental gap in solo and small-team engineering: ensuring that production code is versioned, reviewable, and auditable before it ships. The pattern addresses three staff-level practices: version control discipline, mandatory code review (even solo), and decision documentation that survives projects.

What Was Built

  • Automated git snapshot pipeline: Python script that commits unstaged changes from iCloud directories to a canonical git repository on a Lightsail instance, running on a fixed schedule via launchd.
  • Automated diff review workflow: A second script that fetches new commits, runs them through a structured review checklist, and reports findings via SMS for high-severity issues.
  • Decision documentation system: Formalized structure for decision docs, integrated into the digest workflow so team context survives across sessions.
  • Audit trail (CloudTrail): Multi-region CloudTrail trail logging all AWS API calls, with 90-day auto-expiry to manage cost.
  • Reminders in the digest: CSV-driven reminder system that surfaces recurring tasks in the daily digest without manual re-entry.

Technical Architecture

Git Snapshot Pipeline

The snapshot script (/Users/cb/bin/jada-git-snapshot.py) runs on a timer defined in ~/Library/LaunchAgents/com.jada.git-snapshot.plist. It operates on the canonical mirror repository at /Users/cb/icloud-jada-ops, which is cloned from a bare git repository on Lightsail.

launchctl load ~/Library/LaunchAgents/com.jada.git-snapshot.plist

The script:

  • Stages all unstaged changes in source directories (iCloud working directories for tools, configs, and ops scripts)
  • Creates a deterministic commit with timestamp and summary of what changed
  • Pushes to the Lightsail bare repository via SSH
  • Logs the operation to a persistent snapshot journal
  • On failure, SMS alert is triggered via a watchdog process

Why this pattern: iCloud is a working directory, not a repository. It has no history, no blame, no ability to revert. When a script that handles payments or client email gets edited, there's no diff to inspect. Mirroring to a canonical git repo solves this without moving the working directory — the canonical repo becomes the source of truth, and iCloud remains the deployment target.

Diff Review Workflow

The second script (/Users/cb/bin/jada-diff-review.py) runs via a separate launchd job (~/Library/LaunchAgents/com.jada.diff-review.plist). It:

  • Fetches the latest commits from the Lightsail repository
  • Extracts diffs for each changed file
  • Runs a structured checklist: secrets scanning, logic review, test coverage requirements, permission changes, dependencies
  • Generates a markdown report with findings severity (INFO, MEDIUM, HIGH)
  • For HIGH severity (credentials, dangerous permissions, missing tests), sends SMS alert
  • Logs all reviews to the digest for visibility

Why this pattern: Manual code review is a habit that dies when you're working solo. Automating the review checklist means every commit gets examined by the same lens, whether or not a human is available. It catches the class of bug that nightly test gates miss until after deployment. The SMS alert creates a forcing function — unreviewed changes don't silently accumulate.

Decision Documentation

Each decision is written as a one-page markdown file in /Users/cb/icloud-jada-ops/decisions/, with a standard structure: problem, constraints, decision, rationale, consequences. Files are named by slug (estate-git-and-diff-review.md, cloudtrail-audit.md, etc.) and linked from a CONTEXT.md index.

Why this pattern: Decision maps tell you what was chosen; decision docs tell you why it was chosen and what tradeoffs were accepted. When returning to a project months later (or handing it to a colleague), the decision doc is far more useful than git log. Staff engineers document the reasoning, not just the outcome.

Infrastructure Details

Lightsail Bare Repository

A bare git repository is hosted on the Lightsail instance and serves as the canonical source. The repository is initialized as:

git init --bare /path/to/jada-estate.git

SSH access is restricted to a keypair, and the bare repo has a post-receive hook that:

  • Updates the working branch pointer
  • Runs post-commit checks (e.g., verifying no secrets were added in this push)
  • Notifies watchers of new commits

CloudTrail Audit Trail

A CloudTrail trail named jada-account-trail was created to log all AWS API calls across regions. Logs are stored in an S3 bucket with a 90-day expiry lifecycle rule to manage costs.

aws cloudtrail create-trail --name jada-account-trail --s3-bucket-name jada-cloudtrail-logs --is-multi-region-trail

This trail captures every AWS API call, including IAM changes, secret deletions, and permission modifications. The 90-day window is a reasonable tradeoff: long enough to detect and remediate drift, short enough to stay inexpensive.

Reminders System

The digest builder now reads a CSV file (/Users/cb/icloud-jada-ops/digest/reminders.csv) with columns: task, frequency, due_date, assigned_to, status. Reminders are inserted into the daily digest under a NEEDS YOU section, and persist until manually cleared.

Key Decisions

  • One-way sync from canonical to working: The Lightsail repo is the source of truth; iCloud is a deployment target. Changes go from canonical → iCloud, not the reverse. This prevents accidental loss of history.
  • Automated review over manual: The diff-review script is deterministic and runs on every commit, not just when a human happens to remember. It catches secrets, permission creep, and missing tests — exactly the bugs that manual review often misses because we're tired.
  • Scheduled snapshots over per-action: Snapshots run at fixed times (e.g., 20:05 every evening), not per-edit. This prevents snapshot spam, batches related changes into one commit, and makes the schedule predictable for monitoring.
  • 90-day CloudTrail retention: Long enough to detect drift, short enough to avoid storage costs spinning out of control. Given the account size and activity level, this is a reasonable window for incident investigation.

Integration Points

The snapshot and review jobs are wired into the existing SMS alert system. High-severity findings from diff-review (secrets, dangerous IAM changes, missing tests) trigger SMS immediately. The daily digest includes a summary of all snapshots and reviews from the past 24 hours.

The reminders CSV is version-controlled in the snapshot repo, so reminders survive across sessions and can be reviewed in the git history.

What's Next

  • Failure domain isolation: Mine the CloudTrail logs and past incidents to identify the highest-risk operations, then wrap them in additional gates (approval workflows, dry-run validations, staged rollouts).
  • Test coverage from past fires: Each past incident should have a test case added to prevent regression. The diff-review script will flag commits that touch incident code but lack new test coverage.
  • Dashboard: Surface snapshot and review trends in a simple dashboard so you can spot when the pipeline is silently failing or review findings are being ignored.
```