Sail JADA — Engineering Log

Technical work on sailjada.com and the JADA booking systems

Investigating SMS Infrastructure: Mapping Twilio Integration in the Voice Agent Architecture

During a recent development session, I conducted a comprehensive audit of our SMS and Twilio infrastructure to understand how voice agents handle inbound and outbound messaging. The investigation revealed a distributed credential management pattern and identified gaps in our current SMS data access workflow. This post documents the technical findings and architectural decisions that emerged from this exploration.

What Was Done

The primary objective was to trace SMS capabilities within our voice agent system and identify where Twilio credentials are stored and referenced. The investigation involved:

  • Scanning the codebase for SMS and Twilio-related files and imports
  • Auditing credential storage patterns across multiple configuration locations
  • Analyzing the voice agent server implementation to understand SMS infrastructure
  • Reviewing phone_agent.py to understand SMS-specific capabilities
  • Checking local .env files and secrets management in the voice agent directory
  • Verifying which Twilio credentials are referenced versus actually stored

Technical Details: Credential Storage Patterns

One of the key findings was that Twilio credentials follow a split storage pattern rather than a unified approach. The credentials—specifically TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN—are referenced in multiple locations but not necessarily stored in the primary .secrets/repos.env file.

The voice agent scripts explicitly import and reference these credentials:


# Example pattern found in voice agent implementations
from twilio.rest import Client

account_sid = os.getenv('TWILIO_ACCOUNT_SID')
auth_token = os.getenv('TWILIO_AUTH_TOKEN')
client = Client(account_sid, auth_token)

However, the actual storage location varies:

  • Primary location (expected): .secrets/repos.env — This is where environment variables are typically centralized, but the Twilio credentials were not found here during the audit
  • Secondary locations (discovered): Local .env files within the voice agent directory structure, or potentially stored on the Lightsail deployment server
  • Runtime resolution: The scripts expect these variables at runtime, indicating they must be injected during deployment or available in the execution environment

Voice Agent Architecture and SMS Capabilities

The voice agent system, implemented in phone_agent.py, includes SMS handling as part of its broader communication capabilities. This is architecturally significant because it means the voice agent isn't purely for voice interactions—it's a unified communication endpoint that can handle multiple channels (voice, SMS, potentially others).

The tools directory within the voice agent module contains specialized handlers for different communication types. This modular design allows:

  • Channel abstraction: Individual tools handle SMS-specific logic separately from voice logic
  • Scalability: New communication channels can be added as additional tools without modifying core agent logic
  • Conditional routing: Incoming messages can be routed to appropriate handlers based on channel (SMS vs. voice)

The infrastructure assumes a Twilio phone number (+16199867344 in this case) is provisioned and configured to receive inbound SMS, with webhooks pointing back to our voice agent server for message processing.

Infrastructure and Deployment Considerations

Several infrastructure patterns emerged from this investigation:

Credential Injection at Deployment: Rather than baking credentials into Docker images or committing them to repositories, the system relies on environment variable injection. This is deployed through:

  • Lightsail instance environment variables (most likely location based on the investigation context)
  • Docker container orchestration secrets management
  • GitHub Actions secrets for CI/CD pipelines (if applicable)

SMS Message Flow: The expected flow is:

  1. Inbound SMS arrives at Twilio phone number (+16199867344)
  2. Twilio webhook triggers HTTP POST to voice agent server endpoint
  3. Voice agent server processes message through SMS tools
  4. Response (if applicable) is sent back via Twilio API using the authenticated client
  5. Message state is potentially stored in a database or logging system

Key Findings and Decisions

Why credentials aren't in repos.env: This is likely intentional. The repos.env file is checked into source control (albeit in a private repository), while truly sensitive credentials like Twilio auth tokens should only exist in runtime environments. This follows the principle of least privilege distribution—credentials are only accessible where they're actually needed.

The SMS data access gap: The investigation revealed that reading the SMS inbox from +16199867344 requires the Twilio credentials to be present in the accessible environment. Without them, the voice agent can receive messages through webhooks but cannot proactively fetch message history. This is a design constraint worth documenting:

  • Reactive SMS handling: Messages are processed as they arrive via webhooks
  • Inbox access gap: Historical message retrieval requires direct Twilio API access with valid credentials
  • Mitigation: Messages should be logged to a persistent data store (database) when received, enabling historical queries without requiring Twilio API credentials

Credential location resolution options: The investigation identified two viable paths for making SMS credentials available:

  1. Local development: Add credentials to .secrets/repos.env locally (and to .gitignore to prevent commits)
  2. Production/staging: Pull credentials from the Lightsail server where they're already deployed, via SSH access to the instance or AWS Systems Manager Parameter Store

What's Next

To enable full SMS read capability and improve the overall infrastructure:

  • Credential audit: Verify whether Twilio credentials exist on the Lightsail instance and document their exact location
  • Secrets management standardization: Consider implementing AWS Secrets Manager or similar for centralized credential distribution across development, staging, and production environments
  • SMS message persistence: Implement database logging for all SMS messages received, enabling historical queries independent of Twilio API availability
  • Documentation: Create a runbook documenting the SMS infrastructure setup, credential locations, and troubleshooting procedures
  • Testing: Add integration tests that verify SMS message delivery and processing without requiring live Twilio credentials in test environments

The architecture is sound, but the credential distribution pattern requires explicit documentation and careful environment-specific configuration to function properly across development, staging, and production.