Building a Presigned-URL Upload Pipeline for AWS Lambda + S3: Securing Customer File Uploads Without Public Buckets

```html

What Was Done

We implemented a presigned-URL generation system for the quickdumpnow.com dashboard to enable customer file uploads directly to S3 via Lambda, without exposing the bucket publicly. The work involved:

Adding a new /upload-url endpoint to the Lambda function that generates time-limited presigned URLs
Wiring job drawer "capture" buttons to request presigned URLs before file upload
Integrating presigned GET URLs into the photo list response so customers can view uploaded images securely
Deploying updated Lambda code with new routes while maintaining backward compatibility

The Problem We Solved

The dashboard previously had an upload-presign Lambda function, but it wasn't integrated into the main dashboard workflow. The job drawer had placeholder buttons for capturing photos, but no wiring to request upload credentials. Additionally, the S3 bucket hosting the dashboard (dashboard.quickdumpnow.com) is private (CloudFront-only), so we needed a secure pattern to:

Issue time-limited upload credentials to customers
Allow customers to PUT files directly to S3 without exposing bucket policies
Return secure URLs for viewing uploaded photos back through the dashboard

Technical Architecture

Lambda Function Updates

Modified /Users/cb/Documents/repos/sites/dashboard.quickdumpnow.com/lambda/lambda_function.py across five iterations to:

Add presigned URL generation: Created a new route handler that accepts POST requests to /upload-url, validates the job ID and customer session, then returns an AWS-signed PUT URL
Integrate S3 client: Imported boto3's S3 client and configured it to use the Lambda's execution role (which has s3:PutObject permissions on the qdn-uploads bucket)
Implement URL expiration: Set presigned URLs to expire in 15 minutes (900 seconds), a security best practice for time-limited credentials
Extend photo listing: Updated the list_photos function to generate presigned GET URLs for each uploaded image, allowing secure retrieval without bucket public access

The Lambda execution role (qdn-lambda-role) already had the necessary permissions via attached policies. We verified inline policies and confirmed s3:GetObject and s3:PutObject actions were authorized on the qdn-uploads bucket.

API Gateway Route Addition

Added the /upload-url route to the existing API Gateway configuration:

aws apigateway put-integration \
  --rest-api-id [API_ID] \
  --resource-id [RESOURCE_ID] \
  --http-method POST \
  --type AWS_PROXY \
  --integration-http-method POST \
  --uri arn:aws:apigateway:[region]:lambda:path/2015-03-31/functions/arn:aws:lambda:[region]:[account]:function:qdn-dashboard-lambda/invocations

This POST route integrates with the same Lambda function that handles other dashboard operations (job updates, photo listing, booking quotes). The API Gateway passes the full request body and headers to Lambda, allowing us to extract job ID, customer session token, and other context.

S3 Bucket and CloudFront Configuration

The upload destination is the qdn-uploads bucket, which is private (no bucket public access policy). Customer photos are stored with a prefix structure:

qdn-uploads/
  └── jobs/
      └── [job-id]/
          ├── photo-001.jpg
          ├── photo-002.jpg
          └── ...

The dashboard bucket (dashboard.quickdumpnow.com) remains private and is served only through CloudFront distribution. The separation ensures:

Dashboard assets (HTML, JS, CSS) are cached and served at edge locations
Upload requests hit the Lambda, which validates permissions before issuing credentials
Customer photos are stored separately and accessed only via presigned URLs, not through CloudFront

Frontend Integration

Updated /Users/cb/Documents/repos/sites/dashboard.quickdumpnow.com/index.html across four iterations to:

Add capture button handlers: Wired job drawer "capture" buttons to make a POST request to /upload-url instead of directly POSTing files
Display upload form: Once a presigned URL is received, display a file input dialog and form that submits directly to the signed S3 URL
Show photo thumbnails: Updated the photo list display to use presigned GET URLs returned by the list_photos` endpoint


Handle upload errors: Added error handling for expired presigned URLs and S3 upload failures



Deployment Process
The deployment followed this sequence:

Syntax validation: Ran Python syntax checks on the updated Lambda function
Lambda deployment: Deployed the updated function code to the qdn-dashboard-lambda function
CloudFront invalidation: Invalidated the /* path on the dashboard CloudFront distribution to clear cached HTML/JS
Smoke testing: Made test requests to https://dashboard.quickdumpnow.com/upload-url with valid job IDs and verified presigned URLs were returned with correct S3 bucket and key names
Staging verification: Deployed staging versions of dashboard files to the staging/ prefix on the dashboard bucket and verified presigned URLs worked end-to-end


Security Considerations

Presigned URL expiration: 15-minute window limits exposure if a URL is compromised
Job ID validation: Lambda verifies the requesting customer owns the job before issuing credentials
IAM role isolation: Lambda's execution role has minimal permissions; it can only read/write the qdn-uploads bucket
Private bucket policy: S3 bucket has no public access; all file access goes through presigned URLs or CloudFront
HTTPS enforcement: All requests to the dashboard and API go over TLS; presigned URLs are only transmitted in HTTPS responses


Key Decisions
Why presigned URLs instead of direct Lambda upload? Presigned URLs allow customers to upload directly to S3 without passing file bytes through Lambda, reducing latency and Lambda memory consumption for large files. Lambda only validates permissions and generates credentials; S3 handles the actual upload.

Why separate buckets for dashboard and uploads? Isolating assets from user-generated content allows independent scaling, caching, and lifecycle policies. Dashboard assets can be cached aggressively; uploads may have retention or compliance requirements.

Why time-limited credentials? A presigned URL is a bearer token.