feat(terraform): add AWS ECS + GCP Cloud Run stacks for componentized gateway#27673
Conversation
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
PR overviewTerraform stacks for AWS ECS and GCP Cloud RunThis PR adds componentized LiteLLM infrastructure for AWS and GCP with public load balancers, private database/cache resources, managed secrets, and per-component routing. I reviewed the exposed entry points, TLS opt-in gates, service identities, secret injection paths, storage permissions, and route split against the existing gateway/backend allowlists; I did not find an attacker-reachable security regression. Security review
Risk: 2/10 |
Greptile SummaryThis PR adds two self-contained Terraform root modules (
Confidence Score: 5/5Safe to merge — all blocking concerns from prior review rounds are addressed, and the remaining items are defensive hardening suggestions that do not affect the correctness of a default deployment. The two Terraform stacks are well-structured: TLS is enforced by lifecycle preconditions, storage destruction requires explicit opt-in, IAM follows least-privilege with a dedicated no-permission UI service account, secrets are never written as plaintext env vars, and the bootstrap/migration ordering correctly gates service startup. terraform/litellm/aws/ecs.tf (backend non-config port override) and terraform/litellm/aws/variables.tf (db_username/db_name identifier validation)
|
| Filename | Overview |
|---|---|
| terraform/litellm/aws/ecs.tf | ECS task definitions and services for gateway, backend, and UI; non-config backend case omits explicit --port 4001 unlike gateway and the GCP backend, relying silently on the image default |
| terraform/litellm/aws/iam.tf | IAM roles with improved ARN normalization via slice/split (replaces previous regex); separate UI task role with no data-plane permissions |
| terraform/litellm/aws/bootstrap.tf | ECS-based DB bootstrap and migration trigger; db_username and db_name are interpolated into SQL without identifier validation, unlike tenant/env which have regex guards |
| terraform/litellm/aws/alb.tf | ALB with lifecycle precondition enforcing TLS (acm_certificate_arn) or explicit allow_plaintext_alb opt-out; addresses previous HTTP-only concern |
| terraform/litellm/aws/rds.tf | Aurora cluster with IAM auth, skip_final_snapshot defaults to false via variable; writer/reader instances with correct promotion tiers |
| terraform/litellm/aws/redis.tf | Changed to aws_elasticache_replication_group; TLS and HA enabled when redis_num_replicas >= 1 |
| terraform/litellm/aws/variables.tf | Variables with good documentation; tenant/env have regex validation but db_username and db_name (interpolated into SQL identifiers) do not |
| terraform/litellm/gcp/cloudrun.tf | Cloud Run v2 services with shell wrapper startup, explicit port args for both gateway and backend, allUsers invoker with INGRESS_TRAFFIC_INTERNAL_LOAD_BALANCER (correct external LB pattern) |
| terraform/litellm/gcp/iam.tf | Separate runtime and ui_runtime service accounts; extras IAM binding uses toset(values(merge(...))) to correctly deduplicate secrets |
| terraform/litellm/gcp/load_balancer.tf | External HTTPS LB with lifecycle precondition enforcing TLS or allow_plaintext_lb opt-out; URL map correctly routes gateway/UI/backend prefixes |
| terraform/litellm/gcp/cloudsql.tf | Cloud SQL with deletion_protection = var.cloudsql_deletion_protection (defaults true); pitr and backups enabled on writer |
| terraform/litellm/aws/s3.tf | S3 bucket with force_destroy wired to var.s3_force_destroy (defaults false); SSE-AES256, versioning, and public access block all enabled |
Reviews (5): Last reviewed commit: "feat: add Terraform stacks for deploying..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
yassin-berriai
force-pushed
the
litellm_fix/tf-aws-gcp
branch
4 times, most recently
from
106505d to
10fcb61
Compare
|
@greptileai please re-review — I just pushed 59f4f3a which addresses all 7 prior comments (HTTPS gating on AWS+GCP, Aurora final snapshot, Cloud SQL deletion protection, ECS secret ARN regex, ElastiCache replication group with TLS, missing var.db_edition). |
yassin-berriai
force-pushed
the
litellm_fix/tf-aws-gcp
branch
from
59f4f3a to
7e426fe
Compare
yassin-berriai
force-pushed
the
litellm_fix/tf-aws-gcp
branch
4 times, most recently
from
27a86cd to
106762a
Compare
yassin-berriai
force-pushed
the
litellm_fix/tf-aws-gcp
branch
from
106762a to
5fc1dfc
Compare
yassin-berriai
force-pushed
the
litellm_fix/tf-aws-gcp
branch
2 times, most recently
from
61c9a0d to
3c7e692
Compare
- Add AWS ECS Fargate stack with Aurora Postgres (IAM auth), ElastiCache Redis, S3, ALB with path-based routing to gateway/backend/ui components, Application Auto Scaling, and automated DB bootstrap + prisma migration via local-exec provisioners - Add GCP Cloud Run stack with Cloud SQL Postgres (password auth), Memorystore Redis, GCS, external HTTPS load balancer with serverless NEGs and URL map routing, and automated prisma migration via Cloud Run Job - Both stacks support typed proxy_config input mirroring the helm chart's gateway.config.proxy_config, per-component extra env vars, and Secret Manager references for provider API keys - Gateway/backend services depend on terraform_data.migration so they never start before the schema is in place, eliminating crash-loop windows on first apply - AWS stack uses IAM database authentication with a one-shot Fargate bootstrap task that creates and grants the rds_iam role to the application user; GCP stack uses password auth assembled at container startup to avoid Cloud SQL Auth Proxy sidecar complexity - Add .gitignore rules for Terraform state files, plan files, tfvars inputs, provider binaries, and crash logs while explicitly keeping .terraform.lock.hcl for provider version pinning - Include terraform.tfvars.example files, provider lock files, and comprehensive README documentation covering architecture, TLS setup, image pull strategies, and quick-start instructions for both stacks
yassin-berriai
force-pushed
the
litellm_fix/tf-aws-gcp
branch
from
3c7e692 to
7b80b86
Compare
…iAI#27673) - Add AWS ECS Fargate stack with Aurora Postgres (IAM auth), ElastiCache Redis, S3, ALB with path-based routing to gateway/backend/ui components, Application Auto Scaling, and automated DB bootstrap + prisma migration via local-exec provisioners - Add GCP Cloud Run stack with Cloud SQL Postgres (password auth), Memorystore Redis, GCS, external HTTPS load balancer with serverless NEGs and URL map routing, and automated prisma migration via Cloud Run Job - Both stacks support typed proxy_config input mirroring the helm chart's gateway.config.proxy_config, per-component extra env vars, and Secret Manager references for provider API keys - Gateway/backend services depend on terraform_data.migration so they never start before the schema is in place, eliminating crash-loop windows on first apply - AWS stack uses IAM database authentication with a one-shot Fargate bootstrap task that creates and grants the rds_iam role to the application user; GCP stack uses password auth assembled at container startup to avoid Cloud SQL Auth Proxy sidecar complexity - Add .gitignore rules for Terraform state files, plan files, tfvars inputs, provider binaries, and crash logs while explicitly keeping .terraform.lock.hcl for provider version pinning - Include terraform.tfvars.example files, provider lock files, and comprehensive README documentation covering architecture, TLS setup, image pull strategies, and quick-start instructions for both stacks Co-authored-by: Yassin Kortam <[email protected]>
4 participants
Adds two self-contained Terraform root modules under
terraform/that deploy the componentized LiteLLM proxy (gateway + backend + UI) end-to-end:terraform/aws: VPC, Aurora Postgres cluster (writer + reader, IAM auth), ElastiCache, S3, Secrets Manager, ECS Fargate cluster with three services, ALB with path-routing rules mirroring the helm ingress, one-off prisma_migration ECS task.terraform/gcp: VPC + serverless VPC connector, Cloud SQL Postgres (primary + read replica), Memorystore, GCS, Secret Manager, three Cloud Run v2 services, Cloud Run Job for migrations, external HTTPS LB with serverless NEGs + URL map.Both stacks expose a typed
proxy_configinput that mirrors the helm chart'sgateway.config.proxy_config(YAML-encoded, base64-passed, decoded at container start) plus per-component*_extra_envand*_extra_secretsso callers can layer provider API keys from Secrets Manager / Secret Manager and reference them from the YAML viaos.environ/<NAME>.Relevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
AWS
GCP
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes