Self-Hosted LiteLLM Gateway on Hetzner (Terraform)
An AI-cofounder / autonomous-startup-builder SaaS
Overview
A standalone Terraform/OpenTofu deployment that provisions a self-hosted LiteLLM model gateway for an AI-cofounder / autonomous-startup-builder SaaS on a single Hetzner Cloud VM, fronted by Caddy TLS and locked to Cloudflare IP ranges.
Why It Exists
A unified, self-hosted LLM gateway lets the platform route across model providers behind one endpoint with its own keys and accounting. This repo packages that gateway as infrastructure-as-code so it can be stood up (and reproduced) independently of the larger cluster.
What We Built
HCL (main.tf) declaring Hetzner, TLS, random and HTTP providers; it fetches Cloudflare’s published IPv4/IPv6 ranges and builds a firewall allowlist around them, then provisions a VM whose cloud-init.yaml installs Docker and brings up a Compose stack: LiteLLM (litellm-database image) with a PostgreSQL 16 backing store and a Caddy reverse proxy terminating TLS via Cloudflare origin certificates, serving the gateway on an internal inference domain. Configuration lives in /opt/litellm (config.yaml, .env, Caddyfile, certs, docker-compose.yml).
Technologies & Approach
Terraform/OpenTofu with cloud-init for first-boot configuration, a deliberately lightweight, single-VM pattern for a focused service. Pulling Cloudflare’s live IP ranges into the firewall and terminating TLS with Cloudflare origin certs means the gateway is only reachable through Cloudflare, mirroring the project’s egress-controlled security posture.
Outcome / Impact
Delivers a reproducible, self-hosted LiteLLM gateway as code, giving the platform provider-agnostic model routing behind a hardened, Cloudflare-fronted endpoint, an alternative, standalone form of the LLM gateway that also runs inside the main cluster.
Capabilities Demonstrated
- Infrastructure-as-code provisioning of an LLM gateway
- Single-VM Docker Compose deployment via cloud-init
- Caddy TLS termination with Cloudflare origin certificates
- Firewall allowlisting against live Cloudflare IP ranges
- Self-hosted LiteLLM + PostgreSQL stack