LLM Gateway & Prepaid API-Credit Marketplace
Overview
An internally-built platform that puts a single, billable API surface in front of many large-language-model providers. Users top up credit, receive scoped API keys, and call a unified OpenAI-compatible endpoint while the platform meters spend, enforces budgets, and surfaces usage analytics.
The Challenge
Teams that want to consume multiple LLM providers face fragmented billing, inconsistent APIs, and no shared spend controls. The goal was a self-serve gateway that abstracts providers behind one endpoint, handles prepaid credit and metering, and exposes a discovery/dashboard experience, without standing up heavyweight infrastructure.
What We Built
A Next.js App Router frontend (dashboard, discovery, pricing, checkout, stats, blog and legal pages) backed by a Convex serverless backend. The Convex layer is organized into clear domains: litellm and models/optimizedModels for provider catalog and routing, apiKeys/apiTokens for credential issuance, budget/spend/evalCache/evalEngine for metering and evaluation, purchases/crons for billing cycles, plus auth, onboarding, notify and github integrations. Payments run through Dodo Payments and Stripe with signed webhooks, and the whole app deploys to Cloudflare via OpenNext.
Technologies & Approach
TypeScript end-to-end. Convex provides the reactive data store, scheduled crons and HTTP actions; LiteLLM normalizes provider differences; the AI SDK’s OpenAI-compatible provider drives model calls. The UI uses shadcn/ui on Tailwind with Recharts for usage charts. Deployment targets Cloudflare Workers through the OpenNext adapter and Wrangler.
Outcome / Impact
Demonstrates an end-to-end, monetizable AI infrastructure product: provider abstraction, credential management, prepaid budgeting, real-time usage analytics and payment integration assembled on a serverless, edge-deployed stack.
Capabilities Demonstrated
- Unified, OpenAI-compatible gateway over multiple LLM providers
- Prepaid credit, budget enforcement and per-key spend metering
- API key and token lifecycle management
- Payment processing with signed webhook reconciliation
- Edge deployment of a full-stack Next.js app on Cloudflare