Embeddable AI Voice-Assistant Platform (multi-service)

Overview

A platform for creating embeddable AI assistants that website owners can drop into their pages. It handles assistant configuration, customer authentication, session tracking and prepaid-credit billing, while a separate browser-side widget delivers a full voice agent (speech-to-text, text-to-speech, wake-word and tool calling).

Why It Exists

Bringing a capable, brandable AI assistant onto a third-party website usually means stitching together auth, billing, conversation history and a voice runtime by hand. This project explored packaging all of that into a multi-tenant platform plus a single embeddable script tag.

What We Built

A multi-service monorepo: a portal (React 19 + Vite + Convex + Clerk, with Recharts dashboards and an Anthropic SDK integration) for customers to manage assistants, skills and credit balances; a widget published as a @browser-agent UMD/ESM library with discrete modules for STT, TTS, wake-word, audio, voice, tools and file handling; a gatekeeper Cloudflare Worker; a convex reactive backend; and a sandbox for testing. The product design covers anonymous device-ID end-users versus HMAC-verified identified users, domain-restricted publishable keys plus secret keys for signing, and per-customer prepaid USD credit balances.

Technologies & Approach

Convex provides a reactive database and serverless functions; Clerk handles customer auth; Cloudflare Workers host the edge gatekeeper; and the widget is a framework-agnostic bundle so it can embed anywhere. The voice stack (wake-word, STT, TTS, tool execution) lives entirely client-side in the widget for low latency.

Outcome / Impact

A working architecture for a credit-billed, embeddable AI voice-assistant SaaS, demonstrating end-to-end thinking across frontend, edge backend, reactive data, multi-tenant auth/billing and an in-browser voice agent.

Capabilities Demonstrated

• Distributable embeddable widget SDK (UMD + ESM) for third-party sites
• In-browser voice agent: wake-word, STT, TTS and LLM tool-calling
• Multi-tenant SaaS with Clerk auth and prepaid-credit billing
• HMAC-secured handoff of end-user identity from customer backends
• Reactive serverless backend on Convex with edge gatekeeping on Cloudflare