Serverless Subscription Scheduler & Task Dispatcher

Overview

A Google Cloud Function scheduler that reads subscription configurations from Firestore and fans them out into Cloud Tasks for data collection across multiple sources. It acts as the serverless trigger layer that drives the platform’s ingestion and maintenance work.

The Challenge

A subscription-driven media platform must periodically wake up, decide which subscriptions are due, and dispatch the right collection and maintenance jobs, reliably, at scale, and without standing infrastructure. Doing this as a serverless function keeps cost and operational overhead low while integrating cleanly with the rest of the Google Cloud stack.

What We Built

A Functions Framework (Flask-based) Cloud Function that processes Firestore subscriptions and creates Cloud Tasks per source. The codebase is organized into a task layer (subscriptions, email-delivery transactions, scheduler, synchronizer, repository, and maintenance tasks) with a shared base-task abstraction, structured logging, and helpers. Configuration is environment-aware (app.toml, app.staging.toml, app.production.toml) and infrastructure is codified in Terraform (main.tf, variables, staging tfvars). Database schema changes are managed with Flyway migrations, and local development runs against a PostgreSQL container and Firestore emulator via Docker Compose. A dedicated test suite (unit and integration, with a separate test compose file) and a deploy script round out the delivery story.

Technologies & Approach

Python on Google Cloud Functions using functions-framework, with google-cloud-firestore, google-cloud-tasks, and google-cloud-pubsub clients. Terraform provisions the cloud resources; Flyway governs PostgreSQL schema migrations; uv/pyproject manages dependencies. The Firestore-as-config, Cloud-Tasks-as-dispatch pattern decouples scheduling decisions from the workers that execute them.

Outcome / Impact

Provides the platform’s lightweight, serverless heartbeat, turning subscription configuration into concrete, distributed collection and maintenance jobs, with reproducible local development and infrastructure-as-code deployment.

Capabilities Demonstrated

• Serverless scheduling and task fan-out on Google Cloud Functions and Cloud Tasks
• Firestore-driven configuration with Pub/Sub and PostgreSQL integration
• Infrastructure as code with Terraform across staging and production
• Database migration management with Flyway
• Emulator-based local development and a unit/integration test suite