Loop icon
Script icon
UploadFiles icon
PushNamespaceFiles icon
SlackIncomingWebhook icon
Schedule icon

Snapshot Firestore Collections to Git Every Night

Snapshot Firestore collections to Git nightly with Kestra. Each document change becomes a one-line diff, and it runs on the free Firebase plan.

Categories
Data

Small, hand-edited Firestore collections such as settings, feature flags or price lists have no history: once someone changes a value in the console, the old one is gone. This flow commits those collections to a Git repository every night, one JSON line per document, so each change shows up as a diff.

How it works

  1. A Schedule trigger starts the flow every night at 02:00.
  2. A Loop runs once per collection in the collections input.
  3. A Python script reads every document with the official Firestore client, ordered by ID, and writes {"id": ..., "data": ...} with sorted keys, so an unchanged collection gives an identical file.
  4. UploadFiles stages each file as a namespace file, and PushNamespaceFiles commits them to firestore/firestore-snapshots/ in the repository. No commit is made when nothing changed.
  5. If any step fails, the errors branch posts the execution ID to Slack.

What you get

  • A dated history of each collection where a diff points at the exact document that changed.
  • No empty commits on quiet nights.
  • A Slack alert when a run fails.
  • Works on the free Spark plan with a read-only service account.

Who it's for

  • Teams that keep app settings, feature flags or reference data in Firestore.

Why orchestrate this with Kestra

  • Secrets stay masked in logs and never reach the repository.
  • Scheduling, failure alerting and execution history come with the flow instead of being built around a cron script.

Prerequisites

  • A Firestore database in Native mode. Billing is not required.
  • A service account with roles/datastore.viewer and a JSON key.
  • A repository used only for these snapshots, and a token that can push to it.
  • A worker that can run Docker containers, for the python:3.12-slim image.
  • Small collections only: every document is read on each run, and the free plan allows 50,000 reads per day.

Secrets

  • GCP_SERVICE_ACCOUNT: service account JSON key used to read Firestore.
  • GITHUB_USERNAME: Git username that owns the token.
  • GITHUB_TOKEN: token used to push commits, for example a GitHub fine-grained token with Contents: Read and write on the snapshot repository.
  • SLACK_WEBHOOK: Slack incoming webhook URL for failure alerts.

Quick start

  1. Add the four secrets to your namespace.
  2. Execute the flow with your project_id, repo_url and collections, then check firestore/firestore-snapshots/ in the repository.
  3. Change a document in the Firebase console, execute again, and open the new commit.
  4. Set project_id and repo_url in the trigger inputs to run it nightly.

How to extend

  • Run it hourly for collections that change often.
  • Post the commit diff to Slack so reviewers see changes without opening Git.
  • Push to a separate branch and open a pull request when a protected collection changes.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.