Version Control with Git

Set up version control with Git to store your flows and namespace files.

Kestra supports version control with Git. You can use one or more Git repositories to store your flows, namespace files, apps, tests, and dashboards, enabling you to track changes over time through Git commit history.

There are multiple ways to use Git with Kestra:

SyncFlows task implements GitOps, using Git as a single source of truth for your flows.
SyncNamespaceFiles task implements GitOps, using Git as a single source of truth for namespace files.
PushFlows task commits and pushes flow edits from the UI to Git. This pattern suits teams that want to use the built-in editor while still keeping code in Git.
PushNamespaceFiles task commits and pushes namespace file edits from the UI to Git. This pattern is useful for teams that prefer the built-in editor but still want files tracked in Git.
Clone task clones a repository directly into a flow, making scripts available for execution.
TenantSync task synchronizes all namespaces in a tenant, including flows, files, apps, tests, and dashboards.
NamespaceSync task synchronizes objects within a single namespace with your Git repository.
CI/CD pattern is useful if you want to manage the CI/CD process yourself e.g. via GitHub Actions or Terraform, and keep Git as a single source of truth for your code.

The image below shows how to choose the right pattern based on your needs:

git

Let's dive into each of these patterns and when to use them.

Git SyncFlows and SyncNamespaceFiles

The Git SyncFlows pattern implements GitOps and uses Git as a single source of truth. It allows you to store your flows in Git and use a system flow that automatically syncs changes from Git to Kestra. You can also sync namespace files using the Git SyncNamespaceFiles pattern in the same way.

Here's how that works:

You store your flows and namespace files in Git
You create a system flow that runs on a schedule and syncs changes from Git to Kestra
When you want to make a change to a flow or a namespace file, you modify the file in Git
The system flow syncs changes from Git to Kestra so that even if you make changes to any flows or namespace files from the UI, the changes are overwritten by the changes from Git.

This pattern is useful if you want to use Git as a single source of truth and avoid making changes to flows and namespace files from the UI. Using this pattern, you don't need to manage any CI/CD pipelines.

If your team follows the GitOps methodology, or you're coming from a Kubernetes background, this pattern is for you.

Here is an example system flow that you can use to declaratively sync changes from Git to Kestra:

yaml

id: sync_from_git
namespace: system

tasks:
  - id: git
    type: io.kestra.plugin.git.SyncFlows
    url: https://github.com/kestra/scripts
    branch: main
    username: git_username
    password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
    targetNamespace: git
    includeChildNamespaces: true # optional; by default, it's set to false to allow explicit definition
    gitDirectory: your_git_dir

triggers:
  - id: schedule
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "*/1 * * * *" # every minute

You can either commit this flow to Git or add it from the built-in editor in the Kestra UI. This flow won't be overwritten by the Git reconciliation process.

You can also sync namespace files with the example below:

yaml

id: sync_from_git
namespace: system


tasks:
  - id: git
    type: io.kestra.plugin.git.SyncNamespaceFiles
    namespace: prod
    gitDirectory: _files # optional; set to _files by default
    url: https://github.com/kestra-io/flows
    branch: main
    username: git_username
    password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"

This flow can also be triggered anytime you push changes to Git via a GitHub webhook:

yaml

id: sync_from_git
namespace: system

tasks:
  - id: git
    type: io.kestra.plugin.git.SyncFlows
    url: https://github.com/kestra/scripts
    branch: main
    targetNamespace: git
    username: git_username
    password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"

triggers:
  - id: github_webhook
    type: io.kestra.plugin.core.trigger.Webhook
    key: "{{ secret('WEBHOOK_KEY') }}"

Note that the webhook key is used to authenticate webhook requests and prevent unauthorized access to your Kestra instance. For the above flow, you would paste the following URL in your GitHub repository settings in the Webhooks section:

bash

http://your_kestra_host:8080/api/v1/<your_tenant>/executions/webhook/prod/sync_from_git/your_secret_key

github_webhook

Following the pattern:

bash

http://<host>/api/v1/<tenant>/executions/webhook/<namespace>/<flow>/<webhook_key>

CI/CD

The CI/CD pattern allows you to use Git as a single source of truth and push code changes to Kestra anytime you merge a pull request. However, in contrast to the Git Sync pattern, you need to manage the CI/CD process yourself through automation such as GitHub Actions or Terraform. Check the CI/CD documentation for more details on how to set up CI/CD for Kestra flows and Namespace Files.

Git PushFlows and PushNamespaceFiles

The Git PushFlows pattern allows you to edit your flows from the UI, and regularly push changes to Git. It's particularly helpful if you want to use the built-in Editor in the UI and have your code change history managed via Git. You can also push namespace files using the Git PushNamespaceFiles pattern in the same way.

Here is an example flow you can use to push changes from Kestra to Git:

yaml

id: push_to_git
namespace: system

tasks:
  - id: commit_and_push
    type: io.kestra.plugin.git.PushFlows
    url: https://github.com/kestra-io/scripts
    sourceNamespace: dev
    targetNamespace: pod
    flows: "*"
    branch: kestra
    username: github_username
    password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
    commitMessage: add namespace files changes

triggers:
  - id: schedule
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "* */1 * * *" # every hour

Here is an example you can use to push namespace files from Kestra to Git:

yaml

id: push_to_git
namespace: system

tasks:
  - id: commit_and_push
    type: io.kestra.plugin.git.PushNamespaceFiles
    namespace: dev
    files: "*"
    gitDirectory: _files
    url: https://github.com/kestra-io/scripts # required string
    username: git_username
    password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
    branch: dev
    commitMessage: "add namespace files"


triggers:
  - id: schedule_push_to_git
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "*/15 * * * *"

You can use this pattern to push changes to a feature branch and then create a pull request for review.

Git Clone

The Git Clone pattern allows you to clone a Git repository at runtime. This pattern can be used to orchestrate code maintained in a different code repository (potentially managed by a different team) in the following scenarios:

dbt projects orchestrated via dbt CLI task
infrastructure deployments orchestrated via Terraform CLI or Ansible CLI
Docker builds orchestrated via Docker Build task.

Git TenantSync and NamespaceSync

Both Git TenantSync and Git NamespaceSync give you full control over synchronizing Kestra objects with your Git repository.

TenantSync – synchronizes all namespaces in a tenant, including flows, files, apps, tests, and dashboards.
- Requires kestraUrl and auth properties for validation of tenant-wide RBAC.
- Useful for easily syncing all objects in a tenant, allowing you to back up the complete tenant state to Git and move between environments via a pull request to another branch.
NamespaceSync – synchronizes objects within a single namespace with your Git repository.
- Requires the namespace property but does not require kestraUrl and auth - it relies on namespace-level RBAC and can be run by any user with sufficient permissions.
- Ideal for teams that sync one namespace per Git repository, allowing delegation of sync responsibilities to namespace owners.

Both plugins support:

sourceOfTruth (either GIT or KESTRA) to define the update strategy.
whenMissingInSource with options DELETE, KEEP, or FAIL to control how missing objects should be handled.
An opinionated folder structure for flows, apps, dashboards, tests, and files with one folder per namespace.
protectedNamespaces to ensure your Kestra objects from critical namespaces (such as system) are not accidentally deleted when sourceOfTruth is GIT.
Validation rules requiring explicit Git branch and optional gitDirectory.
Options like dryRun and onInvalidSyntax for safe rollouts and error handling.

Example usage of the TenantSync task:

yaml

id: tenant_git_sync
namespace: system

tasks:
  - id: tenant
    type: io.kestra.plugin.git.TenantSync
    sourceOfTruth: KESTRA
    whenMissingInSource: DELETE
    url: https://github.com/org/repo
    branch: main
    protectedNamespaces:
      - system
    kestraUrl: http://localhost:8080
    auth:
      username: [email protected]
      password: "{{ secret('KESTRA_PASSWORD') }}"

Example usage of the NamespaceSync task:

yaml

id: namespace_git_sync
namespace: system

tasks:
  - id: namespace
    type: io.kestra.plugin.git.NamespaceSync
    namespace: company.team
    sourceOfTruth: GIT
    whenMissingInSource: KEEP
    url: https://github.com/org/repo
    branch: main
    protectedNamespaces:
      - system

Was this page helpful?

DocsVersion Control & CI/CD

Version Control CicdCI/CD Pipeline

​Version ​Control with ​Git

Git SyncFlows and SyncNamespaceFiles

CI/CD

Git PushFlows and PushNamespaceFiles

Git Clone

Git TenantSync and NamespaceSync

Version Control with Git