DocsDeveloper GuideVersion Control

Version Control with Git​Version ​Control with ​Git

Was this page helpful?

Kestra supports version control with Git. You can use one or more Git repositories to store your Flows and Namespace Files, and track changes to them over time via Git commit history.

There are multiple ways to use Git with Kestra:

  • The git.Sync pattern allows you to implement GitOps and use Git as a single source of truth
  • The git.Push pattern allows you to edit your flows and Namespace Files from the UI and regularly commit and push changes to Git; this pattern is useful if you want to use the built-in Editor in the UI and still have your code in Git
  • The CI/CD pattern is useful if you want to manage the CI/CD process yourself e.g. via GitHub Actions or Terraform, and keep Git as a single source of truth for your code.

The image below shows how to choose the right pattern based on your needs:

git

Let's dive into each of these patterns, and when to use them.

Git Sync

The Git Sync pattern implements GitOps and uses Git as a single source of truth. It allows you to store your flows and namepsace files in Git and use a system flow that automatically syncs changes from Git to Kestra.

Here's how that works:

  • You store your flows and Namespace Files in Git
  • You create a system flow that runs on a schedule and syncs changes from Git to Kestra
  • When you want to make a change to a flow or a namespace file, you modify the file in Git
  • The system flow syncs changes from Git to Kestra so that even if you make changes to any flows or Namespace Files from the UI, the changes are overwritten by the changes from Git.

This pattern is useful if you want to use Git as a single source of truth and avoid making changes to flows and Namespace Files from the UI. Using this pattern, you don't need to manage any CI/CD pipelines.

If your team follows the GitOps methodology, or you're coming from a Kubernetes background, this pattern is for you.

Here is an example system flow that you can use to declaratively sync changes from Git to Kestra:

yaml
id: sync_from_git
namespace: prod

tasks:
  - id: git
    type: io.kestra.plugin.git.Sync
    url: https://github.com/anna-geller/kestra-ci-cd
    branch: main
    username: anna-geller
    password: "{{ secret('GITHUB_PAT') }}" # needed for private repositories
    gitDirectory: your_git_dir # optional, otherwise all files
    namespaceFilesDirectory: scripts # if not set, the root directory
    dryRun: true  # if true, you'll see what files will be added, modified
    # or deleted based on the Git version without overwriting the files yet

triggers:
  - id: schedule
    type: io.kestra.core.models.triggers.types.Schedule
    cron: "*/1 * * * *" # every minute

You can choose to commit this flow to Git or add it from the built-in Editor in Kestra UI — this flow won't be overwritten by the Git reconciliation process.

This flow can also be triggered anytime you push changes to Git via a GitHub webhook:

yaml
id: sync_from_git
namespace: prod

tasks:
  - id: git
    type: io.kestra.plugin.git.Sync
    url: https://github.com/anna-geller/kestra-ci-cd
    branch: main
    username: anna-geller
    password: "{{ secret('GITHUB_PAT') }}"

triggers:
  - id: github_webhook
    type: io.kestra.core.models.triggers.types.Webhook
    key: "{{ secret('WEBHOOK_KEY') }}"

Note that the webhook key is used to authenticate webhook requests and prevent unauthorized access to your Kestra instance. For the above flow, you would paste the following URL in your GitHub repository settings in the Webhooks section:

bash
https://us.kestra.cloud/api/v1/your_tenant/executions/webhook/prod/sync_from_git/your_secret_key

github_webhook

Following the pattern:

bash
https://<host>/api/v1/<tenant>/executions/webhook/<namespace>/<flow>/<webhook_key>

CI/CD

The CI/CD pattern allows you to use Git as a single source of truth and push code changes to Kestra anytime you merge a pull request. However, in contrast to the Git Sync pattern, you need to manage the CI/CD process yourself e.g. via GitHub Actions or Terraform. Check the CI/CD documentation for more details on how to set up CI/CD for Kestra flows and Namespace Files.

Git Push

The Git Push pattern allows you to edit your flows and Namespace Files from the UI, and regularly push changes to Git. It's particularly helpful if you want to use the built-in Editor in the UI and have your code change history managed via Git.

Here is example flow that you can use to push code changes from Kestra to Git:

yaml
id: push_to_git
namespace: prod

tasks:
  - id: commit_and_push
    type: io.kestra.plugin.git.Push
    url: https://github.com/kestra-io/scripts
    branch: kestra
    username: github_username
    password: github_personal_access_token
    namespaceFiles:
      enabled: true
    flows:
      enabled: true
    commitMessage: add namespace files changes

triggers:
  - id: schedule
    type: io.kestra.core.models.triggers.types.Schedule
    cron: "* */1 * * *" # every hour

The flows and namespaceFiles properties allow you to declaratively configure which files to push to Git:

  • If flows are enabled, all flows from the current namespace will be pushed to the specified Git branch.
  • If namespaceFiles are enabled, all namespace files will also be pushed to the specified Git branch.

Both of these properties have nested child properties to declare which files and directories should be included, whether to also push flows from child namespaces, and more. Check the Git Push documentation for more details.

You can use that pattern to push changes to a feature branch and create a pull request for review. Once the pull request is approved, you can merge it to the main branch.

Git Clone

The Git Clone pattern allows you to clone a Git repository at runtime. This pattern can be used to orchestrate code maintained in a different code repository (potentially managed by a different team) in the following scenarios: