Version-Control Your Namespace Files with Git

Push files in your namespace to a Git Repository with the PushNamespaceFiles Task.

How it works

The PushNamespaceFiles task is a powerful integration that allows you to push your namespace files to Git from the UI while still managing this process entirely in code! Kestra unifies the development experience between the UI and code so you can combine the best of both worlds without sacrificing the benefits of version control.

The process is simple: you can build your flows and files in a development namespace using all productivity features of the Kestra UI (such as the built-in code editor, autocompletion, syntax validation, documentation, blueprint examples, live-updating topology view, output previews, replays, execution and revision history) and then push them to Git after you have tested and validated them.

The task pushes one or more files from a given namespace (and optionally also child namespaces) to any Git-based Version Control System.

Additionally, the dryRun property will help you see what files will be added, modified, or deleted without overwriting the files on Git yet.

The following examples cover common patterns for the PushNamespaceFiles task.

Before you begin

Before you start using the PushNamespaceFiles task, ensure the following prerequisites are in place:

1. A Git repository where you want to push your files.
2. A Personal Access Token (PAT) for Git authentication.
3. A running Kestra instance in a version 0.17.0 or later with the PAT stored as a secret within the Kestra instance.

Using the dryRun property

Start by creating a single example.py file in the dev namespace and pushing it to a Git repository. Initially set the dryRun property to true to validate changes before committing to Git. You’ll need a flow already in the dev namespace to create a new file.

print("Hello, World")

Here is a system flow that will push the example.py file to a Git repository:

id: push_to_git
namespace: system

tasks:
  - id: commit_and_push
    type: io.kestra.plugin.git.PushNamespaceFiles
    username: git_username
    password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
    url: https://github.com/git_username/scripts
    branch: dev
    namespace: company.team
    files:
      - "example.py"
    gitDirectory: _files
    commitMessage: "add namespace files"
    dryRun: true

Given that the dryRun property is set to true, the task will only output modifications without pushing any files to Git yet:

Pushing a single file to Git

Set the dryRun property to false and push the example.py file to Git:

id: push_to_git
namespace: system

tasks:
  - id: commit_and_push
    type: io.kestra.plugin.git.PushNamespaceFiles
    ...
    dryRun: false

You should see the following log message:

And here is what you should see in the Outputs tab:

When you click on the commit URL from the logs or from the Outputs tab, you’ll be redirected to the commit page on GitHub:

Now, you can create a pull request and merge the changes to the main branch.

Pushing all files from a single namespace to Git

To push all files from a given namespace to Git, create two more files in the dev namespace:

example.sh file:

echo "Hello, World"

example.js file:

console.log("Hello, World")

Adjust the system flow to push all files from the dev namespace to the dev branch:

id: push_to_git
namespace: system

tasks:
  - id: commit_and_push
    type: io.kestra.plugin.git.PushNamespaceFiles
    username: git_username
    password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
    url: https://github.com/git_username/scripts
    branch: dev
    namespace: company.team
    gitDirectory: _files
    commitMessage: "push all namespace files and create a PR"
    dryRun: true

Again, we can set the dryRun property to true to see what files will be added, modified, or deleted based on the Git version without overwriting the files in Git yet:

Now if you change the dryRun property to false and run the system flow again, you should see all three files being pushed to the _files directory on the develop branch with the exact commit messages we have specified in the commitMessage property:

Extra notes

• Git does not guarantee the order of push operations to a remote repository, which can lead to potential conflicts when multiple users or flows attempt to push changes simultaneously. To minimize the risk of data loss and merge conflicts, it is strongly recommended to use sequential workflows or push changes to separate branches.