Function
Functions can be called to generate content. Functions are called by their name followed by parentheses ()
and may have arguments.
For instance, the range function returns a list containing an arithmetic progression of integers:
{% for i in range(0, 3) %}
{{ i }},
{% endfor %}
Each header below represents a built-in function.
block
The block
function is used to render the contents of a block more than once. It is not to be confused
with the block tag which is used to declare blocks.
The following example will render the contents of the "post" block twice; once where it was declared
and again using the block
function:
{% block "post" %} content {% endblock %}
{{ block("post") }}
The above example will output the following:
content
content
currentEachOutput
The currentEachOutput
function retrieves the current output of a sibling task when using an EachSequential task.
Look at the following flow:
tasks:
- id: each
type: io.kestra.plugin.core.flow.EachSequential
tasks:
- id: first
type: io.kestra.plugin.core.debug.Return
format: "{{task.id}}"
- id: second
type: io.kestra.plugin.core.debug.Return
format: "{{ outputs.first[taskrun.value].value }}"
value: ["value 1", "value 2", "value 3"]
To retrieve the output of the first
task from the second
task, you need to use the special taskrun.value
variable to lookup for the execution of the first
task that is on the same sequential execution as the second
task.
And when there are multiple levels of EachSequential, you must use the special parents
variable to lookup the correct execution. For example, outputs.first[parents[1].taskrun.value][parents[0].taskrun.value]
for a 3-level EachSequential.
The currentEachOutput
function will facilitate this by looking up the current output of the sibling task, so you don't need to use the special variables taskrun.value
and parents
.
The previous example can be rewritten as follow:
tasks:
- id: each
type: io.kestra.plugin.core.flow.EachSequential
tasks:
- id: first
type: io.kestra.plugin.core.debug.Return
format: "{{task.id}}"
- id: second
type: io.kestra.plugin.core.debug.Return
format: "{{ currentEachOutput(outputs.first).value }}"
value: ["value 1", "value 2", "value 3"]
And this works no matter the number of levels of EachSequential tasks used.
fromJson
The fromJson
function will convert any JSON string to an object allowing to access its properties using the dot notation:
{{ fromJson('[1, 2, 3]')[0] }}
{# results in: '1' #}
{{ fromJson('{"foo": [666, 1, 2]}').foo[0] }}
{# results in: '666' #}
{{ fromJson('{"bar": "\u0063\u0327"}').bar }}
{# results in: 'ç' #}
If you were using Kestra in a version prior to v0.18.0, this function used to be named json()
. We've renamed it to fromJson
for more clarity. The renaming has been implemented in a non-breaking way — using json()
will raise a warning in the UI but it will still work.
yaml
The yaml
function will convert any string to an object allowing to access its properties.
"{{ yaml('foo: [666, 1, 2]').foo[0] }}"
{# results in: '666' #}
{{ yaml(yaml_object) | jq(...) | yaml }}
{# prints the yaml_object as a yaml string #}
max
The max
function will return the largest of it's numerical arguments.
{{ max(user.age, 80) }}
min
The min
function will return the smallest of it's numerical arguments.
{{ min(user.age, 80) }}
now
The now
function will return the actual datetime. The arguments are the same as the date filter except the format is different.
{{ now() }}
{{ now(timeZone="Europe/Paris") }}
Arguments
- existingFormat
- timeZone
- locale
parent
The parent
function is used inside of a block to render the content that the parent template would
have rendered inside of the block had the current template not overridden it. It is similar to Java's super
keyword.
Let's assume you have a template, "parent.peb" that looks something like this:
{% block "content" %}
parent contents
{% endblock %}
And then you have another template, "child.peb" that extends "parent.peb":
{% extends "parent.peb" %}
{% block "content" %}
child contents
{{ parent() }}
{% endblock %}
The output will look something like the following:
parent contents
child contents
range
The range
function will return a list containing an arithmetic progression of numbers:
{% for i in range(0, 3) %}
{{ i }},
{% endfor %}
{# outputs 0, 1, 2, 3, #}
When step is given (as the third parameter), it specifies the increment (or decrement):
{% for i in range(0, 6, 2) %}
{{ i }},
{% endfor %}
{# outputs 0, 2, 4, 6, #}
Pebble built-in .. operator is just a shortcut for the range function with a step of 1+
{% for i in 0..3 %}
{{ i }},
{% endfor %}
{# outputs 0, 1, 2, 3, #}
printContext
The printContext
function is used to debug and print all variables defined.
{{ printContext() }}
The above example will output the following:
{"outputs": {...}, "execution": {...}, ...}
read
Read an internal storage file and return its content as a string. This function accepts one of the following:
- A path to a Namespace File e.g.
{{ read('myscript.py') }}
- An internal storage URI e.g.
{{ read(inputs.myfile) }}
or{{ read(outputs.extract.uri) }}
.
Reading namespace files is restricted to files in the same namespace as the flow using this function.
Reading internal storage files is restricted to the current execution. Specifically, those are files created by the current flow's execution or the parent flow execution (for flows triggered by a Subflow task or a ForEachItem task).
# Read a namespace file from the path `subdir/file.txt`
{{ read('subdir/file.txt') }}
# Read an internal storage file from the `uri` output of the `readFile` task
{{ read(outputs.readFile.uri) }}
# Read an internal storage file from an input named `file`
{{ read(inputs.file) }}
render
By default, kestra renders all expressions only once. This is safer from a security perspective, and it prevents unintended behavior when parsing JSON elements of a webhook payload that contained a templated string from other applications (such as GitHub Actions or dbt core). However, sometimes recursive rendering is desirable. For example, if you want to parse flow variables that contain Pebble expressions. This is where the render()
function comes in handy.
The render()
function is used to enable recursive rendering of Pebble expressions. It is available since the release 0.14.0.
The syntax for the render()
function is as follows:
{{ render(expression_string, recursive=true) }} # if false, render only once
The function takes two arguments:
- The string of an expression to be rendered e.g.
"{{ trigger.date ?? execution.startDate | date('yyyy-MM-dd') }}"
- A boolean flag that controls whether the rendering should be recursive or not:
- if
true
(default), the expression will be rendered recursively - if
false
, the expression will be rendered only once.
- if
Let's see some examples of how the render()
function works and where you need to use it.
Where the render()
function is not needed
Let's take the following flow as an example:
id: parse_input_and_trigger_expressions
namespace: company.team
inputs:
- id: myinput
type: STRING
defaults: hello
tasks:
- id: parse_date
type: io.kestra.plugin.core.debug.Return
format: "{{ trigger.date ?? execution.startDate | date('yyyy-MM-dd') }}"
- id: parse_input
type: io.kestra.plugin.core.debug.Return
format: "{{ inputs.myinput }}"
triggers:
- id: schedule
type: io.kestra.plugin.core.trigger.Schedule
cron: "* * * * *"
Since we don't use any nested expressions (like using trigger or input expressions in variables, or using variables in other variables), this flow will work just fine without having to use the render()
function.
Where the render()
function is needed
Let's modify our example so that now we store that long expression "{{ trigger.date ?? execution.startDate | date('yyyy-MM-dd') }}"
in a variable:
id: hello
namespace: company.team
variables:
trigger_var: "{{ trigger.date ?? execution.startDate | date('yyyy-MM-dd') }}"
tasks:
- id: parse_date
type: io.kestra.plugin.core.debug.Return
format: "{{ vars.trigger_var }}" # no render function means no recursive rendering for that trigger_var variable
triggers:
- id: schedule
type: io.kestra.plugin.core.trigger.Schedule
disabled: true
cron: "* * * * *"
Here, the task parse_date
will print the expression without recursively rendering it, so the printed output will be a string of that variable rather than a parsed date:
{{ trigger.date ?? execution.startDate | date('yyyy-MM-dd') }}
To fix that, simply wrap the vars.trigger_var
variable in the render()
function:
id: hello
namespace: company.team
variables:
trigger_var: "{{ trigger.date ?? execution.startDate | date('yyyy-MM-dd') }}"
tasks:
- id: parse_date
type: io.kestra.plugin.core.debug.Return
format: "{{ render(vars.trigger_var) }}" # this will print the recursively-rendered expression
triggers:
- id: schedule
type: io.kestra.plugin.core.trigger.Schedule
cron: "* * * * *"
Now you should see the date printed in the logs, e.g. 2024-01-16
.
Using expressions in namespace variables
Let's assume that you have configured a namespace variable myvar
. That variable uses a Pebble function to retrieve a secret e.g. {{ secret('MY_SECRET') }}
. By default, kestra will only render the expression without recursively rendering what's inside of the namespace variable:
id: namespace_variable
namespace: company.team
tasks:
- id: print_variable
type: io.kestra.plugin.core.debug.Return
format: "{{ namespace.myvar }}"
If you want to render the secret contained in that variable, you will need to wrap it in the render()
function:
id: namespaces_variable
namespace: company.team
tasks:
- id: print_variable
type: io.kestra.plugin.core.debug.Return
format: "{{ render(namespace.myvar) }}"
String concatenation
Let's look at another example that will demonstrate how the render()
function works with string concatenation.
id: pebble_string_concatenation
namespace: company.team
inputs:
- id: date
type: DATETIME
defaults: 2024-02-24T22:00:00.000Z
- id: user
type: STRING
defaults: Rick
variables:
day_of_week: "{{ trigger.date ?? inputs.date | date('EEEE') }}"
full_date: "{{ vars.day_of_week }}, the {{ trigger.date ?? inputs.date | date('yyyy-MM-dd') }}"
full_date_concat: "{{ vars.day_of_week ~ ', the ' ~ (trigger.date ?? inputs.date | date('yyyy-MM-dd')) }}"
greeting_concat: "{{'Hello, ' ~ inputs.user ~ ' on ' ~ vars.full_date }}"
greeting_brackets: "Hello, {{ inputs.user }} on {{ vars.full_date }}"
tasks:
- id: not-rendered
type: io.kestra.plugin.core.log.Log
message: |
Concat: {{ vars.greeting_concat }}
Brackets: {{ vars.greeting_brackets }}
Full date: {{ vars.full_date }}
Full date concat: {{ vars.full_date_concat }}
- id: rendered-recursively
type: io.kestra.plugin.core.log.Log
message: |
Concat: {{ render(vars.greeting_concat) }}
Brackets: {{ render(vars.greeting_brackets) }}
Full date: {{ render(vars.full_date) }}
Full date concat: {{ render(vars.full_date_concat) }}
- id: rendered-once
type: io.kestra.plugin.core.log.Log
message: |
Concat: {{ render(vars.greeting_concat, recursive=false) }}
Brackets: {{ render(vars.greeting_brackets, recursive=false) }}
Full date: {{ render(vars.full_date, recursive=false) }}
Full date concat: {{ render(vars.full_date_concat, recursive=false) }}
triggers:
- id: schedule
type: io.kestra.plugin.core.trigger.Schedule
cron: "* * * * *"
Note that:
- the
??
syntax within"{{ trigger.date ?? inputs.date | date('EEEE') }}"
is a null-coalescing operator that returns the first non-null value in the expression. For example, iftrigger.date
is null, the expression will returninputs.date
. - the
~
sign is a string concatenation operator that combines two strings into one.
When you run this flow, you should see the following output in the logs:
INFO Concat: {{'Hello, ' ~ inputs.user ~ ' on ' ~ vars.full_date }}
Brackets: Hello, {{ inputs.user }} on {{ vars.full_date }}
Full date: {{ vars.day_of_week }}, the {{ trigger.date ?? inputs.date | date('yyyy-MM-dd') }}
Full date concat: {{ vars.day_of_week ~ ', the ' ~ (trigger.date ?? inputs.date | date('yyyy-MM-dd')) }}
INFO Concat: Hello, Rick on Saturday, the 2024-02-24
Brackets: Hello, Rick on Saturday, the 2024-02-24
Full date: Saturday, the 2024-02-24
Full date concat: Saturday, the 2024-02-24
INFO Concat: Hello, Rick on {{ vars.day_of_week }}, the {{ trigger.date ?? inputs.date | date('yyyy-MM-dd') }}
Brackets: Hello, Rick on {{ vars.day_of_week }}, the {{ trigger.date ?? inputs.date | date('yyyy-MM-dd') }}
Full date: {{ trigger.date ?? inputs.date | date('EEEE') }}, the 2024-02-24
Full date concat: {{ trigger.date ?? inputs.date | date('EEEE') }}, the 2024-02-24
You may notice that both the vars.greeting_concat
and vars.greeting_brackets
lead to the same output, even though the first one uses the ~
sign for string concatenation within a single Pebble expression {{ }}
, and the second one uses one string with multiple {{ }}
expressions to concatenate the strings. Both are fully supported and you can decide which one to use based on your preference.
Webhook trigger: using render()
with the recursive=false
flag
Let's look at how the render()
function works with the webhook trigger.
Imagine you use a GitHub webhook as a trigger in order to automate deployments after a pull request is merged. Your GitHub webhook payload looks as follows:
{
"pull_request": {
"html_url": "https://github.com/kestra-io/kestra/pull/2834",
"body": "This PR replaces the ${{ env.GITHUB_TOKEN }} with a more secure ${{ secrets.GITHUB_TOKEN }}."
}
}
The pull request body contains templated variables ${{ env.MYENV }}
and ${{ secrets.GITHUB_TOKEN }}
, which are not meant to be rendered by Kestra, but by GitHub Actions. Processing this webhook payload with recursive rendering would result in an error, as those variables are not defined in the flow execution context.
In order to retrieve elements such as the pull_request.body
from that webhook's payload without recursively rendering its content, you can leverage the render()
function with the recursive=false
flag:
id: pebble_in_webhook
namespace: company.team
inputs:
- id: github_url
type: STRING
defaults: https://github.com/kestra-io/kestra/pull/2834
- id: body
type: JSON
defaults: |
{
"pull_request": {
"html_url": "https://github.com/kestra-io/kestra/pull/2834",
"body": "This PR replaces the ${{ env.GITHUB_TOKEN }} with a more secure ${{ secrets.GITHUB_TOKEN }}"
}
}
variables:
body: "{{ trigger.body.pull_request.body ?? trigger.body.issue.body ?? inputs.body }}"
github_url: "{{ trigger.body.pull_request.html_url ?? trigger.body.issue.html_url ?? inputs.github_url }}"
tasks:
- id: render_once
type: io.kestra.plugin.core.log.Log
message: "{{ render(vars.body, recursive=false) }}"
- id: not_recursive
type: io.kestra.plugin.core.log.Log
message: "{{ vars.body }}"
- id: recursive
type: io.kestra.plugin.core.log.Log
message: "{{ render(vars.body) }}"
allowFailure: true # this task will fail because it will try to render the webhook's payload
triggers:
- id: webhook
type: io.kestra.plugin.core.trigger.Webhook
key: test1234
Only the render_once
task is relevant for this use case, as it will render the pull request's body
without recursively rendering its content. The flow includes a recursive and non-recursive configuration for easy comparison.
- The
not_recursive
task will print the{{ trigger.body.pull_request.body ?? trigger.body.issue.body ?? inputs.body }}
expression as a string without rendering it. - The
recursive
task will fail, as it will try to render the webhook's payload containing templating that cannot be parsed by kestra.
Here is how you can call that flow via curl:
curl -i -X POST -H "Content-Type: application/json" \
-d '{ "pull_request": {"html_url": "https://github.com/kestra-io/kestra/pull/2834", "body": "This PR replaces the ${{ env.GITHUB_TOKEN }} with a more secure ${{ secrets.GITHUB_TOKEN }}"} }' \
http://localhost:8080/api/v1/executions/webhook/qa/pebble_in_webhook/test1234
On an instance with multi-tenancy enabled, make sure to also pass the tenant ID in the URL:
curl -i -X POST -H "Content-Type: application/json" \
-d '{ "pull_request": {"html_url": "https://github.com/kestra-io/kestra/pull/2834"}, "body": "This PR replaces the ${{ env.GITHUB_TOKEN }} with a more secure ${{ secrets.GITHUB_TOKEN }}"} }' \
http://localhost:8080/api/v1/your_tenant/executions/webhook/qa/pebble_in_webhook/test1234
You should see similar output in the logs:
INFO This PR replaces the ${{ env.GITHUB_TOKEN }} with a more secure ${{ secrets.GITHUB_TOKEN }}
INFO {{ trigger.body.pull_request.body ?? trigger.body.issue.body ?? inputs.body }}
ERROR io.pebbletemplates.pebble.error.PebbleException: Missing variable: 'env' on 'This PR replaces the ${{ env.GITHUB_TOKEN }} with a more secure ${{ secrets.GITHUB_TOKEN }}' at line 1 (?:?)
ERROR Missing variable: 'env' on 'This PR replaces the ${{ env.GITHUB_TOKEN }} with a more secure ${{ secrets.GITHUB_TOKEN }}' at line 1 (?:?)
renderOnce
The renderOnce()
function is used to enable one-time rendering of nested Pebble expressions. It is available since the release 0.16.0 and is equivalent to render(expression_string, recursive=false).
This function is syntactic sugar to reduce overhead brought by the recursive rendering default behaviour. It can be used to use vars
easily as they may contain themselves pebble expressions.
Basically, if vars.a={{ vars.b }}
, vars.b=42
then renderOnce(vars.a)=42
. Note that if vars.b={{ vars.c }}
, renderOnce(vars.a)={{ vars.c }}
.
The syntax for the renderOnce()
function is as follows:
{{ renderOnce(expression_string) }}
secret
The secret()
function is used to retrieve a secret from a secret backend based on the key provided as input to that function.
Here is an example flow that retrieves the Personal Access Token secret stored using the secret key GITHUB_ACCESS_TOKEN
:
id: secret
namespace: company.team
tasks:
- id: githubPAT
type: io.kestra.plugin.core.log.Log
message: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
The secret('key')
function will lookup up the configured secret manager backend for a secret value using the key GITHUB_ACCESS_TOKEN
. If the key is missing, an exception will be raised. The example flow shown above will look up the secret value by key and will log the output with the secret value.
The purpose of this example is to show how to use secrets in your flows. In practice, you should avoid logging sensitive information.
There are some differences between the secret management backend in the Open-Source and Enterprise editions. By default, Kestra provides a secret management backend based on environment variables. Each environment variable starting with SECRET_
will be available as a secret, and its value must be base64-encoded.
The above example will:
- retrieve the secret
GITHUB_ACCESS_TOKEN
from an environment variableSECRET_GITHUB_ACCESS_TOKEN
- base64-decode it at runtime.
Was this page helpful?