Tasks are the steps within a flow.

Tasks are discrete actions within a flow, capable of taking inputs and variables from the flow, and producing outputs for downstream consumption by end users and other tasks.

Flowable Tasks

Kestra orchestrates your flows using Flowable Tasks. These tasks do not perform any heavy computation. Instead, they control the orchestration behavior, allowing you to implement more advanced workflow patterns.

Example Flowable tasks include:

  • io.kestra.plugin.core.flow.Parallel
  • io.kestra.plugin.core.flow.Switch
  • io.kestra.plugin.core.flow.ForEachItem

Read the full list on the Flowable tasks page.

Runnable Tasks

In Kestra, most data processing workloads are executed using Runnable Tasks.

In contrast to Flowable Tasks, Runnable Tasks are responsible for performing the actual work. For example, file system operations, API calls, database queries, etc. These tasks can be compute-intensive and are handled by workers.

Example runnable tasks include:

  • io.kestra.plugin.scripts.python.Commands
  • io.kestra.plugin.core.http.Request
  • io.kestra.plugin.notifications.slack.SlackExecution

Core task properties

The table below lists the core task properties available to all tasks.

FieldDescription
idThe flow identifier, must be unique inside a flow.
typeThe Java Fully Qualified Class Name of the task.
descriptionThe description of the task
retryTask retry behavior
timeoutTask timeout expressed in ISO 8601 Durations.
disabledSet it to true to disable execution of the task.
workerGroup.keyTo execute this task on a specific Worker Group (EE).
logLevelTo define the log level granularity for which logs will be inserted into the backend database. By default, all logs are stored. However, if you restrict that to e.g., the INFO level, all lower log levels such as DEBUG and TRACE won't be persisted.
allowFailureBoolean flag allowing to continue the execution even if this task fails.

Dynamic vs. static task properties

Task properties can be static or dynamic. Dynamic task properties can be set using expressions. To find out whether a given task property is static or dynamic, check the task documentation available on the plugin's homepage as well as in the UI when you hover over a task and click on the documentation tab on the right.

dynamic_properties

Often some task properties are marked as not dynamic because they are complex types (e.g. maps, list of strings, list of maps), meaning that they are placeholders for other dynamic properties. Let's take the runTasks property of Databrick's SubmitRun task as an example. This property is not dynamic because it's an array of RunSubmitTaskSetting.

On top of that, RunSubmitTaskSetting is a group of other properties which are also either dynamic or of complex type (placeholder for other properties). It's therefore useful to always drill down to the lowest level — most properties at the lowest level are dynamic and can be templated using expressions.

Was this page helpful?