Finally Tasks in Kestra – Always-Run Cleanup icon Finally Tasks in Kestra – Always-Run Cleanup

OVERVIEW

Created by

Kestra Core Team

Managed by

Kestra Core Team

Define a block of tasks that always run at the end of a flow, regardless of task status.

Finally tasks – always-run cleanup

finally tasks are useful for cleanup operations that must run at the end of your flow, whether the execution ends in success or failure.

finally component

finally is a block of tasks that execute at the end of your workflow, regardless of the status of prior tasks. This ensures cleanup or teardown steps always occur, no matter how the flow ends.

For example, you might use a finally block to turn off a cloud service when the flow finishes, regardless of the outcome.

finally example

In the example below, a task is programmed to fail, and an error task logs a message as an alert communicating the failure. The finally task runs after the other tasks have finished regardless of the failure, in this case logging another message, but could alternatively be used to shut down any resources specifically spun up to use in the flow.

id: finally_example
namespace: company.team


tasks:
  - id: fail
    type: io.kestra.plugin.core.execution.Fail
    errorMessage: Test downstream tasks


errors:
  - id: send_alert
    type: io.kestra.plugin.core.log.Log
    message: alert on failure


finally:
- id: cleanup_task
  type: io.kestra.plugin.core.log.Log
  message: cleaning up resources

Change the example to ensure the end state of the first task is a success, like below, and you can see that the finally task runs the same as before:

id: finally_example
namespace: company.team


tasks:
  - id: log
    type: io.kestra.plugin.core.log.Log
    errorMessage: "This flow executes successfully!"


errors:
  - id: send_alert
    type: io.kestra.plugin.core.log.Log
    message: alert on failure


finally:
- id: cleanup_task
  type: io.kestra.plugin.core.log.Log
  message: cleaning up resources

Like in the first iteration of the flow, the finally task runs at the end despite the errors task not sending an alert, ensuring any cleanup operations still take place regardless of status.

Beyond simple cleanup, finally can manage external services. For example, you might spin up Redis, Elasticsearch, or Kafka to run queries or QA checks, and then ensure the service is stopped when the flow ends. The below example demonstrates spinning up a Docker container with Redis to run some database operations and then stop the container when the flow is finished.

id: dockerRedis
namespace: company.team


variables:
  host: host.docker.internal


tasks:
  - id: start
    type: io.kestra.plugin.docker.Run
    containerImage: redis
    wait: false
    portBindings:
      - "6379:6379"


  - id: sleep
    type: io.kestra.plugin.core.flow.Sleep
    duration: PT1S
    description: Wait for the Redis container to start


  - id: set
    type: io.kestra.plugin.redis.string.Set
    url: "redis://:redis@{{vars.host}}:6379/0"
    key: "key_string_{{execution.id}}"
    value: "{{flow.id}}"
    serdeType: STRING


  - id: get
    type: io.kestra.plugin.redis.string.Get
    url: "redis://:redis@{{vars.host}}:6379/0"
    key: "key_string_{{execution.id}}"
    serdeType: STRING


  - id: assert
    type: io.kestra.plugin.core.execution.Assert
    errorMessage: "Invalid get data {{outputs.get}}"
    conditions:
      - "{{outputs.get.data == flow.id}}"


  - id: delete
    type: io.kestra.plugin.redis.string.Delete
    url: "redis://:redis@{{vars.host}}:6379/0"
    keys:
      - "key_string_{{execution.id}}"


  - id: getAfterDelete
    type: io.kestra.plugin.redis.string.Get
    url: "redis://:redis@{{vars.host}}:6379/0"
    key: "key_string_{{execution.id}}"
    serdeType: STRING


  - id: assertAfterDelete
    type: io.kestra.plugin.core.execution.Assert
    errorMessage: "Invalid get data {{outputs.getAfterDelete}}"
    conditions:
      - "{{(outputs.getAfterDelete contains 'data') == false}}"


finally:
  - id: stop
    type: io.kestra.plugin.docker.Stop
    containerId: "{{outputs.start.taskRunner.containerId}}"

Was this page helpful?