​Purge

Purge old executions and logs to save disk space.

As your workflows grow, you may need to clean up old executions and logs to save disk space.

The recommended way to clean execution and logs is using a combination of io.kestra.plugin.core.execution.PurgeExecutions and the newly added io.kestra.plugin.core.log.PurgeLogs task as shown below. The PurgeLogs task removes all logs (both Execution logs and Trigger logs) in a performant batch operation. Combining those two together will give you the same functionality as the previous io.kestra.plugin.core.storage.Purge task but in a more performant and reliable way (roughly 10x faster).

id: purge
namespace: company.myteam
description: |
  This flow will remove all executions and logs older than 1 month.
  We recommend running this flow daily to avoid running out of disk space.

tasks:
  - id: purge_executions
    type: io.kestra.plugin.core.execution.PurgeExecutions
    endDate: "{{ now() | dateAdd(-1, 'MONTHS') }}"
    purgeLog: false

  - id: purge_logs
    type: io.kestra.plugin.core.log.PurgeLogs
    endDate: "{{ now() | dateAdd(-1, 'MONTHS') }}"

triggers:
  - id: daily
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "@daily"

Purge tasks vs. UI deletion

It is important to note the difference between executing a purge task to clean your Kestra instance versus deleting items such as flows, executions, or logs in the UI. Using a purge task hard deletes those records from your Kestra instance (i.e., the deleted data is unrecoverable). This clears up disk space storage and is an effective practice to take with Kestra data that is no longer needed or stored externally. For example, you can purge any log data more than one month old on a schedule to keep your instance clean.

Alternatively, deleting executions or logs from the UI performs a soft deletion of the data and does not fully wipe the storage as a purge task would. For example, if you delete a flow and then create a flow with the same name and id, then the historical data from that flow will reappear such as revision history and past executions. This data is retained because it provides a full history of the flow and its revisions for troubleshooting or auditing.