Purge Old Data in Kestra – Executions, Logs, Key-Value Store, Files

Use purge tasks to remove old executions, logs, and key-value pairs, helping reduce storage usage.

Purge old execution data safely

To keep storage optimized, use io.kestra.plugin.core.execution.PurgeExecutions, io.kestra.plugin.core.log.PurgeLogs, and io.kestra.plugin.core.kv.PurgeKV.

• PurgeExecutions: deletes execution records
• PurgeLogs: removes both Execution and Trigger logs in bulk
• PurgeKV: deletes expired keys globally for a specific namespace

Together, these replace the legacy io.kestra.plugin.core.storage.Purge task with a faster and more reliable process (~10x faster).

The flow below purges executions and logs:

id: purge
namespace: company.myteam
description: |
  This flow will remove all executions and logs older than 1 month.
  We recommend running it daily to prevent storage issues.

tasks:
  - id: purge_executions
    type: io.kestra.plugin.core.execution.PurgeExecutions
    endDate: "{{ now() | dateAdd(-1, 'MONTHS') }}"
    purgeLog: false

  - id: purge_logs
    type: io.kestra.plugin.core.log.PurgeLogs
    endDate: "{{ now() | dateAdd(-1, 'MONTHS') }}"

triggers:
  - id: daily
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "@daily"

Purge Key-value pairs

The example below purges expired Key-value pairs from the company Namespace. It’s set up as a flow in the system Namespace.

id: purge_kv_store
namespace: system

tasks:
  - id: purge_kv
    type: io.kestra.plugin.core.kv.PurgeKV
    expiredOnly: true
    namespaces:
      - company
    includeChildNamespaces: true

Auto-delete expired key-value pairs

Rather than creating a system flow to regularly purge Key-value pairs, you can add a global configuration to your Kestra Configuration/Application file that auto-deletes expired key-value pairs:

kestra:
  kv:
    purge-expired:
      enabled: true # default true
      initial-delay: PT5S # default PT6H
      fixed-delay: PT5S # default PT6H
      batch-size: 10 # default 1000

Purge Namespace files

The example below purges old versions of Namespace files for a Namespace tree (parents + children Namespaces). Use a filePattern and specify the behavior (e.g., keep the last N versions and/or delete versions older than a given date):

id: purge_namespace_files
namespace: system

tasks:
  - id: purge_files
    type: io.kestra.plugin.core.namespace.PurgeFiles
    namespaces:
      - company
    includeChildNamespaces: true
    filePattern: "**/*.sql"
    behavior:
      type: version
      before: "2025-01-01T00:00:00Z"

Refer to the PurgeFiles documentation for more details.

Purge assets and lineage (retention)

Use the io.kestra.plugin.ee.assets.PurgeAssets task to enforce asset retention without touching executions or logs. By default, this task purges assets, asset usage events (execution view), and asset lineage events (for asset exporters) matching the filters. You can configure it to only purge specific types of records.

Filters:

Purge scope:

Outputs: purgedAssetsCount, purgedAssetUsagesCount, purgedAssetLineagesCount.

Example: purge old VM assets on a monthly schedule.

id: asset_retention_policy
namespace: company.infra

triggers:
  - id: monthly_cleanup
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "0 0 1 * *"

tasks:
  - id: purge_old_vms
    type: io.kestra.plugin.ee.assets.PurgeAssets
    assetType:
      - io.kestra.plugin.ee.assets.VM
    endDate: "{{ now() | dateAdd(-180, 'DAYS') }}"

Purge tasks vs. UI deletion

Purge tasks perform hard deletion, permanently removing records and reclaiming storage. In contrast, deleting items in the UI is a soft deletion—the data is hidden but retained (e.g., revision history and past executions can reappear if a flow with the same ID is recreated).

This distinction matters for compliance and troubleshooting: purge flows are best for cleaning up space, while UI deletions preserve history for auditability.