Dagger Engine Cache Maintenance Gate with Kestra

id: dagger-engine-cache-maintenance-gate
namespace: company.team

description: |
  Run a trivial Dagger pipeline to confirm the engine is reachable, classify it HEALTHY or
  PROBE_FAILED from the real command outcome, then on a healthy engine optionally run the
  engine's own local-cache prune as a scheduled maintenance step instead of waiting on
  Dagger's own automatic, threshold-triggered garbage collection.

inputs:
  - id: auto_prune_cache
    type: BOOL
    displayName: Auto prune cache
    description: When true, after a healthy connectivity probe, run the engine's own
      local-cache prune as scheduled maintenance.
    defaults: false

triggers:
  - id: every_night
    type: io.kestra.plugin.core.trigger.Schedule
    description: Shipped disabled.
    cron: "0 2 * * *"
    disabled: true
    inputs:
      auto_prune_cache: false

tasks:
  - id: probe_engine_connectivity
    type: io.kestra.plugin.dagger.Commands
    description: Runs a trivial pipeline with no external dependency, confirming the
      Dagger CLI installs and the engine actually executes a container, not just
      that the binary is present.
    allowFailure: true
    taskRunner:
      type: io.kestra.plugin.core.runner.Process
    commands:
      - container | from alpine | with-exec echo DAGGER_ENGINE_OK | stdout

  - id: classify_engine_status
    type: io.kestra.plugin.core.debug.Return
    description: "PROBE_FAILED when the probe did not return at all, meaning the
      pipeline threw (a non-zero exit from the Dagger CLI or the Process task
      runner), HEALTHY otherwise. ScriptOutput only exposes structured vars
      through the echo-marker convention documented on AwsCLI, which this probe
      does not use, so classification relies on allowFailure plus outputs
      existing, the same verified mechanism used by every other probe task in
      this flow style, not on parsing stdout content."
    format: "{{ outputs.probe_engine_connectivity is not defined ? 'PROBE_FAILED' :
      'HEALTHY' }}"

  - id: route_by_engine_status
    type: io.kestra.plugin.core.flow.Switch
    description: Branch A proceeds to the optional cache maintenance step. Branch B
      alerts Slack, a broken engine connection means a prune would not run
      anyway.
    value: "{{ outputs.classify_engine_status.value }}"
    cases:
      "HEALTHY":
        - id: maintain_cache_gate
          type: io.kestra.plugin.core.flow.If
          description: Runs the engine's own cache prune only when auto_prune_cache is
            true, on an already-confirmed-healthy engine.
          condition: "{{ inputs.auto_prune_cache }}"
          then:
            - id: prune_local_cache
              type: io.kestra.plugin.dagger.Commands
              description: Calls the engine's own documented local-cache prune command. This
                is the engine's own scheduled maintenance, not a separate
                invented cleanup, Dagger's engine otherwise only prunes
                automatically once disk usage crosses its own internal
                threshold.
              allowFailure: true
              taskRunner:
                type: io.kestra.plugin.core.runner.Process
              commands:
                - dagger core engine local-cache prune

            - id: report_prune_result
              type: io.kestra.plugin.core.flow.If
              description: Fails loudly if the prune command itself did not complete, rather
                than silently treating a failed prune as a successful one.
              condition: "{{ outputs.prune_local_cache is not defined }}"
              then:
                - id: alert_prune_failed
                  type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
                  url: "{{ secret('SLACK_WEBHOOK') }}"
                  payload: |
                    {{ {"text": "Dagger engine cache prune did not complete. The engine was reachable moments earlier, check execution " ~ execution.id ~ " for the real error."} | toJson }}
              else:
                - id: log_prune_succeeded
                  type: io.kestra.plugin.core.log.Log
                  message: "Dagger engine local-cache prune completed on a healthy engine."
          else:
            - id: log_healthy_no_prune
              type: io.kestra.plugin.core.log.Log
              message: "Dagger engine reachable, auto_prune_cache is false, no maintenance
                action taken."
      "PROBE_FAILED":
        - id: alert_probe_failed
          type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
          url: "{{ secret('SLACK_WEBHOOK') }}"
          payload: |
            {{ {"text": "Dagger engine connectivity probe failed. The engine did not execute the trivial container pipeline. Check the worker's Dagger CLI install and engine reachability."} | toJson }}
    defaults:
      - id: log_unexpected_status
        type: io.kestra.plugin.core.log.Log
        message: "Unexpected classify_engine_status value: {{
          outputs.classify_engine_status.value }}"

  - id: log_audit_summary
    type: io.kestra.plugin.core.log.Log
    description: Final audit log confirming the gate ran, regardless of the branch
      taken above.
    message: "Dagger engine cache maintenance gate completed. status={{
      outputs.classify_engine_status.value }} auto_prune_cache={{
      inputs.auto_prune_cache }}"

errors:
  - id: alert_flow_failure
    type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
    url: "{{ secret('SLACK_WEBHOOK') }}"
    payload: |
      {{ {"text": "Dagger engine cache maintenance gate flow failed (execution " ~ execution.id ~ "). Check the execution logs."} | toJson }}

The Dagger engine's own garbage collector only prunes the local cache once disk usage crosses its own internal threshold, which means cache entries a team wanted to keep can disappear at an unpredictable moment chosen by disk pressure, not by policy. Running the engine's own documented prune command on a schedule, instead, puts that decision back under a team's control. This blueprint confirms the engine is actually executing containers, not just that the CLI binary is present, before deciding whether to run that maintenance step at all.

How it works

probe_engine_connectivity (io.kestra.plugin.dagger.Commands) runs a trivial pipeline, container | from alpine | with-exec echo DAGGER_ENGINE_OK | stdout, with allowFailure: true so a broken install or unreachable engine does not stop the flow before it can be classified as PROBE_FAILED below.
classify_engine_status (io.kestra.plugin.core.debug.Return) returns PROBE_FAILED when the probe did not return at all, HEALTHY otherwise.
route_by_engine_status (io.kestra.plugin.core.flow.Switch) branches on that status. On HEALTHY, a nested io.kestra.plugin.core.flow.If runs dagger core engine local-cache prune only when auto_prune_cache is true, and a second If fails loudly to Slack if the prune itself did not complete, rather than reporting a failed prune as a success. On PROBE_FAILED, Slack is alerted instead. A defaults branch logs anything unexpected.
log_audit_summary always runs last. The errors block posts to Slack if the flow itself fails outside the gate logic above.

What you get

A real connectivity check that the engine executes a container, not a weaker check that only confirms the CLI is installed.
Cache maintenance on your own schedule, not only when the engine's internal disk threshold forces it.
A prune failure that is reported as a failure, never silently treated as a success.

Who it's for

Platform teams running Dagger as shared CI/CD infrastructure who want predictable cache maintenance instead of relying on the engine's own threshold-triggered GC.
Teams evaluating the Dagger plugin who want a working example of io.kestra.plugin.dagger.Commands beyond a single one-off pipeline call.

Why orchestrate this with Kestra

Dagger's engine has no scheduler of its own, cache maintenance either happens automatically at a disk-pressure threshold or has to be triggered by hand. Kestra adds the schedule, the pre-check that the engine is actually healthy first, and the Slack alert on either a connectivity or a prune failure, as one declarative, auditable flow.

Prerequisites

A worker able to run the io.kestra.plugin.core.runner.Process task runner, with network access to install the pinned Dagger CLI if not already present, and a reachable Dagger engine, either local (Docker Desktop style) or remote.
For local testing, Dagger's own official quickstart, which runs an engine inside Docker automatically the first time any dagger command executes, no separate engine container needs to be started by hand.
A Slack incoming webhook for alerts.

Secrets

SLACK_WEBHOOK: Slack incoming webhook URL used by the alert tasks and the errors block.

In Kestra OSS, set this as a base64-encoded environment variable, for example SECRET_SLACK_WEBHOOK.

Quick start

Install Docker and the Dagger CLI is auto-installed by the task itself on first run.
Add the SLACK_WEBHOOK secret.
Run this flow manually with the defaults, it should report HEALTHY and take no maintenance action since auto_prune_cache defaults to false.
Run it again with auto_prune_cache: true, it should report the prune completing on the already-confirmed-healthy engine.
Enable the every_night trigger.

How to extend

Parse dagger core engine local-cache entry-set for real cache size and entry count once you have confirmed its exact output format against your own installed Dagger version, this blueprint deliberately does not parse that output since the exact field names were not confirmed against primary documentation at build time.
Add --use-default-policy or an explicit space-threshold flag to local-cache prune, once confirmed against your installed version's own --help output, to prune to a specific target rather than the command's own default behavior.
Chain a second pipeline after a healthy probe that actually builds your real project, turning this into a pre-flight check before a CI run rather than a synthetic probe.

Pitfalls

This blueprint deliberately does not parse local-cache entry-set for a cache size or entry count, I could not confirm its exact JSON or table output shape against primary Dagger documentation, only secondary sources, so no threshold-based classification is built on top of it. See How to extend before relying on that data.
probe_engine_connectivity only confirms the engine executed one Alpine container successfully, not that a real, larger pipeline using your own images and registries will also succeed.
Running local-cache prune removes cache entries. On a shared engine serving several pipelines, this can increase build time for whichever pipeline runs next and has to rebuild from a cold cache, time this trigger for a genuinely idle window.

Dagger Engine Cache Maintenance Gate

How it works

What you get

Who it's for

Why orchestrate this with Kestra

Prerequisites

Secrets

Quick start

How to extend

Pitfalls

Links

Tasks

Orchestrate with Kestra

Share this Blueprint

Create a Blueprint

New to Kestra?

Dagger Engine Cache Maintenance Gate

How it works

What you get

Who it's for

Why orchestrate this with Kestra

Prerequisites

Secrets

Quick start

How to extend

Pitfalls

Links

Tasks

Orchestrate with Kestra

Share this Blueprint

Create a Blueprint

More Infrastructure Blueprints

Automatically Create a GitHub Issue When a Workflow or Job Fails

CouchDB Active Task and Continuous Replication Sentinel

Apache CouchDB Database Compaction and Conflict Resolution Gate

Audit Couchbase GSI Secondary Indexes and Detect Offline States

Couchbase TTL expiry triage sentinel

CrateDB Shard Allocation and Cluster Reroute Sentinel

More Core Blueprints

Crony

Trigger PagerDuty Alerts Automatically for Incident Response

Automatically Create a GitHub Issue When a Workflow or Job Fails

Create Jira Issues Automatically for Incident Management

Create HubSpot Tickets Automatically for Incident Management

Create a ticket in Zendesk when a workflow fails

New to Kestra?