TriggerDagRun icon

Trigger an Airflow DAG

Trigger an Apache Airflow DAG from Kestra over the REST API, wait for completion, and pass Kestra execution metadata into the DAG run for end-to-end lineage.

Categories
Data

Trigger an Apache Airflow DAG run directly from Kestra, wait for it to complete, and capture its final state. This blueprint calls the Airflow REST API with basic authentication, so you can drive existing Airflow DAGs from a Kestra flow without rewriting them. It solves the common migration and coexistence problem of running Kestra and Airflow side by side: Kestra becomes the control plane that launches, waits on, and reacts to Airflow workloads while passing its own execution metadata into each run for full cross-platform lineage.

How it works

  1. The dag_id input sets which DAG to trigger (default example_astronauts), so the same flow can target any DAG without editing tasks.
  2. The trigger_dag task of type io.kestra.plugin.airflow.dags.TriggerDagRun posts a DAG run request to the Airflow REST API at baseUrl, authenticating with basicAuthUser and basicAuthPassword.
  3. wait: true with pollFrequency: PT1S makes Kestra block and poll the run every second until Airflow reports a terminal state.
  4. The request body.conf injects Kestra context (source, namespace, flow, task, execution) into the DAG run, so each Airflow run records exactly which Kestra execution launched it.

What you get

  • A single declarative task that launches an Airflow DAG and blocks until it finishes.
  • Outputs you can branch on downstream: {{ outputs.trigger_dag.dagId }}, {{ outputs.trigger_dag.dagRunId }}, and {{ outputs.trigger_dag.state }}.
  • Cross-platform lineage: every triggered DAG run carries the originating Kestra namespace, flow, task, and execution ID.

Who it's for

  • Data engineers migrating from Airflow to Kestra who need both systems to interoperate during the transition.
  • Platform teams that want one orchestrator to coordinate workloads spread across multiple tools.
  • Analytics and ML teams that depend on existing Airflow DAGs but want event-driven, parameterized triggering.

Why orchestrate this with Kestra

Airflow's own scheduler is built around time-based DAG schedules and cannot easily start a DAG from arbitrary external events, downstream flows, or other tools. With Kestra you wrap the DAG trigger in declarative YAML, launch it from event triggers, webhooks, or upstream flows, add retries and timeouts, and capture structured outputs for the rest of your pipeline. The injected conf metadata gives you lineage across both platforms, which a standalone Airflow deployment does not provide.

Prerequisites

  • An Apache Airflow instance reachable from Kestra with the REST API enabled and basic auth configured.
  • At least one DAG present in Airflow that can be triggered via the API.

Secrets

  • AIRFLOW_BASE_URL: base URL of the Airflow instance (for example http://airflow:8080).
  • AIRFLOW_USERNAME: Airflow basic auth username.
  • AIRFLOW_PASSWORD: Airflow basic auth password.

Quick start

  1. Add the AIRFLOW_BASE_URL, AIRFLOW_USERNAME, and AIRFLOW_PASSWORD secrets to your Kestra instance.
  2. Add this blueprint to a namespace.
  3. Execute the flow, keeping the default dag_id or setting your own DAG ID.
  4. Inspect the outputs to confirm state is success and note the returned dagRunId.

How to extend

  • Add a Schedule or webhook trigger to launch the DAG on a cadence or external event instead of manually.
  • Chain a downstream task on {{ outputs.trigger_dag.state }} to send alerts or run cleanup only on success or failure.
  • Parameterize body.conf with more inputs to pass run-specific configuration into the DAG.
  • Add retry and timeout properties to the trigger_dag task to harden long-running runs.

Links

Share this Blueprint
See How

New to Kestra?

Use blueprints to kickstart your first workflows.