Ship Kestra execution logs to Datadog for centralized observability

Ship Kestra execution logs to Datadog so workflow and task output lands in the same observability platform your teams already use for metrics, traces, and alerts. This blueprint sends batched Kestra logs to the Datadog HTTP log intake API on a daily schedule, replacing scattered local logs and manual exports with a centralized, searchable log stream you can correlate with infrastructure health and incident timelines.

How it works

A single io.kestra.plugin.ee.core.log.LogShipper task named log_export collects logs and forwards them to Datadog. It filters by severity with logLevelFilter: INFO, groups entries with batchSize: 1000 for efficient transmission, and replays recent history with lookbackPeriod: P1D (the previous day). The shipper defines one exporter, io.kestra.plugin.ee.datadog.LogExporter, pointed at the EU intake endpoint basePath: https://http-intake.logs.datadoghq.eu and authenticated with apiKey. A io.kestra.plugin.core.trigger.Schedule trigger named daily runs the flow on the cron 0 7 * * *, every morning at 7am.

What you get

• Kestra execution and task logs centralized in Datadog Logs
• Batched, secure delivery over the Datadog HTTP log intake API
• A configurable lookback window so no recent logs are missed
• A foundation for dashboards, log-based monitors, and alerting
• Correlation between orchestration logs and your APM traces and metrics

Who it's for

• Platform and SRE teams running Kestra in production
• DevOps engineers consolidating observability in Datadog
• Data engineers who need pipeline logs alongside infrastructure signals
• On-call responders who debug incidents from a single log view

Why orchestrate this with Kestra

Datadog ingests logs but does not pull them out of Kestra on its own. Kestra closes that gap declaratively: the entire pipeline is defined in version-controlled YAML, the Schedule trigger drives delivery on an event-based cadence instead of an external cron you have to babysit, and built-in retries plus full execution lineage mean a failed shipment is visible, retryable, and auditable. You manage one source of truth rather than wiring brittle log-forwarding scripts.

Prerequisites

• A Kestra instance with the Enterprise log shipper available
• A Datadog account with log management enabled
• Network access from Kestra to the Datadog intake endpoint

Secrets

• DATADOG_API_KEY: your Datadog API key used to authenticate log intake

Quick start

1. Add the DATADOG_API_KEY secret to your Kestra instance.
2. Confirm basePath matches your Datadog site (the default targets the EU site).
3. Add this blueprint to a namespace and save the flow.
4. Trigger one manual execution and confirm logs appear in Datadog Logs.
5. Leave the daily schedule enabled to keep logs flowing every morning.

How to extend

• Lower logLevelFilter to WARN or ERROR to reduce volume, or DEBUG for deep troubleshooting.
• Tune batchSize and lookbackPeriod to match your log throughput and run cadence.
• Change the Schedule cron to ship more frequently, for example hourly.
• Add more entries under logExporters to fan logs out to other destinations alongside Datadog.

Links

• Datadog plugin documentation