Gate a Publish on Metaplane Monitors

Data observability tools tell you when something broke; orchestration decides what happens next. This blueprint wires the two together with io.kestra.plugin.metaplane.Gate, the synchronous quality gate from the Metaplane plugin. Before the publish step runs, the gate triggers a fresh execution of every monitor guarding the tables about to ship, polls until the new results land, and fails the flow the moment any monitor reports a failing status. The publish step only ever executes behind a green gate, and a blocked publish posts to Slack with instructions instead of failing silently.

How it works

1. quality_gate (io.kestra.plugin.metaplane.Gate) receives the list of monitorIds guarding the publish. With runFirst: true it enqueues a fresh run of each monitor instead of trusting whatever result happens to be cached.
2. The task polls every 15 seconds (pollInterval: PT15S) for up to 10 minutes (timeout: PT10M). maxAge: PT1H rejects results older than an hour, so a monitor that never ran again cannot sneak a stale PASS through the gate.
3. failStrategy: FAIL_FAST stops polling and fails the task the moment any single monitor lands in the failOn set, here FAIL and ERROR. The other strategies are FAIL_IF_ANY, which waits for all results before deciding, FAIL_IF_ALL, and NONE, which never fails and only reports through the passed output.
4. publish runs only when the gate passed. It is a stand-in log line; replace it with the real exposure step.
5. notify_published confirms the shipped publish in Slack, and the errors block posts a distinct blocked-publish alert pointing at the gate task logs, where the failing monitor IDs are listed.

What you get

• A publish that structurally cannot ship on red data, because the gate task fails before the publish task starts.
• Fresh evidence on every run: runFirst plus maxAge means the gate never trusts a stale result.
• Fast feedback with FAIL_FAST, since one failing monitor ends the wait immediately instead of burning the full timeout.
• Both outcomes visible in Slack, a shipped publish and a blocked one each get their own message.

Who it's for

• Analytics engineers who want dbt builds, dashboard refreshes, or reverse ETL syncs to stop when Metaplane flags an anomaly.
• Data platform teams turning observability alerts into enforcement instead of after-the-fact notifications.
• Anyone who has shipped a dashboard on top of a table that a monitor had been flagging for hours.

Why orchestrate this with Kestra

Metaplane knows the health of the tables; Kestra owns the moment of decision. Running the gate as a task in the same flow as the publish gives you an atomic, auditable checkpoint: the execution history shows exactly which publishes shipped, which were blocked, and what the monitors said at that moment. No glue code polls the Metaplane API, the Gate task handles enqueueing, polling, staleness, and the failure strategy declaratively.

Prerequisites

• A Metaplane workspace with monitors on the tables the publish depends on, and their monitor UUIDs.
• A Metaplane API token.
• A Slack incoming webhook for publish and blocked-publish notifications.

Secrets

• METAPLANE_API_TOKEN: Metaplane API token.
• SLACK_WEBHOOK_URL: Slack incoming webhook URL.

Quick start

1. Add both secrets to your Kestra namespace.
2. Replace the two placeholder monitorIds with UUIDs of monitors from your Metaplane workspace.
3. Execute the flow; a healthy table set posts the shipped message, a failing monitor blocks the publish and posts the alert.
4. Replace the publish stand-in with your real exposure step and enable the nightly_publish trigger.

How to extend

• Switch to failStrategy: FAIL_IF_ANY when you want the full picture of all monitors before failing, at the cost of waiting for every result.
• Add UNKNOWN or FAILED_TO_PREDICT to failOn to treat inconclusive monitors as blockers for high-stakes publishes.
• Set failStrategy: NONE and branch on the passed output with io.kestra.plugin.core.flow.If to publish to a quarantine location instead of blocking outright.
• Pair with the Metaplane post-load monitor run blueprint so monitors are refreshed right after loads, keeping gate waits short.

Links

• Metaplane plugin documentation
• Metaplane documentation