Airbyte Connection Sync Failure Sentinel

This repo's own airbyte-cloud-sync.yaml blueprint runs a bare per-minute sync with no failure handling at all - its own "How to extend" section names exactly what is missing: "Add retry and failure alerting (Slack, email) for resilient ingestion." This blueprint is that extension. It syncs a connection on a schedule, classifies whether the sync actually finished successfully, alerts Slack the moment it does not, and can launch one fresh retry sync behind two independent gates.

How it works

1. periodic_sync_check (io.kestra.plugin.core.trigger.Schedule) runs every 6 hours, always forcing auto_remediate: "false" and dry_run: "true" through its own inputs: override regardless of this flow's defaults. Shipped disabled so you can validate a manual run first.
2. sync_connection (io.kestra.plugin.airbyte.connections.Sync, allowFailure: true) starts (or adopts an already-running) sync and, with wait enabled by the task's own default, polls until the job reaches a terminal state. Verified from the plugin's own Java source: the task only returns its Output (jobId, alreadyRunning, adopted) once the job finishes SUCCEEDED; any other terminal state makes it throw.
3. evaluate_sync_health (io.kestra.plugin.core.debug.Return) classifies HEALTHY when sync_connection's output is defined (the sync succeeded) or AT_RISK when it is not (the task threw and allowFailure caught it).
4. route_by_health (io.kestra.plugin.core.flow.Switch) branches on that classification:
   • HEALTHY: log_healthy records the successful sync, naming its job ID.
   • AT_RISK: a nested route_remediation (Switch on auto_remediate) either considers remediation ("true", gated again by dry_run through check_dry_run) or only logs that remediation is disabled ("false"). When both gates allow it, retry_sync_connection launches one fresh sync attempt. alert_at_risk always fires to Slack regardless of which path ran.
5. log_audit_complete always runs last, printing the classification regardless of which branch fired.
6. The errors block alerts Slack separately if the flow itself fails outside the handled AT_RISK branch.

What you get

• A scheduled Airbyte sync with real failure handling, where the base blueprint it extends has none.
• A clean HEALTHY/AT_RISK classification derived directly from the plugin's own success/throw behavior, not an invented status field.
• Two independent gates (auto_remediate, dry_run) between "sync did not succeed" and "a retry sync actually launched".
• A final status log every run, so the execution history doubles as a sync-reliability trend for connection_id.

Who it's for

• Data engineering teams running Airbyte connections who need the schedule itself to notice a failed sync, instead of discovering it from a stale downstream table.
• Anyone extending this repo's own airbyte-cloud-sync.yaml - its own documentation names this exact gap.
• Teams who want an Airbyte-native alternative to the Soda/Metaplane post-load quality checks already in this repo, which monitor data quality after a sync rather than the sync's own terminal status.

Why orchestrate this with Kestra

Airbyte's own Sync call either finishes or throws; it has no independent retry policy, no alerting path of its own, and no gate between "the sync failed" and "try again automatically." Kestra supplies the schedule, the classification as a first-class branch, two independent authorization gates before a retry is launched, and an execution history that shows exactly when a sync first stopped succeeding and whether a retry was attempted.

Prerequisites

• A reachable Airbyte instance (OSS or Cloud) with an existing connection to audit.
• A Slack incoming webhook for alerts.

Local testing: docker run -d --name airbyte-sentinel -p 8001:8000 airbyte/airbyte-abctl:latest (Airbyte OSS's own install path varies by version; follow Airbyte's current self-hosted quickstart for the exact compose/install command for your target version, then point airbyte_url at the resulting API endpoint - this has not been exercised end to end in this environment). docker network connect YOUR_KESTRA_NETWORK <container> (check docker network ls first; only needed if the Kestra Worker runs in a separate Docker network than the Airbyte containers).

Secrets

• AIRBYTE_USERNAME / AIRBYTE_PASSWORD: basic auth credentials, matching this plugin's own documented pattern (see airbyte-cloud-sync.yaml).
• SLACK_WEBHOOK_URL: Slack incoming webhook used by alert_at_risk and the errors block.
• In Kestra OSS (no Enterprise secrets backend), secrets are supplied as environment variables prefixed SECRET_, base64-encoded, and read back in flows with {{ secret('NAME') }} - for example SECRET_AIRBYTE_PASSWORD=$(echo -n '<your-airbyte-password>' | base64). This keeps credentials out of the flow YAML but, per Kestra's own documentation, offers no encryption at rest or access control beyond the host environment; use the Enterprise secrets backend for stronger guarantees.

Inputs

• connection_id (STRING, no default): Airbyte connection synced and audited.
• airbyte_url (STRING, default http://localhost:8000/): Airbyte API base URL.
• max_wait_duration (STRING, default PT60M): ceiling this flow waits for a terminal state.
• auto_remediate (SELECT: "false", "true"; default "false"): must be "true" for a retry sync to even be considered.
• dry_run (SELECT: "true", "false"; default "true"): must be explicitly "false", together with auto_remediate: "true", for a fresh sync to actually launch.

Outputs

• outputs.sync_connection.jobId / .alreadyRunning / .adopted: present only when the sync succeeded.
• outputs.evaluate_sync_health.value: HEALTHY or AT_RISK for every run.
• outputs.retry_sync_connection: present only when remediation actually launched a retry and that retry also succeeded.

Quick start

1. Point connection_id/airbyte_url at a real Airbyte connection that is known to sync successfully.
2. Add the AIRBYTE_USERNAME, AIRBYTE_PASSWORD, and SLACK_WEBHOOK_URL secrets.
3. Run the flow manually and confirm log_healthy fires with the real jobId.
4. Point connection_id at a connection configured to fail (an invalid source credential is an easy way to force this in a test environment) and re-run; confirm alert_at_risk fires, and that log_remediation_disabled logs without launching a retry.
5. Run once with auto_remediate: "true" and dry_run: "true" against that same failing connection and confirm log_dry_run_remediation describes the retry without launching it.
6. Run once with auto_remediate: "true" and dry_run: "false" and confirm retry_sync_connection runs, visible as a new job in the Airbyte UI.
7. Enable periodic_sync_check once you trust the check; it always runs in the safe auto_remediate: "false" / dry_run: "true" mode regardless of what you leave the flow's own defaults set to.

How to extend

• Loop over several connection_id values with io.kestra.plugin.core.flow.Loop to audit every critical connection in an Airbyte workspace in one pass.
• Cap the number of automatic retries by tracking a counter in vars or an external store, so a connection that keeps failing escalates to a human instead of retrying forever on every scheduled audit.
• Chain this flow with this repo's own soda-post-load-quality-scan.yaml or metaplane-post-load-monitor-run.yaml so a single schedule covers both "did the sync finish" and "is the resulting data actually good."
• Route the remediation branch through a Pause approval task instead of (or alongside) the static auto_remediate input, for a human-reviewed retry on a production connection.

Pitfalls

• This gate is two-state (HEALTHY/AT_RISK), not three, by design of the plugin itself. Verified directly from Sync's Java source: when wait is enabled, the task returns its Output object only after confirming the job reached SUCCEEDED; any other terminal state (FAILED, CANCELLED, INCOMPLETE) or a timeout past maxDuration makes the task throw instead of returning a classifiable status string. There is no UNREACHABLE-style third state here, because a connectivity failure and a sync failure both surface the same way: the task throws and allowFailure catches it.
• Sync adopts an already-running sync rather than always starting a new one. Verified from source (onActiveSync, default behavior documented as ADOPT): if a sync for connection_id is already in progress when this flow runs, it polls that existing job to completion instead of triggering a duplicate. retry_sync_connection's "fresh retry" is therefore only guaranteed to be a genuinely new sync if no sync was already active; see onActiveSync: FAIL in the plugin's own documented example if your use case requires rejecting that adoption behavior instead.
• allowFailure: true is what lets this flow continue past a failed sync instead of aborting the execution. A base Task property, not specific to this plugin; without it, evaluate_sync_health would never run.
• auto_remediate and dry_run are independent gates, not a single boolean. Both must be "true"/"false" respectively for retry_sync_connection to run; setting only one leaves the other still blocking.
• Switch case keys "true"/"false" are quoted strings. auto_remediate renders as the literal string "true" or "false"; unquoted true:/false: YAML map keys would parse as booleans instead and would not match.
• The exact Airbyte OSS self-hosted install command was not exercised in this environment - current Airbyte OSS installs typically go through abctl, whose exact Docker invocation varies by release; verify against Airbyte's own current quickstart rather than treating the Prerequisites command above as proven.

Links

• plugin-airbyte documentation
• plugin-airbyte Sync task source
• Kestra secrets (OSS vs Enterprise)