Schedule icon
SkopeoCLI icon
If icon
Switch icon
Log icon
SlackIncomingWebhook icon

Skopeo Container Registry Sync Gate & Drift Reconciliation

Compare container image digests across registries with Skopeo, conditionally sync on drift, and verify post-sync alignment.

Categories
DataInfrastructure

Two registries holding "the same" image tag can silently drift apart - a tag gets repushed, a mirror job fails partway, or a promotion step is skipped. There is no built-in alert for that drift; you find out when a deploy pulls the wrong bits. This blueprint runs skopeo inspect against both a source and a destination image, compares their manifest digests, and only copies the source over the destination when drift is found and auto_sync is explicitly "true" - itself still gated by dry_run, so a schedule can safely audit without ever mutating a registry on its own.

How it works

  1. weekly_sync_check (io.kestra.plugin.core.trigger.Schedule) runs weekly, always forcing auto_sync: "false" and dry_run: "true" through its own inputs: override regardless of this flow's defaults. Shipped disabled so you can validate a manual run first.
  2. inspect_registry_images (io.kestra.plugin.skopeo.cli.SkopeoCLI) runs skopeo inspect docker://... against both images, redirecting each result to its own file (source_inspect.json, dest_inspect.json). A missing destination image (first sync) is caught with a shell fallback that writes a NOT_FOUND placeholder instead of failing the task.
  3. evaluate_drift (io.kestra.plugin.core.flow.If) extracts each file's .Digest field with jq and compares them.
    • then (drift found): route_sync_decision (io.kestra.plugin.core.flow.Switch) branches on auto_sync. "false" only logs that drift exists and nothing ran. "true" enters check_dry_run (io.kestra.plugin.core.flow.If) on dry_run: "false" runs sync_images (skopeo copy), then verify_post_sync re-inspects the destination and log_sync_verified confirms whether the digests now match; "true" only logs what would have been copied.
    • else (no drift): log_no_drift records a clean run.
  4. notify_slack posts one summary message every run, and the errors block alerts separately if the flow itself fails outside these handled branches.

What you get

  • A registry-to-registry digest comparison using Skopeo's own inspect, not a guess from tags or timestamps.
  • Two independent gates (auto_sync, dry_run) between "drift detected" and "image actually copied".
  • A post-sync re-inspection that confirms the destination's digest actually changed to match the source, not just that skopeo copy exited zero.
  • One Slack message per run covering all outcomes: clean, drift-logged-only, dry-run audit, and synced-and-verified.

Who it's for

  • Platform teams running their own registry mirrors or promotion pipelines who need to know when a mirror silently falls behind.
  • Release engineers who want a reviewable gate before any automated cross-registry copy.
  • Anyone who has shipped a stale image because "the mirror job usually works".

Why orchestrate this with Kestra

Skopeo has no built-in scheduler, no diffing memory between runs, and no branching logic of its own - it runs the command you give it and exits. Kestra supplies the schedule, the digest comparison as a first-class branch, two independent authorization gates before anything is copied, and an execution history that shows exactly when drift was found, whether it was synced, and whether the sync was verified.

Prerequisites

  • Docker available to the Kestra Worker (SkopeoCLI defaults to the Docker task runner with the quay.io/skopeo/stable image).
  • Network access from the Kestra Worker to both the source and destination registries.
  • For a private destination registry, credentials passed via Skopeo's own CLI flags (--dest-creds, --creds) added to the commands in sync_images - Skopeo has no separate connection/credentials task property; see Pitfalls.
  • A Slack incoming webhook for notifications.

Local testing with a throwaway registry: docker run -d --name registry -p 5001:5000 registry:2 docker network connect YOUR_KESTRA_NETWORK registry (only needed if the Kestra Worker runs in a separate Docker network than this container).

Secrets

  • SLACK_WEBHOOK_URL: Slack incoming webhook used by notify_slack and the errors block.
  • This blueprint's default source_image/dest_image need no registry credentials. If you add --dest-creds/--creds flags for a private registry, use {{ secret('REGISTRY_USER') }}:{{ secret('REGISTRY_PASSWORD') }} as shown in the official plugin examples - never place credentials in source_image/dest_image or any other input.
  • In Kestra OSS (no Enterprise secrets backend), secrets are supplied as environment variables prefixed SECRET_, base64-encoded, and read back in flows with {{ secret('NAME') }} - for example SECRET_SLACK_WEBHOOK_URL=$(echo -n 'https://hooks.slack.com/...' | base64). This keeps credentials out of the flow YAML but, per Kestra's own documentation, offers no encryption at rest or access control beyond the host environment; use the Enterprise secrets backend for stronger guarantees.

Inputs

  • source_image (STRING, default registry.hub.docker.com/library/alpine:latest): source image reference.
  • dest_image (STRING, default localhost:5001/library/alpine:latest): destination image reference.
  • auto_sync (SELECT: "false", "true"; default "false"): must be "true" for the sync branch to be entered at all.
  • dry_run (SELECT: "true", "false"; default "true"): must be explicitly "false", together with auto_sync: "true", for skopeo copy to actually run.

Outputs

  • outputs.inspect_registry_images.outputFiles['source_inspect.json'] / ['dest_inspect.json']: raw skopeo inspect JSON for each image (destination is a {"Digest":"NOT_FOUND"} placeholder if it does not exist yet).
  • outputs.sync_images: present only when the copy branch actually ran.
  • outputs.verify_post_sync.outputFiles['post_sync_inspect.json']: the destination's re-inspected manifest after the copy, present only when a sync ran.

Quick start

  1. Start a local registry with the command above and push a test image to it (or use the defaults, which compare Docker Hub's alpine:latest against a non-existent local image - a guaranteed first-run drift).
  2. Add the SLACK_WEBHOOK_URL secret.
  3. Run the flow manually with the defaults (auto_sync: "false") and confirm log_sync_disabled fires and nothing is copied.
  4. Run once with auto_sync: "true" and dry_run: "true" and confirm log_dry_run_sync describes the copy without making it.
  5. Run once with auto_sync: "true" and dry_run: "false" against a test destination and confirm log_sync_verified reports PASSED.
  6. Enable weekly_sync_check once you trust the audit step; it always runs in the safe auto_sync: "false" / dry_run: "true" mode regardless of what you leave the flow's own defaults set to.

How to extend

  • Loop over a list of image pairs with io.kestra.plugin.core.flow.Loop to audit an entire mirror set in one run.
  • Add a Pause approval task between evaluate_drift and the sync branch for a human-reviewed gate instead of (or alongside) the static auto_sync input.
  • Compare more than the top-level Digest - skopeo inspect's JSON also includes RepoTags, Created, and Labels, useful for detecting a retag without a digest change.
  • Add retry policy to sync_images for transient registry errors instead of letting the whole execution fail into the errors block.

Pitfalls

  • io.kestra.plugin.skopeo.cli.SkopeoCLI has no source/destination/credentials/tlsVerify property. It is a single generic CLI-wrapper task (commands, containerImage, outputFiles) - Skopeo has no separate connection class at all; authentication is passed as raw --src-creds/--dest-creds/--creds flags inside commands, per the plugin's own documentation.
  • There is no digest output field. SkopeoCLI returns a core ScriptOutput (exitCode, outputFiles, vars, log line counts) - this blueprint gets the digest by redirecting skopeo inspect's JSON to a file, declaring it in outputFiles, and extracting .Digest with read(...) | jq(...), exactly as the plugin's own sanity-check test does for a different field.
  • A missing destination image is not a drift-detection failure. The first run against a destination that does not exist yet would make skopeo inspect fail the whole task; the fallback || echo '{"Digest":"NOT_FOUND"}' turns that into a normal "drift found" result (since NOT_FOUND never equals a real digest) instead of an errors-block alert.
  • auto_sync and dry_run are SELECT strings, not booleans. Comparisons use == "false" / == "true" string equality, and Switch case keys are quoted ("true":, "false":) to stop YAML from parsing them as boolean map keys, which would silently fail to match the string value Switch actually receives.
  • The Schedule trigger's own inputs: override is the real safety net, not this flow's defaults. weekly_sync_check always forces auto_sync: "false" / dry_run: "true" on every scheduled fire; a manual execution is the only way to pass different values and trigger an actual sync.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.