Log icon
Command icon
If icon
SlackIncomingWebhook icon
Pause icon
OutputValues icon

VM Patching over SSH with Approval and Automatic Rollback

Apply apt patches to VMs over SSH with Kestra. A human validates the patch, and failures or rejections trigger automatic rollback to the prior version.

Categories
Infrastructure

Patching without a rollback plan is gambling. This blueprint applies a specific package upgrade to a VM over SSH, notifies the team, and pauses so a human can validate the patched system. If the install fails, or the validator rejects it, the flow parses the previous version out of the patch identifier and reinstalls it automatically. Structured outputs (actionPerformed, actionState, stateReason) make the flow composable: a parent orchestrator can patch UAT first and only proceed to production when this flow reports a validated success.

How it works

  1. The takeSnapshot task is a placeholder for your hypervisor or cloud snapshot call, the coarse-grained safety net before any change.
  2. The patchVM task (io.kestra.plugin.fs.ssh.Command) runs apt update and installs the package parsed from patchId with the Pebble expression {{ (inputs.patchId | split('/'))[0] }}.
  3. On a zero exit code, checkSuccess (io.kestra.plugin.core.flow.If) notifies Slack and the waitForApproval task (io.kestra.plugin.core.flow.Pause) suspends the run with onResume inputs approvePatch (BOOL) and reason (STRING).
  4. The rollBackIfRequired If triggers when the install failed or approval was denied; rollBackVM reinstalls the previous version extracted from the patch string: apt install -y pkg={{ ((inputs.patchId | split('from: '))[1] | split(']'))[0] }}.
  5. patchState and rollBackState (io.kestra.plugin.core.output.OutputValues) normalize the result, and the flow outputs expose the final action, state, and reason for parent flows.

What you get

  • A patch-validate-rollback loop with an explicit human decision recorded per run.
  • Automatic version pinning for rollback, derived from the apt upgradable string itself.
  • Structured outputs designed for staged UAT-then-prod orchestration.
  • Slack notifications at the exact moment validation is needed.

Who it's for

  • IT operations teams patching Linux fleets that lack a full patch management platform.
  • SREs who want every patch attempt logged with its outcome and rollback status.
  • Teams building staged patch pipelines (UAT gate before production).

Why orchestrate this with Kestra

A patching script cannot wait a day for someone to validate the application, and it forgets the previous version the moment it upgrades. Kestra's Pause with onResume inputs captures the human verdict and reason durably, expressions parse rollback targets from the patch metadata, conditional branches make rollback deterministic rather than a manual scramble, and flow outputs let a parent orchestrate multi-stage rollouts.

Prerequisites

  • Debian or Ubuntu targets reachable over SSH from the Kestra worker (adapt the apt commands for other distributions).
  • An SSH account permitted to install packages.
  • A Slack incoming webhook for validation notifications.

Secrets

  • SSH_VM_USER: SSH username for the target VM.
  • SSH_VM_PASSWORD: password for the SSH account (or switch to authMethod: PUBLIC_KEY with a private key secret).
  • SLACK_WEBHOOK_URL: Slack incoming webhook for approval notifications.

Quick start

  1. Add the secrets above to your Kestra namespace.
  2. Run apt list --upgradable on the target to pick a real patchId string.
  3. Execute the flow, validate the patched service, then resume with approvePatch: true or false.
  4. Check the flow outputs to confirm PATCH or ROLLBACK and the recorded reason.

How to extend

  • Replace the snapshot stub with a real call (GCP disk snapshot, AWS AMI, Proxmox, or vSphere snapshot task).
  • Drive it from the companion orchestrator blueprint to stage UAT then production automatically.
  • Add a post-patch health check (curl a health endpoint, systemctl is-active) before requesting approval.
  • Switch to PUBLIC_KEY auth and pull the patch list dynamically from the KV store.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.