Script icon
Docker icon
Get icon
If icon
Log icon
Pause icon
Deploy icon
Assert icon
CreateProcessInstance icon
Set icon
PublishMessage icon
Delete icon
CancelProcessInstance icon
SlackIncomingWebhook icon
Schedule icon

Smoke test a Camunda BPMN deployment before you trust it

Gate a staged Camunda BPMN deployment with approval, a pinned definition key, unique message correlation and proof the canary instance completed.

Categories
Infrastructure

Safety

Deploying a BPMN model is a write that keeps writing. Camunda retains every version you deploy, routes new instances to the latest one and cannot delete a version through its public API. Creating an instance can start real work: a service task calls a system, a call activity starts another process, a timer fires later. A smoke test that deploys the wrong model or one that starts an instance of a model carrying real tasks leaves work running that nobody asked for and a version history nobody can clean up. That is why this flow refuses by default and why every guard below is load bearing. Each one was proven by its own run against a real Camunda 8.9.19 engine.

Guard What it stops
One exact canary_ process ID, compared with the ID inside the model deploying a model whose process ID is not the dedicated canary
Element allowlist: start, message catch, end, two flows, one message a service task, call activity, script, timer or signal reaching the engine
Shape check on the parsed XML a model that branches into work nobody reviewed
65536 byte model ceiling a large unreviewed model arriving through a smoke job input
auto_remediate false, dry_run true, Pause three separate gates before one deployment
Recheck of cluster ID, model SHA-256 and the retained version list after the Pause deploying against a plan that went stale while the approval waited
Recheck of the residual _active key after the Pause creating a canary while a previous one is still unaccounted for
50 retained version ceiling a smoke job that quietly accumulates versions forever
processDefinitionKey pinned from the deployment response driving or verifying a different version than the one just deployed
execution.id as correlation key and message ID correlating with somebody else's message or delivering twice
Completion proof: COMPLETED, no incident, the exact correlated messageKey a deployment that returned success being reported as proven
errors block acts only on the instance key this run created cleanup cancelling a canary or a production instance it does not own
concurrency: limit 1 with QUEUE two executions deploying and correlating at the same time

How it works

  1. plan (io.kestra.plugin.scripts.python.Script) parses the BPMN with xml.etree. It refuses a model over 65536 bytes, a process ID that is not the exact canary_ input, any element outside the allowlist, anything other than one plain start plus one message catch plus one end. It also refuses a start event carrying an event definition. It then reads /v2/topology for the cluster ID and /v2/process-definitions/search for the versions already retained, failing above
    1. The model SHA-256, cluster ID and version list become the approved plan.
  2. read_receipt and read_active (io.kestra.plugin.core.kv.Get) load the last proven receipt and any residual instance key left by an earlier pass that did not finish.
  3. receipt_check (Script) decides whether a test is needed. A receipt counts only if the cluster ID, model SHA-256, process ID and message name all still match and the engine still shows that exact instance COMPLETED on that exact definition with that exact correlated message. A residual _active key must point at an instance the engine reports COMPLETED or TERMINATED, on the definition it was saved with, before the run may create another.
  4. needs_test (io.kestra.plugin.core.flow.If) either logs healthy_receipt or runs report, which names the planned cluster, the model SHA-256 and every retained version.
  5. mode (If) continues only with auto_remediate: true and dry_run: false. Otherwise the run ends at dry_run as a report.
  6. approval (io.kestra.plugin.core.flow.Pause, behavior: FAIL, PT30M) waits for a person. approved (If) reads outputs.approval.onResume.approved ?? false, so a resume that answers nothing is a refusal.
  7. reread_active (kv.Get) and recheck (Script) treat the approval as a stale snapshot. recheck re-parses the model, re-reads the cluster and version list, then fails unless they are byte identical to the approved plan. It fails if the residual key changed during the Pause. It fails again if the residual instance became active.
  8. deploy (io.kestra.plugin.camunda.Deploy) deploys the single resource. deployed_scope (io.kestra.plugin.core.execution.Assert) fails unless the engine returned exactly one process whose bpmnProcessId is the canary ID.
  9. start_canary (io.kestra.plugin.camunda.CreateProcessInstance) creates one instance of the returned processDefinitionKey, never of the process ID, with the Kestra execution ID as canaryCorrelationKey. save_instance (kv.Set) records that key before anything else can fail.
  10. waiting_subscription (Script) polls /v2/message-subscriptions/search until exactly one CREATED subscription exists for the owned instance, the pinned definition, the expected message name and this execution's correlation key. publish (io.kestra.plugin.camunda.PublishMessage) then publishes once with messageId set to the execution ID, so a retry cannot deliver twice.
  11. prove_completion (Script) polls until the owned instance is COMPLETED with no incident and /v2/correlated-message-subscriptions/search returns exactly the published messageKey against this execution's correlation key. save_receipt (kv.Set) stores the proof, clear_active (kv.Delete) releases the residual key and completed logs the keys.
  12. On any failure the errors block runs cleanup_owned, inspect_owned, cancel_if_active, cancel_canary (io.kestra.plugin.camunda.CancelProcessInstance) and verify_cleanup, all keyed on the one instance key this run created, then failure_alert (io.kestra.plugin.slack.notifications.SlackIncomingWebhook) posts the failure.

A Schedule trigger ships disabled: true and forces dry_run: true with auto_remediate: false, so enabling it cannot deploy anything.

What you get

  • A deployment key, a pinned process definition key, one isolated instance key and one message key, all read back from the engine rather than taken from the deployment response.
  • Proof that the version you just deployed can actually be driven through its message catch to COMPLETED, not just that the deployment call returned 200.
  • A receipt that stops the next run deploying and correlating again for a model that is already proven.
  • A failed run that cancels the canary it created, leaves every other instance alone, keeps the residual key and lets the next run reconcile it.

Who it's for

Platform and process automation teams who deploy Camunda 8 BPMN models through a pipeline and want the message wiring checked in a dedicated staging cluster before the model reaches anything that matters.

Why orchestrate this with Kestra

Three parts of this are orchestration, not engine work. A deployment that must not happen without a person needs a durable Pause and a re-read of eligibility afterwards, because the cluster can change while the approval waits. A canary that was created but never completed needs its key remembered across runs, which is what the KV receipt and residual key do. Proving the model works needs the deployment response, the subscription, the message key and the final instance state held together in one record. Camunda does the deployment, the correlation and the state itself.

Tested end to end

Kestra 2.0.5, Camunda 8.9.19 (camunda/camunda@sha256:174f43860c96f25e259785b0d65bc245fbcb20e3d8ff39eef8ff01ae888813ed, single broker, one partition, RDBMS secondary storage), plugin-camunda 1.0.3, plugin-slack 2.1.0, plugin-script-python 1.14.0. Validate was clean: no constraints, warnings, infos or deprecations. Every state below was read from the engine's own REST v2 API on its published port, never from the flow's own output.

The fixture before the final suite, read from /v2/process-definitions/search and /v2/process-instances/search:

process definition        version  instances            note
canary_smoke              1        3 COMPLETED          earlier proven canaries
                                   1 TERMINATED         earlier cancelled canary
order_fulfilment_prod     1        1 ACTIVE             protected production control
cluster b198d5c8-cf49-4ad9-945e-1390c6f823c9, gateway 8.9.19
Case Execution Result
Healthy 5diMlUPUCo0KKZDe7mFbcf SUCCESS. Matching receipt verified against the engine. No deployment, instance or message
Dry run 6nuEVlTF4qvek3BVfwMsYw SUCCESS. Model and cluster reported, the Pause never reached, nothing created
Declined approval 6UjJGoe57REPquvKlJSvbC SUCCESS. Resume carried no approval. No deployment, instance or message
Remediation 77Jr3hEskeZy1FxncOu1VP SUCCESS. Instance 2251799813685439 COMPLETED on pinned definition 2251799813685320 after message 2251799813685448, correlation key equal to the execution ID, 1.345 s from start to completion. Completed canaries 3 to 4
Repeat 6UFTFAYaXNKMqRInenlfJL SUCCESS. Same write mode inputs, receipt matched, zero deployments or instances
Ineffective write 69sHu8YflCSPrGBQUMRabg FAILED as intended. Expected message name did not match the model, so instance 2251799813685454 got 0 correlated subscriptions. The errors block cancelled it to TERMINATED and no receipt was written
Stale plan during the Pause 2bifymnt40ssaqXsEj03aE FAILED as intended. A third party deployed canary_smoke version 2 while the approval waited. recheck refused before deploy. Completed and terminated counts unchanged
Re-plan after that version 5JNGGPxRJl4yinBR5ijmhW SUCCESS. Fresh plan over versions 1 and 2, deployed version 3, instance 2251799813685469 COMPLETED
Residual key changed during the Pause 2vuM6VVmBSpmJtGtH4zv3O FAILED as intended. The _active key was rewritten to another owner during the approval. recheck refused before deploy
Lookalike with a service task 7NBaheEjB5GKOc5bO4HDln FAILED as intended. Model declared the exact canary ID but carried serviceTask. Nothing deployed
Oversize model 61Ei4I0azpaaixoRiJLL87 FAILED as intended. 72206 bytes against the 65536 byte ceiling. Nothing deployed
Protected production process 2eJLB56UCUzEhvzckvWdFk SUCCESS. Canary completed while order_fulfilment_prod version 1 and its ACTIVE instance 2251799813685389 stayed untouched with no incident
Queued second execution 3QWSgRU58n2gl1VOmpuGCA SUCCESS. Launched while another execution held the slot, recorded QUEUED, then ran report only
Interrupted pass 3jV5AcWl2ymx3cBy8WvU80 FAILED as intended. Instance 2251799813685502 was created, never completed, cancelled to TERMINATED by the errors block. Residual key kept, no receipt
Reconciled next run 4UWSPmT21XphBk2NARdJK8 SUCCESS. Read the residual key from the interrupted execution, confirmed that instance terminal, created 2251799813685517, proved COMPLETED, cleared the key. No duplicate version

Repeated on a second fresh Camunda container with its own volume: healthy, remediation and the stale plan refusal gave the same results. Slack was a local HTTP sink.

Prerequisites

  • A dedicated Camunda 8 staging cluster with searchable secondary storage, so /v2/process-instances/search and the message subscription endpoints answer. Tested on 8.9.19 only.
  • Kestra 2.0.5 with the Camunda, Slack and Python script plugins plus a Docker task runner that can reach the cluster.
  • Minimum permissions on the cluster: create deployment, create process instance, publish message, cancel process instance, plus read access to process definitions, process instances and message subscriptions. Nothing else is used.
  • Never point this at a production process cluster. Deployed versions cannot be removed through the public API.

Secrets

  • CAMUNDA_REST_ADDRESS: base URL of the staging cluster REST API.
  • CAMUNDA_USERNAME, CAMUNDA_PASSWORD: basic authentication for that cluster. Supply both or neither.
  • SLACK_WEBHOOK_URL: Slack incoming webhook for failures.

Inputs

Input Default Purpose
camunda_rest_address secret('CAMUNDA_REST_ADDRESS') Staging cluster REST API
canary_process_id canary_smoke The exact canary_ process ID allowed in the model
bpmn_content a minimal start, message catch, end model The model to stage. At most 65536 bytes
message_name canary_smoke_continue Catch message the model declares
receipt_key camunda_canary_smoke KV receipt key. One per cluster and process
verify_seconds 30 Bound on each engine wait, 10 to 60
runner_network empty Docker network for the script runner when the cluster is private
dry_run true Report only
auto_remediate false Ask for approval before deploying

Quick start

  1. Add the four secrets. Point camunda_rest_address at a staging cluster you own.
  2. Execute with the defaults. Read report: the cluster ID, the model SHA-256 and every version already retained.
  3. Check that canary_process_id and message_name match your model, then execute with dry_run: false and auto_remediate: true.
  4. Read the Pause description. It names the cluster and the model. Approve only if both are the staging ones.
  5. Read completed: the instance key, the message key and the definition key. The receipt is now stored.
  6. Run again. It should log healthy_receipt and write nothing.
  7. Enable the Schedule for a daily read-only report once you trust the model allowlist.

How to extend

  • Point receipt_key at a new value to force a reviewed retest after a model change the SHA-256 does not catch, for example a change of cluster.
  • Widen the element allowlist in vars.reader_script only together with a plan for the side effects those elements cause plus how a failed run cancels them.
  • Pair the receipt with your pipeline so a deployment to the next environment reads the proven definition key rather than the process ID.
  • Send failure_alert to a ticket instead of Slack so every failed smoke test has an owner.

Limits: this proves one linear message-catch model on one staging cluster. It does not prove a model with tasks, gateways or timers. It is not a promotion path to production. There is exactly one write target per run, so the flow uses the flow level concurrency: limit 1 with QUEUE instead of a Loop with concurrencyLimit: 1. Retained versions are never rolled back, because Camunda offers no public delete. The residual key handles an instance whose key came back; a create that times out before the engine returns a key leaves an instance that only the execution correlation variable can identify, which is why failure_alert says so. If the engine's history is purged, a residual key whose instance the engine no longer knows blocks the next run until somebody clears it, which is deliberate. Receipt verification covers identity and completion, not model semantics. The connection fields are inputs defaulting to secret(...), so the rendered value is stored with the execution: treat execution inputs as sensitive.

Links

This blueprint was created by zkasuran.

See How

New to Kestra?

Use blueprints to kickstart your first workflows.