Run Multiple Python Versions Side by Side in One Flow

Upgrading a language runtime is scary when every job shares one interpreter: bump Python for one pipeline and you may break ten others. This Kestra blueprint shows why that constraint disappears with containerized task isolation. The same script runs twice in one flow, once on python:3.9-slim and once on python:3.13-slim, each task in its own container. You can evaluate a new version next to the old one, in the same execution, without touching anything already running in production.

How it works

1. The run_on_python_3_9 task (io.kestra.plugin.scripts.python.Script) executes a small script that prints sys.version, with containerImage: python:3.9-slim pinning the runtime.
2. The run_on_python_3_13 task runs the identical script with containerImage: python:3.13-slim.
3. The confirm_isolation task (io.kestra.plugin.core.log.Log) points you at the two log outputs, where the different interpreter versions are visible side by side.

Each script task gets a fresh container from its pinned image, so dependencies, interpreter version, and OS packages are scoped to the task, not the worker, not the flow, not the instance.

What you get

• Two Python versions executing in one flow with zero cross-contamination.
• A safe evaluation lane: trial Python 3.13 (or any image) next to the version production depends on.
• Per-task pinning via containerImage, so upgrades are one line per task and roll back just as easily.
• A pattern that generalizes to any containerized tool: Node versions, JDKs, dbt versions, CLI releases.

Who it's for

• Data platform teams planning language or dependency upgrades across many flows.
• Engineers maintaining legacy scripts stuck on old runtimes alongside modern code.
• Solutions engineers demonstrating Kestra's extensibility and isolation story.

Why orchestrate this with Kestra

On a shared cron box or a monolithic scheduler worker, the interpreter is global state: one pyenv slip breaks unrelated jobs, and evaluating a new version means standing up a second environment. In Kestra, the runtime is a task property. Declare containerImage and the worker spins up exactly that environment for exactly that task, with logs and outputs captured per run. Kestra Enterprise extends the same idea to plugins themselves with Versioned Plugins, running two versions of the same plugin side by side (for example Ansible 2.16 in production while 2.19 is under evaluation), and custom plugin development covers connectors that do not exist yet.

Prerequisites

• A Kestra instance whose worker can run Docker (the default Docker task runner).
• Network access to pull the python:3.9-slim and python:3.13-slim images.

Secrets

No secrets are required. If you swap in private registry images, configure registry credentials on the Docker task runner using {{ secret('REGISTRY_USERNAME') }} and {{ secret('REGISTRY_PASSWORD') }}.

Quick start

1. Import this flow into the company.team namespace.
2. Execute it; the first run pulls both images, so allow a moment.
3. Open the logs of run_on_python_3_9 and run_on_python_3_13 and compare the printed sys.version values.
4. Change one containerImage to another tag (for example python:3.14-rc-slim) and rerun to test a future release.

How to extend

• Replace the demo script with your real workload and add dependencies per task to test library compatibility across versions.
• Run the two tasks inside io.kestra.plugin.core.flow.Parallel to compare wall-clock behavior concurrently.
• Capture structured results with Kestra outputs and diff them downstream to detect behavior changes between versions.
• On Kestra Enterprise, use Versioned Plugins to run two versions of the same plugin side by side across namespaces.

Links

• Versioned Plugins (Enterprise): https://kestra.io/docs/enterprise/instance/versioned-plugins
• Build a custom plugin: https://kestra.io/docs/how-to-guides/custom-plugin
• Python script tasks: https://kestra.io/plugins/plugin-script-python