Add a parametrized Python script as a Namespace File and run it in

Run a single parametrized Python script across many parameter values at once, each execution isolated in its own Docker container. This blueprint shows how to fan out a CPU bound or IO bound Python workload (parameter sweeps, batch processing, per item scoring) so that nine values run concurrently instead of one after another, cutting total runtime without writing any multiprocessing or queue code yourself.

How it works

• A single io.kestra.plugin.core.flow.ForEach task iterates over the values list (1 through 9) with concurrencyLimit: 0, meaning every value is dispatched in parallel with no cap.
• For each value, a child io.kestra.plugin.scripts.python.Commands task runs the Python script. The script parametrized.py is supplied inline through the inputFiles property.
• The script uses argparse to read a --num argument, doubles it, and prints the result.
• The current iteration value is injected on the command line with the {{ taskrun.value }} Pebble expression: python parametrized.py --num {{ taskrun.value }}.
• Each child task uses the io.kestra.plugin.scripts.runner.docker.Docker task runner, so every parallel run executes in a clean, isolated container.

What you get

• Concurrent execution of one script across many inputs in a single flow.
• Full container isolation per run, so dependencies and state never leak between iterations.
• Per iteration logs, status, and outputs visible in the Kestra UI.
• A reusable pattern for parameter sweeps and batch fan out.

Who it's for

• Data engineers running parameter sweeps or batch transforms.
• ML practitioners scoring or training across many inputs.
• Python developers who want parallelism without writing concurrency code.

Why orchestrate this with Kestra

A bare Python script or a for loop runs items sequentially and gives you no retries, no isolation, and no visibility. Kestra turns the same logic into a declarative YAML flow: ForEach parallelizes the work, the Docker task runner isolates each run, and you get per task retries, logs, and lineage out of the box. You can add event triggers, schedules, and downstream tasks that the Python interpreter alone cannot provide.

Prerequisites

• A running Kestra instance.
• Docker available to the worker so the Docker task runner can pull and run containers.

Secrets

This blueprint references no secrets. Add secrets only if you extend the script to call external services.

Quick start

1. Copy this blueprint into your Kestra instance.
2. Confirm Docker is reachable from the Kestra worker.
3. Execute the flow and watch the nine iterations run in parallel.
4. Open each python task run to inspect its printed result.

How to extend

• Replace the inline parametrized.py with a Namespace File by enabling namespaceFiles instead of inputFiles.
• Swap the values list for dynamic inputs or an output from an upstream task.
• Set a positive concurrencyLimit to cap how many containers run at once.
• Add a trigger to run the fan out on a schedule or on an event.

Links

• Python script plugin docs: https://kestra.io/plugins/plugin-script-python