Blueprints

Pass data between Python script tasks and Shell tasks using Outputs

Source

yaml
id: pass-data-between-tasks
namespace: company.team

tasks:
  - id: pass_output
    type: io.kestra.plugin.core.debug.Return
    format: hello

  - id: py_outputs
    type: io.kestra.plugin.scripts.python.Script
    taskRunner:
      type: io.kestra.plugin.scripts.runner.docker.Docker
    containerImage: ghcr.io/kestra-io/pydata:latest
    outputFiles:
      - myoutput.json
    script: |
      import json
      from kestra import Kestra

      my_kv_pair = {'mykey': 'from Kestra'}
      Kestra.outputs(my_kv_pair)

      with open('myoutput.json', 'w') as f:
          json.dump(my_kv_pair, f)

  - id: take_inputs
    type: io.kestra.plugin.core.log.Log
    message: >
      data from previous tasks: {{ outputs.pass_output.value }} and {{
      outputs.py_outputs.vars.mykey }}

  - id: check_output_file
    type: io.kestra.plugin.scripts.shell.Commands
    taskRunner:
      type: io.kestra.plugin.core.runner.Process
    commands:
      - cat {{ outputs.py_outputs.outputFiles['myoutput.json'] }}

About this blueprint

Python Outputs

This flow shows how to pass data between tasks using outputs. The first two tasks return some outputs and the next 2 tasks read those values for further processing. Check the "Outputs" section in each task documentation to see what outputs it returns, and check the "Outputs" tab on the Execution page to validate what outputs are generated by this flow.

  • In case of the Return task, it returns a value under the value key. - All script tasks, including Python, return a map of outputs under the vars key. To access outputs in the downstream tasks, use the format {{ outputs.task_name.vars.key_name }}. Additionally, script tasks can return files as shown with the myoutput.json file.

Return

Script

Docker

Log

Commands

Process

More Related Blueprints

New to Kestra?

Use blueprints to kickstart your first workflows.

Get started with Kestra