PluginsPythonTasks

Script​Script

yaml
type: "io.kestra.plugin.scripts.python.Script"

Execute a Python script.

Examples

Execute a Python script and generate an output.

yaml
id: python_use_input_file
namespace: company.team

tasks:
  - id: python
    type: io.kestra.plugin.scripts.python.Script
    script: |
      from kestra import Kestra
      import requests

      response = requests.get('https://kestra.io')
      print(response.status_code)

      Kestra.outputs({'status': response.status_code, 'text': response.text})
    beforeCommands:
      - pip install requests kestra

Log messages at different log levels using Kestra logger.

yaml
id: python_logs
namespace: company.team

tasks:
  - id: python_logger
    type: io.kestra.plugin.scripts.python.Script
    allowFailure: true
    warningOnStdErr: false
    beforeCommands:
      - pip install kestra
    script: |
      import time
      from kestra import Kestra

      logger = Kestra.logger()

      logger.debug("DEBUG is used for diagnostic info.")
      time.sleep(0.5)

      logger.info("INFO confirms normal operation.")
      time.sleep(0.5)

      logger.warning("WARNING signals something unexpected.")
      time.sleep(0.5)

      logger.error("ERROR indicates a serious issue.")
      time.sleep(0.5)

      logger.critical("CRITICAL means a severe failure.")

Execute a Python script with an input file from Kestra's local storage created by a previous task.

yaml
id: python_use_input_file
namespace: company.team

tasks:
  - id: python
    type: io.kestra.plugin.scripts.python.Script
    script: |
      with open('{{ outputs.previousTaskId.uri }}', 'r') as f:
        print(f.read())

Execute a Python script that outputs a file.

yaml
id: python_output_file
namespace: company.team

tasks:
  - id: python
    type: io.kestra.plugin.scripts.python.Script
    outputFiles:
      - "myfile.txt"
    script: |
      f = open("myfile.txt", "a")
      f.write("Hello from a Kestra task!")
      f.close()

If you want to generate files in your script to make them available for download and use in downstream tasks, you can leverage the outputFiles property as shown in the example above. Files will be persisted in Kestra's internal storage. The first task in this example creates a file 'clean_dataset.csv' and the next task can access it by leveraging the syntax {{outputs.yourTaskId.outputFiles['yourFileName.fileExtension']}}.

yaml
id: python_outputs
namespace: company.team

tasks:
  - id: clean_dataset
    type: io.kestra.plugin.scripts.python.Script
    containerImage: ghcr.io/kestra-io/pydata:latest
    outputFiles:
      - "clean_dataset.csv"
    script: |
      import pandas as pd
      df = pd.read_csv("https://huggingface.co/datasets/kestra/datasets/raw/main/csv/messy_dataset.csv")

      # Replace non-numeric age values with NaN
      df["Age"] = pd.to_numeric(df["Age"], errors="coerce")

      # mean imputation: fill NaN values with the mean age
      mean_age = int(df["Age"].mean())
      print(f"Filling NULL values with mean: {mean_age}")
      df["Age"] = df["Age"].fillna(mean_age)
      df.to_csv("clean_dataset.csv", index=False)

  - id: readFileFromPython
    type: io.kestra.plugin.scripts.shell.Commands
    taskRunner:
      type: io.kestra.plugin.core.runner.Process
    commands:
      - head -n 10 {{ outputs.clean_dataset.outputFiles['clean_dataset.csv'] }}

Properties

interpreter

  • Type: array
  • SubType: string
  • Dynamic: ❌
  • Required: ✔️
  • Default: [ "/bin/sh", "-c" ]
  • Min items: 1

Which interpreter to use.

script

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️
  • Min length: 1

The inline script content. This property is intended for the script file's content as a (multiline) string, not a path to a file. To run a command from a file such as bash myscript.sh or python myscript.py, use the Commands task instead.

warningOnStdErr

  • Type: boolean
  • Dynamic: ❌
  • Required: ✔️
  • Default: true

Whether to set the task state to WARNING when any stdErr output is detected.

Note that a script error will set the state to FAILED regardless.

beforeCommands

  • Type: array
  • SubType: string
  • Dynamic: ✔️
  • Required: ❌

A list of commands that will run before the commands, allowing to set up the environment e.g. pip install -r requirements.txt.

containerImage

  • Type: string
  • Dynamic: ✔️
  • Required: ❌
  • Default: ghcr.io/kestra-io/kestrapy:latest

The task runner container image, only used if the task runner is container-based.

docker

Deprecated - use the 'taskRunner' property instead.

Only used if the taskRunner property is not set

env

  • Type: object
  • SubType: string
  • Dynamic: ✔️
  • Required: ❌

Additional environment variables for the current process.

failFast

  • Type: boolean
  • Dynamic: ❌
  • Required: ❌
  • Default: true

Fail the task on the first command with a non-zero status.

If set to false all commands will be executed one after the other. The final state of task execution is determined by the last command. Note that this property maybe be ignored if a non compatible interpreter is specified. You can also disable it if your interpreter does not support the set -eoption.

inputFiles

  • Type:
    • object
    • string
  • Dynamic: ✔️
  • Required: ❌

The files to create on the local filesystem. It can be a map or a JSON object.

namespaceFiles

Inject namespace files.

Inject namespace files to this task. When enabled, it will, by default, load all namespace files into the working directory. However, you can use the include or exclude properties to limit which namespace files will be injected.

outputDirectory

  • Type: boolean
  • Dynamic: ❌
  • Required: ❌
  • Default: false

Whether to setup the output directory mechanism.

Required to use the expression. Note that it could increase the starting time. Deprecated, use the outputFiles property instead.

outputFiles

  • Type: array
  • SubType: string
  • Dynamic: ✔️
  • Required: ❌

The files from the local filesystem to send to Kestra's internal storage.

Must be a list of glob) expressions relative to the current working directory, some examples: my-dir/, my-dir/*/ or my-dir/my-file.txt.

runner

  • Type: string
  • Dynamic: ❌
  • Required: ❌
  • Possible Values:
    • PROCESS
    • DOCKER

Deprecated - use the 'taskRunner' property instead.

Only used if the taskRunner property is not set

targetOS

  • Type: string
  • Dynamic: ❌
  • Required: ❌
  • Default: AUTO
  • Possible Values:
    • LINUX
    • WINDOWS
    • AUTO

The target operating system where the script will run.

taskRunner

  • Type: TaskRunner
  • Dynamic: ❌
  • Required: ❌
  • Default: { "type": "io.kestra.plugin.scripts.runner.docker.Docker" }

The task runner to use.

Task runners are provided by plugins, each have their own properties.

Outputs

exitCode

  • Type: integer
  • Required: ✔️
  • Default: 0

outputFiles

  • Type: object
  • SubType: string
  • Required: ❌

vars

  • Type: object
  • Required: ❌

Definitions

io.kestra.core.models.tasks.NamespaceFiles

  • enabled
    • Type: boolean
    • Dynamic: ❌
    • Required: ❌
    • Default: true
  • exclude
    • Type: array
    • SubType: string
    • Dynamic: ❌
    • Required: ❌
  • include
    • Type: array
    • SubType: string
    • Dynamic: ❌
    • Required: ❌

io.kestra.plugin.scripts.runner.docker.Cpu

  • cpus
    • Type: integer
    • Dynamic: ❌
    • Required: ❌

io.kestra.core.models.tasks.runners.TaskRunner

  • type
    • Type: string
    • Dynamic: ❌
    • Required: ✔️
    • Validation RegExp: \p{javaJavaIdentifierStart}\p{javaJavaIdentifierPart}*(\.\p{javaJavaIdentifierStart}\p{javaJavaIdentifierPart}*)*
    • Min length: 1

io.kestra.plugin.scripts.runner.docker.Memory

  • kernelMemory
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • memory
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • memoryReservation
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • memorySwap
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • memorySwappiness
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • oomKillDisable
    • Type: boolean
    • Dynamic: ❌
    • Required: ❌

io.kestra.plugin.scripts.exec.scripts.models.DockerOptions

  • image
    • Type: string
    • Dynamic: ✔️
    • Required: ✔️
    • Min length: 1
  • config
    • Type:
      • string
      • object
    • Dynamic: ✔️
    • Required: ❌
  • cpu
    • Type: Cpu
    • Dynamic: ❌
    • Required: ❌
  • credentials
  • deviceRequests
  • entryPoint
    • Type: array
    • SubType: string
    • Dynamic: ✔️
    • Required: ❌
  • extraHosts
    • Type: array
    • SubType: string
    • Dynamic: ✔️
    • Required: ❌
  • host
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • memory
    • Type: Memory
    • Dynamic: ❌
    • Required: ❌
  • networkMode
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • pullPolicy
    • Type: string
    • Dynamic: ❌
    • Required: ❌
    • Default: ALWAYS
    • Possible Values:
      • IF_NOT_PRESENT
      • ALWAYS
      • NEVER
  • shmSize
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • user
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • volumes
    • Type: array
    • SubType: string
    • Dynamic: ✔️
    • Required: ❌

io.kestra.plugin.scripts.runner.docker.Credentials

  • auth
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • identityToken
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • password
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • registry
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • registryToken
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • username
    • Type: string
    • Dynamic: ✔️
    • Required: ❌

io.kestra.plugin.scripts.runner.docker.DeviceRequest

  • capabilities
    • Type: array
    • SubType: array
    • Dynamic: ❌
    • Required: ❌
  • count
    • Type: integer
    • Dynamic: ❌
    • Required: ❌
  • deviceIds
    • Type: array
    • SubType: string
    • Dynamic: ✔️
    • Required: ❌
  • driver
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • options
    • Type: object
    • SubType: string
    • Dynamic: ❌
    • Required: ❌