PluginsKubernetes (EE)Task RunnersRunnerKubernetes

Kubernetes​Kubernetes

Kubernetes Kubernetes

yaml
type: "io.kestra.plugin.ee.kubernetes.runner.Kubernetes"

Task runner that executes a task inside a pod in a Kubernetes cluster.

This task runner is container-based so the containerImage property must be set to be able to use it.

To access the task's working directory, use the {{ workingDir }} Pebble expression or the WORKING_DIR environment variable. Input files and namespace files will be available in this directory.

To generate output files you can either use the outputFiles task's property and create a file with the same name in the task's working directory, or create any file in the output directory which can be accessed by the {{ outputDir }} Pebble expression or the OUTPUT_DIR environment variables.

Note that when the Kestra Worker running this task is terminated, the pod will still runs until completion, then after restarting, the Worker will resume processing on the existing pod unless resume is set to false.

If your cluster is configure with RBAC, you need to configure the service account running your pod need to have the following authorizations:

  • pods: get, create, delete, watch, list
  • pods/log: get, watch As an example, here is a role that grant those authorizations:
yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: task-runner
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "create", "delete", "watch", "list"]
- apiGroups: [""]
  resources: ["pods/logs"]
  verbs: ["get", "watch"]

Examples

Execute a Shell command.

yaml
id: new-shell
namespace: company.team

tasks:
  - id: shell
    type: io.kestra.plugin.scripts.shell.Commands
    taskRunner:
      type: io.kestra.plugin.ee.kubernetes.runner.Kubernetes
    commands:
      - echo "Hello World"

Pass input files to the task, execute a Shell command, then retrieve output files.

yaml
id: new-shell-with-file
namespace: company.team

inputs:
  - id: file
    type: FILE

tasks:
  - id: shell
    type: io.kestra.plugin.scripts.shell.Commands
    inputFiles:
      data.txt: "{{ inputs.file }}"
    outputFiles:
      - out.txt
    containerImage: centos
    taskRunner:
      type: io.kestra.plugin.ee.kubernetes.runner.Kubernetes
    commands:
      - cp {{ workingDir }}/data.txt {{ workingDir }}/out.txt

Properties

delete

  • Type: boolean
  • Dynamic: ❌
  • Required: ✔️
  • Default: true

Whether the pod should be deleted upon completion.

namespace

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️
  • Default: default

The namespace where the pod will be created.

pullPolicy

  • Type: string
  • Dynamic: ❌
  • Required: ✔️
  • Default: ALWAYS
  • Possible Values:
    • IF_NOT_PRESENT
    • ALWAYS
    • NEVER

The image pull policy for a container image and the tag of the image, which affect when Docker attempts to pull (download) the specified image.

resume

  • Type: boolean
  • Dynamic: ❌
  • Required: ✔️
  • Default: true

Whether to reconnect to the current pod if it already exists.

waitForLogs

  • Type: string
  • Dynamic: ❌
  • Required: ✔️
  • Default: 1.000000000
  • Format: duration

The additional duration to wait for logs to arrive after pod completion.

As logs are not retrieved in real time, we cannot guarantee that we have fetched all logs when the pod complete, therefore we wait for a fixed amount of time to fetch late logs.

waitUntilCompletion

  • Type: string
  • Dynamic: ❌
  • Required: ✔️
  • Default: 3600.000000000
  • Format: duration

The maximum duration to wait for the pod completion unless the task timeout property is set which will take precedence over this property.

waitUntilRunning

  • Type: string
  • Dynamic: ❌
  • Required: ✔️
  • Default: 600.000000000
  • Format: duration

The maximum duration to wait until the pod is created.

This timeout is the maximum time that Kubernetes scheduler can take to

  • schedule the pod
  • pull the pod image
  • and start the pod.

config

The configuration of the target Kubernetes cluster.

fileSidecar

  • Type: SideCar
  • Dynamic: ❌
  • Required: ❌
  • Default: {image=busybox}

The configuration of the file sidecar container that handle download and upload of files.

labels

  • Type: object
  • SubType: string
  • Dynamic: ✔️
  • Required: ❌

The pod custom labels

Kestra will add default labels to the pod with execution and flow identifiers.

nodeSelector

  • Type: object
  • SubType: string
  • Dynamic: ✔️
  • Required: ❌

Node selector for pod scheduling

Kestra will assign the pod to the nodes you want (see https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes/)

resources

The pod custom resources

serviceAccountName

  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The name of the service account.

Definitions

io.kestra.plugin.ee.kubernetes.models.OAuthTokenProvider

Properties

output
  • Type: string
  • Dynamic: ❓
  • Required: ❌
task
  • Type: Task
  • Dynamic: ❓
  • Required: ❌

io.kestra.core.models.tasks.retrys.Constant

Properties

interval
  • Type: string
  • Dynamic: ❓
  • Required: ✔️
  • Format: duration
type
  • Type: string
  • Dynamic: ❓
  • Required: ✔️
  • Default: constant
behavior
  • Type: string
  • Dynamic: ❓
  • Required: ❌
  • Default: RETRY_FAILED_TASK
  • Possible Values:
    • RETRY_FAILED_TASK
    • CREATE_NEW_EXECUTION
maxAttempt
  • Type: integer
  • Dynamic: ❓
  • Required: ❌
  • Minimum: >= 1
maxDuration
  • Type: string
  • Dynamic: ❓
  • Required: ❌
  • Format: duration
warningOnRetry
  • Type: boolean
  • Dynamic: ❓
  • Required: ❌
  • Default: false

io.kestra.plugin.ee.kubernetes.runner.SideCar

Properties

image
  • Type: string
  • Dynamic: ✔️
  • Required: ❌
  • Default: busybox

The image used for the file sidecar container.

io.kestra.core.models.tasks.retrys.Random

Properties

maxInterval
  • Type: string
  • Dynamic: ❓
  • Required: ✔️
  • Format: duration
minInterval
  • Type: string
  • Dynamic: ❓
  • Required: ✔️
  • Format: duration
type
  • Type: string
  • Dynamic: ❓
  • Required: ✔️
  • Default: random
behavior
  • Type: string
  • Dynamic: ❓
  • Required: ❌
  • Default: RETRY_FAILED_TASK
  • Possible Values:
    • RETRY_FAILED_TASK
    • CREATE_NEW_EXECUTION
maxAttempt
  • Type: integer
  • Dynamic: ❓
  • Required: ❌
  • Minimum: >= 1
maxDuration
  • Type: string
  • Dynamic: ❓
  • Required: ❌
  • Format: duration
warningOnRetry
  • Type: boolean
  • Dynamic: ❓
  • Required: ❌
  • Default: false

io.kestra.plugin.ee.kubernetes.runner.Kubernetes-Resource

Properties

cpu
  • Type: string
  • Dynamic: ❓
  • Required: ❌
memory
  • Type: string
  • Dynamic: ❓
  • Required: ❌

io.kestra.plugin.ee.kubernetes.runner.Kubernetes-Resources

Properties

limit
request

io.kestra.core.models.tasks.retrys.Exponential

Properties

interval
  • Type: string
  • Dynamic: ❓
  • Required: ✔️
  • Format: duration
maxInterval
  • Type: string
  • Dynamic: ❓
  • Required: ✔️
  • Format: duration
type
  • Type: string
  • Dynamic: ❓
  • Required: ✔️
  • Default: exponential
behavior
  • Type: string
  • Dynamic: ❓
  • Required: ❌
  • Default: RETRY_FAILED_TASK
  • Possible Values:
    • RETRY_FAILED_TASK
    • CREATE_NEW_EXECUTION
delayFactor
  • Type: number
  • Dynamic: ❓
  • Required: ❌
maxAttempt
  • Type: integer
  • Dynamic: ❓
  • Required: ❌
  • Minimum: >= 1
maxDuration
  • Type: string
  • Dynamic: ❓
  • Required: ❌
  • Format: duration
warningOnRetry
  • Type: boolean
  • Dynamic: ❓
  • Required: ❌
  • Default: false

io.kestra.plugin.ee.kubernetes.models.Connection

Properties

apiVersion
  • Type: string
  • Dynamic: ✔️
  • Required: ❌
  • Default: v1

The API version

caCertData
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

CA certificate as data

caCertFile
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

CA certificate as file path

clientCertData
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Client certificate as data

clientCertFile
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Client certificate as a file path

clientKeyAlgo
  • Type: string
  • Dynamic: ✔️
  • Required: ❌
  • Default: RSA

Client key encryption algorithm

default is RSA

clientKeyData
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Client key as data

clientKeyFile
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Client key as a file path

clientKeyPassphrase
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Client key passphrase

disableHostnameVerification
  • Type: boolean
  • Dynamic: ❓
  • Required: ❌

Disable hostname verification

keyStoreFile
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Key store file

keyStorePassphrase
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Key store passphrase

masterUrl
  • Type: string
  • Dynamic: ✔️
  • Required: ❌
  • Default: https://kubernetes.default.svc

The url to the Kubernetes API

namespace
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

The namespace used

oauthToken
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Oauth token

oauthTokenProvider

Oauth token provider

password
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Password

trustCerts
  • Type: boolean
  • Dynamic: ❓
  • Required: ❌

Trust all certificates

trustStoreFile
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Truststore file

trustStorePassphrase
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Truststore passphrase

username
  • Type: string
  • Dynamic: ✔️
  • Required: ❌

Username

io.kestra.core.models.tasks.WorkerGroup

Properties

fallback
  • Type: string
  • Dynamic: ❓
  • Required: ❌
  • Possible Values:
    • FAIL
    • WAIT
    • CANCEL
key
  • Type: string
  • Dynamic: ❓
  • Required: ❌
  • Validation regExp: ^[a-zA-Z0-9][a-zA-Z0-9_-]*

Was this page helpful?