PluginsAzureTasksBatchJob

Create​Create

yaml
type: "io.kestra.plugin.azure.batch.job.Create"

Create a Azure Batch job with tasks.

Examples

yaml
id: azure_batch_job_create
namespace: company.team

tasks:
  - id: create
    type: io.kestra.plugin.azure.batch.job.Create
    endpoint: https://***.francecentral.batch.azure.com
    account: <batch-account>
    accessKey: <access-key>
    poolId: <pool-id>
    job:
      id: <job-name>
    tasks:
      - id: env
        commands:
          - 'echo t1=$ENV_STRING'
        environments:
          ENV_STRING: "{{ inputs.first }}"

      - id: echo
        commands:
          - 'echo t2={{ inputs.second }} 1>&2'

      - id: for
        commands:
          -  'for i in $(seq 10); do echo t3=$i; done'

      - id: vars
        commands:
          - echo '::{"outputs":{"extract":"'$(cat files/in/in.txt)'"}::'
        resourceFiles:
          - httpUrl: https://unittestkt.blob.core.windows.net/tasks/***?sv=***&se=***&sr=***&sp=***&sig=***
          filePath: files/in/in.txt

      - id: output
        commands:
          - 'mkdir -p outs/child/sub'
          - 'echo 1 > outs/1.txt'
          - 'echo 2 > outs/child/2.txt'
          - 'echo 3 > outs/child/sub/3.txt'
        outputFiles:
          - outs/1.txt
        outputDirs:
          - outs/child

Use a container to start the task, the pool must use a microsoft-azure-batch publisher.

yaml
id: azure_batch_job_create
namespace: company.team

tasks:
  - id: create
    type: io.kestra.plugin.azure.batch.job.Create
    endpoint: https://***.francecentral.batch.azure.com
    account: <batch-account>
    accessKey: <access-key>
    poolId: <pool-id>
    job:
      id: <job-name>
    tasks:
      - id: echo
        commands:
          - 'python --version'
        containerSettings:
          imageName: python

Properties

delete

  • Type: boolean
  • Dynamic: ❌
  • Required: ✔️
  • Default: true

Whether the job should be deleted upon completion.

endpoint

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️

The blob service endpoint.

job

  • Type: Job
  • Dynamic: ✔️
  • Required: ✔️

The job to create.

poolId

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️

The ID of the pool.

resume

  • Type: boolean
  • Dynamic: ❌
  • Required: ✔️
  • Default: true

Whether to reconnect to the current job if it already exists.

tasks

  • Type: array
  • SubType: Task
  • Dynamic: ❌
  • Required: ✔️

The list of tasks to be run.

accessKey

  • Type: string
  • Dynamic: ❌
  • Required: ❌

account

  • Type: string
  • Dynamic: ❌
  • Required: ❌

completionCheckInterval

  • Type: string
  • Dynamic: ❌
  • Required: ❌
  • Default: 1
  • Format: duration

The frequency with which the task checks whether the job is completed.

maxDuration

  • Type: string
  • Dynamic: ❌
  • Required: ❌
  • Format: duration

The maximum total wait duration.

If null, there is no timeout and the task is delegated to Azure Batch.

Outputs

outputFiles

  • Type: object
  • SubType: string
  • Required: ❌

vars

  • Type: object
  • Required: ❌

Definitions

io.kestra.plugin.azure.batch.models.OutputFileBlobContainerDestination

  • containerUrl
    • Type: string
    • Dynamic: ❌
    • Required: ✔️
  • identityReference
  • path
    • Type: string
    • Dynamic: ✔️
    • Required: ❌

io.kestra.plugin.azure.batch.models.ContainerRegistry

  • identityReference
  • password
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • registryServer
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • userName
    • Type: string
    • Dynamic: ✔️
    • Required: ❌

io.kestra.plugin.azure.batch.models.OutputFileUploadOptions

  • uploadCondition
    • Type: string
    • Dynamic: ❌
    • Required: ✔️
    • Default: taskcompletion
    • Possible Values:
      • TASK_SUCCESS
      • TASK_FAILURE
      • TASK_COMPLETION

io.kestra.plugin.azure.batch.models.ComputeNodeIdentityReference

  • resourceId
    • Type: string
    • Dynamic: ✔️
    • Required: ❌

io.kestra.plugin.azure.batch.models.ResourceFile

  • autoStorageContainerName
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • blobPrefix
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • fileMode
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • filePath
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • httpUrl
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • identityReference
  • storageContainerUrl
    • Type: string
    • Dynamic: ✔️
    • Required: ❌

io.kestra.plugin.azure.batch.models.TaskContainerSettings

  • imageName
    • Type: string
    • Dynamic: ✔️
    • Required: ✔️
  • containerRunOptions
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
  • registry
  • workingDirectory
    • Type: string
    • Dynamic: ❌
    • Required: ❌
    • Possible Values:
      • TASK_WORKING_DIRECTORY
      • CONTAINER_IMAGE_DEFAULT

io.kestra.plugin.azure.batch.models.Task

  • commands
    • Type: array
    • SubType: string
    • Dynamic: ✔️
    • Required: ✔️
  • id
    • Type: string
    • Dynamic: ✔️
    • Required: ✔️
    • Max length: 64
  • interpreter
    • Type: string
    • Dynamic: ❌
    • Required: ✔️
    • Default: /bin/sh
    • Min length: 1
  • constraints
  • containerSettings
  • displayName
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
    • Max length: 1024
  • environments
    • Type: object
    • SubType: string
    • Dynamic: ✔️
    • Required: ❌
  • interpreterArgs
    • Type: array
    • SubType: string
    • Dynamic: ❌
    • Required: ❌
    • Default: [ "-c" ]
  • outputDirs
    • Type: array
    • SubType: string
    • Dynamic: ❌
    • Required: ❌
  • outputFiles
    • Type: array
    • SubType: string
    • Dynamic: ❌
    • Required: ❌
  • requiredSlots
    • Type: integer
    • Dynamic: ❌
    • Required: ❌
  • resourceFiles
    • Type: array
    • SubType: ResourceFile
    • Dynamic: ✔️
    • Required: ❌
  • uploadFiles
    • Type: array
    • SubType: OutputFile
    • Dynamic: ✔️
    • Required: ❌

io.kestra.plugin.azure.batch.models.OutputFile

  • destination
  • uploadOptions
  • filePattern
    • Type: string
    • Dynamic: ✔️
    • Required: ❌

io.kestra.plugin.azure.batch.models.Job

  • id
    • Type: string
    • Dynamic: ✔️
    • Required: ✔️
    • Max length: 64
  • displayName
    • Type: string
    • Dynamic: ✔️
    • Required: ❌
    • Max length: 1024
  • labels
    • Type: object
    • SubType: string
    • Dynamic: ✔️
    • Required: ❌
  • maxParallelTasks
    • Type: integer
    • Dynamic: ❌
    • Required: ❌
  • priority
    • Type: integer
    • Dynamic: ❌
    • Required: ❌

io.kestra.plugin.azure.batch.models.TaskConstraints

  • maxTaskRetryCount
    • Type: integer
    • Dynamic: ❌
    • Required: ❌
  • maxWallClockTime
    • Type: string
    • Dynamic: ❌
    • Required: ❌
    • Format: duration
  • retentionTime
    • Type: string
    • Dynamic: ❌
    • Required: ❌
    • Format: duration

io.kestra.plugin.azure.batch.models.OutputFileDestination