Azure Trigger

Azure Trigger
Certified
Trigger a flow on a new file arrival in an Azure Blob Storage container

This trigger will poll the specified Azure Blob Storage container every interval. Using the prefix and regexp properties, you can define which files' arrival will trigger the flow. Under the hood, we use the Azure Blob Storage API to list the files in a specified location and download them to the internal storage and process them with the declared action. You can use the action property to move or delete the files from the container after processing to avoid the trigger being fired again for the same files during the next polling interval.

yaml
type: io.kestra.plugin.azure.storage.blob.Trigger

Examples

Run a flow if one or more files arrived in the specified Azure Blob Storage container location. Then, process all files in a for-loop either sequentially or concurrently, depending on the concurrencyLimit property.

yaml
id: react_to_files
namespace: company.team

tasks:
  - id: each
    type: io.kestra.plugin.core.flow.Loop
    concurrencyLimit: 1
    values: "{{ trigger.blobs | jq('.[].uri') }}"
    tasks:
      - id: return
        type: io.kestra.plugin.core.debug.Return
        format: "{{ item.value }}"

triggers:
  - id: watch
    type: io.kestra.plugin.azure.storage.blob.Trigger
    interval: PT5M
    endpoint: "https://yourblob.blob.core.windows.net"
    connectionString: "{{ secret('AZURE_CONNECTION_STRING') }}"
    container: myBlobContainer
    prefix: yourDirectory/subdirectory
    action: MOVE
    moveTo:
      container: mydata
      name: archive

Run a flow whenever one or more files arrived in the specified Azure Blob Storage container location. Then, process files and delete processed files to avoid re-triggering the flow for the same Blob objects during the next polling interval.

yaml
id: process_and_delete_files
namespace: company.team

tasks:
  - id: each
    type: io.kestra.plugin.core.flow.Loop
    values: "{{ trigger.blobs | jq('.[].name') }}"
    tasks:
      - id: return
        type: io.kestra.plugin.core.debug.Return
        format: "{{ item.value }}"

      - id: delete
        type: io.kestra.plugin.azure.storage.blob.Delete
        endpoint: "https://yourblob.blob.core.windows.net"
        connectionString: "{{ secret('AZURE_CONNECTION_STRING') }}"
        container: myBlobContainer
        name: "{{ item.value }}"

triggers:
  - id: watch
    type: io.kestra.plugin.azure.storage.blob.Trigger
    endpoint: "https://yourblob.blob.core.windows.net"
    connectionString: "{{ secret('AZURE_CONNECTION_STRING') }}"
    container: myBlobContainer
    prefix: yourDirectory/subdirectory
    action: NONE
    moveTo:
      container: myBlobContainer
      name: archive

Properties

action *string

Possible Values

MOVEDELETENONE

The action to perform on the retrieved files. If using NONE, make sure to handle the files inside your flow to avoid infinite triggering

container *string

The blob container.

allowConcurrent boolean

Defaultfalse

Specifies whether a trigger is allowed to start a new execution even if a previous run is still in progress.

connectionString string

Connection string of the Storage Account.

delimiter string

The delimiter for blob hierarchy, "/" for hierarchy based on directories.

endpoint string

The blob service endpoint.

filter string

DefaultFILES

Possible Values

FILESDIRECTORYBOTH

The filter for files or directories.

interval string

DefaultPT1M

Formatduration

Interval between polling.

The interval between 2 different polls of schedule, this can avoid to overload the remote system with too many calls. For most of the triggers that depend on external systems, a minimal interval must be at least PT30S. See ISO_8601 Durations for more information of available interval values.

maxFiles integerstring

Default25

The maximum number of files to retrieve at once

Limits the number of blobs retrieved per polling interval. If not specified, all matching blobs will be retrieved.

moveTo

The destination container and blob name

Definitions

io.kestra.plugin.azure.storage.blob.Copy-CopyObject

container*string

Source/destination container

name*string

Blob path

Full blob name (virtual path) inside the container

on string

DefaultCREATE_OR_UPDATE

Possible Values

CREATEUPDATECREATE_OR_UPDATE

Trigger event type

Defines when the trigger fires.

CREATE: only for newly discovered entities.
UPDATE: only when an already-seen entity changes.
CREATE_OR_UPDATE: fires on either event.

prefix string

Limits the response to keys that begin with the specified prefix.

regexp string

A regular expression to filter on the full key.

ex: regExp: .* to match all files regExp: .*2020-01-0.\\.csv to match files between 01 and 09 of january ending with .csv

sasToken string

The SAS token to use for authenticating requests.

This string should only be the query parameters (with or without a leading '?') and not a full URL.

sharedKeyAccountAccessKey string

Shared Key access key for authenticating requests.

sharedKeyAccountName string

Shared Key account name for authenticating requests.

stateKey string

State key

JSON-type KV key for persisted state. Default: <namespace>__<flowId>__<triggerId>

stateTtl string

State TTL

TTL for persisted state entries (e.g., PT24H, P7D).

stopAfter array

SubTypestring

Possible Values

CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTED

List of execution states after which a trigger should be stopped (a.k.a. disabled).

when string

Defaulttrue

A condition that determines whether the trigger should run.

A Pebble expression evaluated at trigger time. The trigger fires only when the expression evaluates to a truthy value (true, a non-empty string, a non-zero number). Use this to gate trigger execution on dynamic runtime values such as execution labels, flow variables, or environment conditions.

Outputs

blobs array

List of blobs that triggered the flow, each with its change type

Definitions

io.kestra.plugin.azure.storage.blob.Trigger-TriggeredBlob

changeTypestring

Possible Values

CREATEUPDATE

containerstring

eTagstring

lastModifiedstring

Formatdate-time