Azure DeleteFiles

Azure DeleteFiles

Certified

Delete a list of objects from Azure Data Lake Storage

Delete a list of objects from Azure Data Lake Storage using the Azure SDK.

yaml
type: io.kestra.plugin.azure.storage.adls.DeleteFiles

Download files from a remote server, upload them to Azure Data Lake Storage, finally delete them all at one

yaml
id: azure_storage_blob_delete_list
namespace: company.team

tasks:
  - id: for_each
    type: io.kestra.plugin.core.flow.Loop
    values: ["pikachu", "charmander"]
    tasks:
      - id: download_request
        type: io.kestra.plugin.core.http.Download
        uri: https://pokeapi.co/api/v2/pokemon/{{ item.value }}

      - id: to_ion
        type: io.kestra.plugin.serdes.json.JsonToIon
        from: "{{ outputs.download_request[item.value].uri }}"

      - id: upload_file
        type: io.kestra.plugin.azure.storage.adls.Upload
        filePath: "adls/pokemon/{{ item.value }}.json"
        from: "{{ outputs.to_ion[item.value].uri }}"
        connectionString: "{{ secret('AZURE_CONNECTION_STRING') }}"
        fileSystem: "tasks"
        endpoint: "https://yourblob.blob.core.windows.net"

  - id: delete_file
    type: io.kestra.plugin.azure.storage.adls.DeleteFiles
    concurrent: 2
    directoryPath: "adls/pokemon/"
    connectionString: "{{ secret('AZURE_CONNECTION_STRING') }}"
    fileSystem: "tasks"
    endpoint: "https://yourblob.blob.core.windows.net"
Properties

Directory Name

Service endpoint URL

Base HTTPS endpoint of the target service.

The name of the file systems. If the path name contains special characters, pass in the url encoded version of the path name

Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.

Definitions
assetFailureBehaviorstring
Possible Values
IGNOREFAILWARN

Asset failure behavior

Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.

enableAutobooleanstring

Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.

inputsarray

The assets consumed as inputs.

id*string
Min length1
typestring
outputs

The assets produced as outputs.

id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*string
Min length1

Custom asset type

descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
Minimum>= 2

Number of concurrent parallel deletions

Connection string of the Storage Account.

Defaultfalse

Whether to raise an error if the file is not found

The SAS token to use for authenticating requests.

This string should only be the query parameters (with or without a leading '?') and not a full URL.

Shared Key access key for authenticating requests.

Shared Key account name for authenticating requests.

Default0

The count of deleted files

Default0

The size of all the deleted files

The total number of files deleted.

The total size of all files deleted, in bytes.

More Plugins in this Category

Ashby
Ashby

Ashby plugin for Kestra

Data

1 tasks

1 blueprints

Dremio
Dremio

Tasks that connect to Dremio via JDBC to run queries and loads.

Data

3 tasks

3 blueprints