Huawei DeleteList

Huawei DeleteList

Certified

Delete all OBS objects matching a filter

Lists objects matching the prefix/regexp filter (reusing the same logic as the List task) and batch-deletes them in chunks of up to 1000 (the OBS multi-object delete limit). By default the task fails if OBS reports any per-object delete error (errorOnFailure=true). Set it to false for best-effort deletion: errors are then logged as warnings and the number of failures is surfaced in the errors output so the caller can react programmatically. Set errorOnEmpty to true to also fail the task when no objects match the filter.

yaml
type: io.kestra.plugin.huawei.obs.DeleteList
yaml
id: obs_delete_list
namespace: company.team

tasks:
  - id: delete_prefix
    type: io.kestra.plugin.huawei.obs.DeleteList
    accessKeyId: "{{ secret('HUAWEI_AK') }}"
    secretAccessKey: "{{ secret('HUAWEI_SK') }}"
    region: "eu-west-101"
    bucket: "my-bucket"
    prefix: "tmp/2024/"

Delete matching a regexp, fail if no objects found

yaml
id: obs_delete_stale
namespace: company.team

tasks:
  - id: delete_stale
    type: io.kestra.plugin.huawei.obs.DeleteList
    accessKeyId: "{{ secret('HUAWEI_AK') }}"
    secretAccessKey: "{{ secret('HUAWEI_SK') }}"
    region: "eu-west-101"
    bucket: "my-bucket"
    prefix: "reports/"
    regexp: '.*_stale\.csv'
    errorOnEmpty: true
Properties

OBS bucket name

Name of the OBS bucket to operate on. The bucket must already exist.

Access Key (AK) used to authenticate with Huawei Cloud

Huawei Cloud access key used together with secretAccessKey to sign API requests. Required for AK/SK-based authentication; not required when providing a pre-obtained securityToken. Sensitive — always provide via {{ secret('NAME') }}.

Possible Values
OBSV2V4

Request signing algorithm

Controls how OBS client signs each request:

  • OBS — native Huawei OBS signing (default; use for real OBS endpoints).
  • V2 — S3 v2 HMAC signing; required for MinIO and other S3-compatible endpoints.
  • V4 — S3 v4 signing; not compatible with MinIO via the OBS SDK due to a date-format mismatch. Do not use V4 with S3-compatible endpoints.

A delimiter that groups keys

Keys containing the delimiter after the prefix are grouped under a common prefix in the response. Useful for emulating a folder hierarchy (e.g. delimiter: /).

Huawei Cloud Account Domain ID

Identifies the Huawei Cloud account (domain). Required when authenticating against global services such as IAM, or when requesting a domain-scoped IAM token.

Override for the OBS endpoint URL

Full URL of the OBS endpoint to connect to instead of the region-derived default (https://obs.<region>.myhuaweicloud.com). Required when using S3-compatible endpoints such as MinIO (e.g. http://localhost: 9000). Trailing slashes are stripped automatically.

Defaultmyhuaweicloud.com

Domain suffix for the region-derived OBS endpoint

Suffix appended to build https://obs.<region>.<endpointSuffix> when no endpointOverride is set. Defaults to myhuaweicloud.com. Set to myhuaweicloud.eu for the European sovereign region (e.g. eu-west-101). Ignored when endpointOverride is set.

Defaultfalse

Fail the task when no objects match the filter

When true, the task throws an exception if the listing returns zero matching objects. Useful to catch misconfigured filters in production workflows.

Defaulttrue

Fail the task when OBS reports any per-object delete error

When true (the default), the task throws after attempting every batch if OBS returned one or more error results, so a partial failure cannot be silently swallowed. Set to false for best-effort deletion: failures are logged as warnings and counted in the errors output instead of failing the task.

Marker to start listing from (exclusive)

Returned objects will appear after this key in lexicographic order. Use the nextMarker from a previous response to paginate.

Default1000

Maximum number of keys to return per page (default 1000)

The SDK may return fewer keys than requested if the page boundary falls mid-prefix. ObsService.list iterates all pages automatically so the total result set is not bounded by this value — it only affects the page size sent to OBS.

Use path-style access for object keys

When true, the bucket name is placed in the URL path (http://host/bucket/key) instead of the virtual-hosted style (http://bucket.host/key). Required for MinIO and most S3-compatible endpoints. Default is false (virtual-hosted style, as used by real OBS).

Reference (ref) of the pluginDefaults to apply to this task.

Limits the response to keys that begin with the specified prefix

Huawei Cloud Project ID

Identifies the region-scoped project against which most regional services authenticate. Mutually exclusive with domainId for global services such as IAM.

Client-side regular expression applied to the full object key after server-side listing

Only objects whose key matches this pattern (evaluated with String.matches) are included in the output. Applied after prefix/delimiter filtering done by OBS, so the regexp sees the complete key including any prefix.

Huawei Cloud region

Region identifier such as eu-west-101, ap-southeast-1, or cn-north-4.

Secret Key (SK) used to authenticate with Huawei Cloud

Huawei Cloud secret key paired with accessKeyId. Required for AK/SK-based authentication. Sensitive — always provide via {{ secret('NAME') }}.

Pre-obtained Huawei Cloud IAM token used as bearer credential for downstream API calls

When set, downstream Huawei tasks send this value in the X-Auth-Token header instead of signing requests with AK/SK. Sensitive.

Inline IAM credential exchange

When set, the connection layer calls the Huawei IAM STS API once per task execution and uses the returned temporary AK/SK + security token instead of the static accessKeyId and secretAccessKey properties.

Configure once via pluginDefaults to apply transparently to every task in a namespace without per-task credential wiring:

pluginDefaults: 
  - type: io.kestra.plugin.huawei.obs
    values: 
      region: eu-west-101
      temporaryCredentials: 
        authMethod: PASSWORD
        username: my-iam-user
        password: "{{ secret('HUAWEI_IAM_PASSWORD') }}"
        domainName: my-account-domain
        durationSeconds: 3600

**Long-running tasks: ** the exchange runs once at execution start. For RealtimeTrigger or long-running Consume tasks that outlive durationSeconds, credentials will expire mid-run. Use long-lived AK/SK properties or refresh externally in that case.

Definitions
authMethodstring
DefaultPASSWORD
Possible Values
PASSWORDTOKEN

Authentication method

Controls which credentials are used to obtain the session token before exchanging for temporary STS credentials.

  • PASSWORD (default): provide username, password, and domainName.
  • TOKEN: provide an existing iamToken (X-Auth-Token).
domainNamestring

Account domain name (PASSWORD method only)

The Huawei Cloud account name (domain name) that owns the IAM user. Required when authMethod is PASSWORD. Visible in the Huawei Cloud console under My Credentials → Domain Name.

durationSecondsintegerstring
Default900

Lifetime of the temporary credentials in seconds

How long the returned temporary AK/SK/security-token should remain valid. Huawei Cloud accepts values between 900 (15 minutes) and 86400 (24 hours). Defaults to 900 seconds.

endpointSuffixstring
Defaultmyhuaweicloud.com

Huawei Cloud IAM endpoint suffix

Domain suffix used to build the IAM endpoint URL when no explicit endpoint override is set. Defaults to myhuaweicloud.com. Set to myhuaweicloud.eu for the European sovereign cloud (region eu-west-101 / EU-Dublin).

iamTokenstring

IAM token to exchange (TOKEN method only)

An existing Huawei Cloud X-Auth-Token to exchange for temporary STS credentials. Required when authMethod is TOKEN. Sensitive — always provide via {{ secret('NAME') }}.

passwordstring

IAM password (PASSWORD method only)

Password for the IAM user identified by username. Required when authMethod is PASSWORD. Sensitive — always provide via {{ secret('NAME') }}.

projectNamestring

Project name for project-scoped tokens (PASSWORD method only)

Overrides the project name used for scope=PROJECT token requests. Defaults to the task's region value when omitted, which is correct for most regions.

scopestring
DefaultPROJECT
Possible Values
PROJECTDOMAIN

Token scope (PASSWORD method only)

Scope of the session token obtained during password authentication.

  • PROJECT (default): token is scoped to the project matching projectName (or the task's region when projectName is omitted). Use for most downstream tasks.
  • DOMAIN: token is scoped to the domain.
usernamestring

IAM username (PASSWORD method only)

Huawei Cloud IAM username. Required when authMethod is PASSWORD.

Default0

Number of objects successfully deleted

Default0

Number of objects OBS reported as failed to delete

Always 0 when errorOnFailure=true (the task throws instead). With errorOnFailure=false this lets the caller detect partial failures programmatically.

Default0

Total size in bytes of all deleted objects