
Huawei PutRecords
CertifiedWrite a batch of records to a Huawei Cloud DIS (Data Ingestion Service) stream
Huawei PutRecords
Write a batch of records to a Huawei Cloud DIS (Data Ingestion Service) stream
Sends one or more records to a DIS stream — the Huawei Cloud equivalent of
io.kestra.plugin.aws.kinesis.PutRecords. Each record requires data and either
partitionKey (routes by hash) or partitionId (targets a specific partition directly);
explicitHashKey optionally overrides the hash computed from partitionKey.
Records are chunked automatically to respect DIS's per-request limits (500 records and
5 MB per request); a single record larger than 1 MB fails the task naming the offending
record. DIS can reject individual records within an otherwise-successful (HTTP 200) batch —
failedRecordCount and the uri output surface exactly which ones failed and why.
type: io.kestra.plugin.huawei.dis.PutRecordsExamples
Send records defined inline as maps.
id: dis_put_records
namespace: company.team
tasks:
- id: put_records
type: io.kestra.plugin.huawei.dis.PutRecords
accessKeyId: "{{ secret('HUAWEI_AK') }}"
secretAccessKey: "{{ secret('HUAWEI_SK') }}"
region: eu-west-101
streamName: my-stream
from:
- data: "user sign-in event"
partitionKey: "user-1"
- data: "user sign-out event"
partitionKey: "user-1"
Send JSON records read from an internal storage ION file.
id: dis_put_records_json
namespace: company.team
inputs:
- id: dataFile
type: FILE
tasks:
- id: put_records
type: io.kestra.plugin.huawei.dis.PutRecords
accessKeyId: "{{ secret('HUAWEI_AK') }}"
secretAccessKey: "{{ secret('HUAWEI_SK') }}"
region: eu-west-101
streamName: my-stream
serdeType: JSON
from: "{{ inputs.dataFile }}"
Properties
from *RequiredNon-dynamicobject
Records to send
A single map, a list of maps, or a URI pointing to an ION file in Kestra internal storage.
Each map requires data and either partitionKey (routes the record to a partition by
hash) or partitionId (targets a specific partition directly); explicitHashKey may
optionally override the hash computed from partitionKey.
streamName *Requiredstring
DIS stream name
accessKeyId string
Access Key (AK) used to authenticate with Huawei Cloud
Huawei Cloud access key used together with secretAccessKey to sign API requests. Required for AK/SK-based authentication; not required when providing a pre-obtained securityToken. Sensitive — always provide via {{ secret('NAME') }}.
domainId string
Huawei Cloud Account Domain ID
Identifies the Huawei Cloud account (domain). Required when authenticating against global services such as IAM, or when requesting a domain-scoped IAM token.
endpointOverride string
DIS endpoint URL override
Overrides the default endpoint derived from region and endpointSuffix. Use this for
private endpoints, non-standard deployments, or tests. When set, endpointSuffix is
ignored.
Format: https://dis.<region>.myhuaweicloud.com (without trailing slash).
endpointSuffix string
Huawei Cloud domain suffix
Controls the top-level domain used when deriving the DIS endpoint from region.
Defaults to myhuaweicloud.com. Set to myhuaweicloud.eu for the Huawei Cloud European
sovereign cloud.
Ignored when endpointOverride is set.
failOnUnsuccessfulRecords booleanstring
trueFail the task if any record is rejected
If true (default), the task fails when DIS rejects at least one record in the batch. Set to false to continue despite partial failures — inspect failedRecordCount and uri instead.
pluginDefaultsRef Non-dynamicstring
Reference (ref) of the pluginDefaults to apply to this task.
projectId string
Huawei Cloud Project ID
Identifies the region-scoped project against which most regional services authenticate. Mutually exclusive with domainId for global services such as IAM.
region string
Huawei Cloud region
Region identifier such as eu-west-101, ap-southeast-1, or cn-north-4.
secretAccessKey string
Secret Key (SK) used to authenticate with Huawei Cloud
Huawei Cloud secret key paired with accessKeyId. Required for AK/SK-based authentication. Sensitive — always provide via {{ secret('NAME') }}.
securityToken string
Pre-obtained Huawei Cloud IAM token used as bearer credential for downstream API calls
When set, downstream Huawei tasks send this value in the X-Auth-Token header instead of signing requests with AK/SK. Sensitive.
serdeType string
STRINGSTRINGJSONBINARYSerialization type applied to each record's data value
STRING (default) encodes text as UTF-8; JSON serializes a map to a JSON string; BINARY expects the value to already be a byte array. All three are base64-encoded on the wire.
temporaryCredentials string
Inline IAM credential exchange
When set, the connection layer calls the Huawei IAM STS API once per task execution and
uses the returned temporary AK/SK + security token instead of the static accessKeyId
and secretAccessKey properties.
Configure once via pluginDefaults to apply transparently to every task in a namespace
without per-task credential wiring:
pluginDefaults:
- type: io.kestra.plugin.huawei.obs
values:
region: eu-west-101
temporaryCredentials:
authMethod: PASSWORD
username: my-iam-user
password: "{{ secret('HUAWEI_IAM_PASSWORD') }}"
domainName: my-account-domain
durationSeconds: 3600
**Long-running tasks: ** the exchange runs once at execution start. For RealtimeTrigger
or long-running Consume tasks that outlive durationSeconds, credentials will expire
mid-run. Use long-lived AK/SK properties or refresh externally in that case.
io.kestra.plugin.huawei.TemporaryCredentialsConfig
PASSWORDPASSWORDTOKENAuthentication method
Controls which credentials are used to obtain the session token before exchanging for temporary STS credentials.
PASSWORD(default): provideusername,password, anddomainName.TOKEN: provide an existingiamToken(X-Auth-Token).
Account domain name (PASSWORD method only)
The Huawei Cloud account name (domain name) that owns the IAM user.
Required when authMethod is PASSWORD. Visible in the Huawei Cloud console under
My Credentials → Domain Name.
900Lifetime of the temporary credentials in seconds
How long the returned temporary AK/SK/security-token should remain valid. Huawei Cloud accepts values between 900 (15 minutes) and 86400 (24 hours). Defaults to 900 seconds.
myhuaweicloud.comHuawei Cloud IAM endpoint suffix
Domain suffix used to build the IAM endpoint URL when no explicit endpoint override is set.
Defaults to myhuaweicloud.com. Set to myhuaweicloud.eu for the European sovereign cloud
(region eu-west-101 / EU-Dublin).
IAM token to exchange (TOKEN method only)
An existing Huawei Cloud X-Auth-Token to exchange for temporary STS credentials.
Required when authMethod is TOKEN. Sensitive — always provide via {{ secret('NAME') }}.
IAM password (PASSWORD method only)
Password for the IAM user identified by username.
Required when authMethod is PASSWORD.
Sensitive — always provide via {{ secret('NAME') }}.
Project name for project-scoped tokens (PASSWORD method only)
Overrides the project name used for scope=PROJECT token requests.
Defaults to the task's region value when omitted, which is correct for most regions.
PROJECTPROJECTDOMAINToken scope (PASSWORD method only)
Scope of the session token obtained during password authentication.
PROJECT(default): token is scoped to the project matchingprojectName(or the task'sregionwhenprojectNameis omitted). Use for most downstream tasks.DOMAIN: token is scoped to the domain.
IAM username (PASSWORD method only)
Huawei Cloud IAM username. Required when authMethod is PASSWORD.
Outputs
failedRecordCount integer
Number of records that DIS rejected within the batch
recordCount integer
Number of records sent to the DIS stream
uri string
uriURI of the ION file in Kestra internal storage containing per-record results
Metrics
dis.putrecords.count counter
recordsNumber of records attempted to be sent to the DIS stream.
dis.putrecords.failed counter
recordsNumber of records that DIS rejected within the batch.