AWS DynamoDB BatchWriteItem
CertifiedBatch write items to DynamoDB
AWS DynamoDB BatchWriteItem
Batch write items to DynamoDB
Puts or deletes items with BatchWriteItem, 25 per request.
from takes a list of items, a single item, or a kestra:// file such as the output of a Scan with fetchType: STORE.
An item is written as a put, or wrapped in putRequest or deleteRequest (with the key) to choose explicitly.
Two requests for the same key in one batch of 25 fail that batch, after the earlier batches are written.
type: io.kestra.plugin.aws.dynamodb.BatchWriteItemExamples
Batch write items from a list into a DynamoDB table
id: aws_dynamodb_batch_write_list
namespace: company.team
tasks:
- id: batch_write
type: io.kestra.plugin.aws.dynamodb.BatchWriteItem
accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
region: "eu-central-1"
tableName: "persons"
from:
- id: "1"
firstname: "John"
lastname: "Doe"
- id: "2"
firstname: "Jane"
lastname: "Smith"
Batch write items with explicit putRequest and deleteRequest wrappers
id: aws_dynamodb_batch_write_puts_and_deletes
namespace: company.team
tasks:
- id: batch_write
type: io.kestra.plugin.aws.dynamodb.BatchWriteItem
accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
region: "eu-central-1"
tableName: "persons"
from:
- putRequest:
id: "1"
firstname: "John"
- deleteRequest:
id: "2"
Load the items read by a Scan into another table
id: aws_dynamodb_batch_write_file
namespace: company.team
tasks:
- id: extract_customers
type: io.kestra.plugin.aws.dynamodb.Scan
accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
region: "eu-central-1"
tableName: "customers_staging"
- id: batch_write
type: io.kestra.plugin.aws.dynamodb.BatchWriteItem
accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
region: "eu-central-1"
tableName: "customers"
from: "{{ outputs.extract_customers.uri }}"
Properties
from *stringarrayobject
Structured data items, either as a map, a list of map, a URI, or a JSON string.
Structured data items can be defined in the following ways:
- A single item as a map (a document).
- A list of items as a list of maps (a list of documents).
- A URI, supported schemes are
kestrafor internal storage files,filefor host local files, andnsfilefor namespace files. - A JSON String that will then be serialized either as a single item or a list of items.
tableName *string
Table name
Target DynamoDB table for the operation.
accessKeyId string
Access Key Id in order to connect to AWS
If no credentials are defined, we will use the default credentials provider chain to fetch credentials.
assets
Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.
io.kestra.core.models.assets.AssetsDeclaration
IGNOREFAILWARNAsset failure behavior
Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.
Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.
The assets consumed as inputs.
io.kestra.core.models.assets.AssetIdentifier
1The assets produced as outputs.
io.kestra.plugin.ee.assets.Dataset
1150{}1150io.kestra.plugin.ee.assets.File
1150{}1150io.kestra.plugin.ee.assets.Table
1150{}1150io.kestra.plugin.ee.assets.VM
1150{}1150io.kestra.core.models.assets.External
1150{}1150io.kestra.core.models.assets.Custom
11501Custom asset type
{}1150endpointOverride string
The endpoint with which the SDK should communicate
This property allows you to use a different S3 compatible storage backend.
maxRetries integerstring
5Max retries for unprocessed items
Retries when DynamoDB returns unprocessed items because of throttling, between 0 and 20. Defaults to 5.
region string
AWS region with which the SDK should communicate
secretKeyId string
Secret Key Id in order to connect to AWS
If no credentials are defined, we will use the default credentials provider chain to fetch credentials.
sessionToken string
AWS session token, retrieved from an AWS token service, used for authenticating that this user has received temporary permissions to access a given resource
If no credentials are defined, we will use the default credentials provider chain to fetch credentials.
stsEndpointOverride string
The AWS STS endpoint with which the SDKClient should communicate
stsRoleArn string
AWS STS Role
The Amazon Resource Name (ARN) of the role to assume. If set the task will use the StsAssumeRoleCredentialsProvider. If no credentials are defined, we will use the default credentials provider chain to fetch credentials.
stsRoleExternalId string
AWS STS External Id
A unique identifier that might be required when you assume a role in another account. This property is only used when an stsRoleArn is defined.
stsRoleSessionDuration string
PT15MAWS STS Session duration
The duration of the role session (default: 15 minutes, i.e., PT15M). This property is only used when an stsRoleArn is defined.
stsRoleSessionName string
AWS STS Session name
This property is only used when an stsRoleArn is defined.
Outputs
count integer
0Number of put and delete requests applied
Metrics
dynamodb.items.written.count counter
itemsThe total number of put and delete requests applied to DynamoDB.
