AWS Query
CertifiedRun an Athena query and fetch results
AWS Query
Run an Athena query and fetch results
Executes an Athena SQL query and waits for completion unless fetchType is NONE. Converts primitive columns to native types and stores results according to fetchType; default STORE writes an ION file to internal storage. skipHeader defaults to true to drop the header row; complex Athena types (array/map/struct) are returned as strings.
type: io.kestra.plugin.aws.athena.QueryExamples
id: aws_athena_query
namespace: company.team
tasks:
- id: query
type: io.kestra.plugin.aws.athena.Query
accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
region: "eu-central-1"
database: my_database
outputLocation: s3://some-s3-bucket
query: |
select * from cloudfront_logs limit 10
Properties
database *Requiredstring
Athena database
Target database for the query.
outputLocation *Requiredstring
Athena output location
Existing S3 URI where Athena writes result files (required by service).
query *Requiredstring
Athena SQL query
SQL statement executed by Athena.
accessKeyId string
Access Key Id in order to connect to AWS
If no credentials are defined, we will use the default credentials provider chain to fetch credentials.
catalog string
Athena catalog
Optional catalog name; use default if omitted.
compatibilityMode booleanstring
Enable compatibility mode
Use it to connect to S3 bucket with S3 compatible services that don't support the new transport client.
endpointOverride string
The endpoint with which the SDK should communicate
This property allows you to use a different S3 compatible storage backend.
fetchType string
STORESTOREFETCHFETCH_ONENONEFetch strategy
Default STORE writes all rows to internal storage (recommended for large result sets). FETCH loads ALL rows into memory and should only be used for small, bounded result sets. FETCH_ONE returns the first row. NONE submits the query without waiting for results.
forcePathStyle booleanstring
Force path style access
Must only be used when compatibilityMode is enabled.
pluginDefaultsRef Non-dynamicstring
Reference (ref) of the pluginDefaults to apply to this task.
region string
AWS region with which the SDK should communicate
secretKeyId string
Secret Key Id in order to connect to AWS
If no credentials are defined, we will use the default credentials provider chain to fetch credentials.
sessionToken string
AWS session token, retrieved from an AWS token service, used for authenticating that this user has received temporary permissions to access a given resource
If no credentials are defined, we will use the default credentials provider chain to fetch credentials.
skipHeader booleanstring
trueSkip header row
If true (default), drop the first returned row, which is usually the column header.
stsEndpointOverride string
The AWS STS endpoint with which the SDKClient should communicate
stsRoleArn string
AWS STS Role
The Amazon Resource Name (ARN) of the role to assume. If set the task will use the StsAssumeRoleCredentialsProvider. If no credentials are defined, we will use the default credentials provider chain to fetch credentials.
stsRoleExternalId string
AWS STS External Id
A unique identifier that might be required when you assume a role in another account. This property is only used when an stsRoleArn is defined.
stsRoleSessionDuration string
PT15MAWS STS Session duration
The duration of the role session (default: 15 minutes, i.e., PT15M). This property is only used when an stsRoleArn is defined.
stsRoleSessionName string
AWS STS Session name
This property is only used when an stsRoleArn is defined.
Outputs
queryExecutionId string
Query execution ID
Identifier returned by StartQueryExecution.
row object
First row
Single row when fetchType is FETCH_ONE.
rows array
Rows
All rows when fetchType is FETCH.
size integer
Fetched row count
Number of rows returned or stored.
uri string
uriStored data URI
Internal storage URI when fetchType is STORE.
Metrics
data.scanned.bytes counter
bytesThe amount of data scanned by the Athena query in bytes.
engine.execution.duration counter
millisecondsThe time taken by the Athena engine to execute the query.
fetch.rows counter
rowsThe number of rows fetched or stored by the task, based on the fetchType.
query.planning.duration counter
millisecondsThe time taken to plan the Athena query.
query.queue.duration counter
millisecondsThe time the Athena query spent in the queue.
service.processing.duration counter
millisecondsThe time taken for service processing of the Athena query.
total.execution.duration counter
millisecondsThe total execution time of the Athena query.
total.rows counter
rowsThe total number of rows returned by the Athena query.
