Trigger

yaml
type: "io.kestra.plugin.aws.s3.Trigger"

Examples

yaml
id: s3_listen
namespace: company.team

tasks:
  - id: each
    type: io.kestra.plugin.core.flow.ForEach
    values: "{{ trigger.objects | jq('.[].uri') }}"
    tasks:
      - id: return
        type: io.kestra.plugin.core.debug.Return
        format: "{{ taskrun.value }}"

triggers:
  - id: watch
    type: io.kestra.plugin.aws.s3.Trigger
    interval: "PT5M"
    accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
    secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
    region: "eu-central-1"
    bucket: "my-bucket"
    prefix: "sub-dir"
    action: MOVE
    moveTo:
      key: archive
      bucket: "new-bucket"

yaml
id: s3_listen
namespace: company.team

tasks:
  - id: each
    type: io.kestra.plugin.core.flow.ForEach
    values: "{{ trigger.objects | jq('.[].key') }}"
    tasks:
      - id: return
        type: io.kestra.plugin.core.debug.Return
        format: "{{ taskrun.value }}"

      - id: delete
        type: io.kestra.plugin.aws.s3.Delete
        accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
        secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
        region: "eu-central-1"
        bucket: "my-bucket"
        key: "{{ taskrun.value }}"

triggers:
  - id: watch
    type: io.kestra.plugin.aws.s3.Trigger
    interval: "PT5M"
    accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
    secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
    region: "eu-central-1"
    bucket: "my-bucket"
    prefix: "sub-dir"
    action: NONE

Properties

action *string

Possible Values

MOVEDELETENONE

bucket *string

accessKeyId string

allowConcurrent boolean

Defaultfalse

compatibilityMode booleanstring

Defaultfalse

conditions

Definitions

Condition to allow events between two specific datetime values.

type*object

Possible Values

io.kestra.plugin.core.condition.DateTimeBetweenio.kestra.core.models.conditions.types.DateTimeBetweenConditionio.kestra.plugin.core.condition.DateTimeBetweenCondition

afterstring

beforestring

datestring

Default{{ trigger.date }}

Condition to allow events on a particular day of the week.

dayOfWeek*string

Possible Values

MONDAYTUESDAYWEDNESDAYTHURSDAYFRIDAYSATURDAYSUNDAY

type*object

Possible Values

io.kestra.plugin.core.condition.DayWeekio.kestra.core.models.conditions.types.DayWeekConditionio.kestra.plugin.core.condition.DayWeekCondition

datestring

Default{{ trigger.date }}

Condition to execute tasks on a specific day of the week relative to the current month (first, last, ...)

dayInMonth*string

Possible Values

FIRSTLASTSECONDTHIRDFOURTH

dayOfWeek*string

Possible Values

MONDAYTUESDAYWEDNESDAYTHURSDAYFRIDAYSATURDAYSUNDAY

type*object

Possible Values

io.kestra.plugin.core.condition.DayWeekInMonthio.kestra.plugin.core.condition.DayWeekInMonthConditionio.kestra.core.models.conditions.types.DayWeekInMonthCondition

datestring

Default{{ trigger.date }}

Condition for a specific flow of an execution.

flowId*string

namespace*string

type*object

Possible Values

io.kestra.plugin.core.condition.ExecutionFlowio.kestra.plugin.core.condition.ExecutionFlowConditionio.kestra.core.models.conditions.types.ExecutionFlowCondition

Condition that checks labels of an execution.

labels*arrayobject

type*object

Possible Values

io.kestra.plugin.core.condition.ExecutionLabelsio.kestra.core.models.conditions.types.ExecutionLabelsConditionio.kestra.plugin.core.condition.ExecutionLabelsCondition

Condition for an execution namespace.

namespace*string

type*object

Possible Values

io.kestra.plugin.core.condition.ExecutionNamespaceio.kestra.core.models.conditions.types.ExecutionNamespaceConditionio.kestra.plugin.core.condition.ExecutionNamespaceCondition

comparisonstring

Possible Values

EQUALSPREFIXSUFFIX

prefixbooleanstring

Defaultfalse

Condition based on the outputs of an upstream execution.

expression*booleanstring

type*object

Possible Values

io.kestra.plugin.core.condition.ExecutionOutputsio.kestra.core.models.conditions.types.ExecutionOutputsConditionio.kestra.plugin.core.condition.ExecutionOutputsCondition

Condition based on execution status.

type*object

Possible Values

io.kestra.plugin.core.condition.ExecutionStatusio.kestra.core.models.conditions.types.ExecutionStatusConditionio.kestra.plugin.core.condition.ExecutionStatusCondition

inarray

SubTypestring

Possible Values

CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTED

notInarray

SubTypestring

Possible Values

CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTED

Condition based on variable expression.

expression*string

type*object

Possible Values

io.kestra.plugin.core.condition.Expressionio.kestra.plugin.core.condition.ExpressionConditionio.kestra.core.models.conditions.types.VariableCondition

Condition for a specific flow. Note that this condition is deprecated, use `io.kestra.plugin.core.condition.ExecutionFlow` instead.

flowId*string

namespace*string

type*object

Possible Values

io.kestra.plugin.core.condition.FlowConditionio.kestra.core.models.conditions.types.FlowCondition

Condition for a flow namespace.

namespace*string

type*object

Possible Values

io.kestra.plugin.core.condition.FlowNamespaceConditionio.kestra.core.models.conditions.types.FlowNamespaceCondition

prefixboolean

Defaultfalse

Condition that matches if any taskRun has retry attempts.

type*object

Possible Values

io.kestra.plugin.core.condition.HasRetryAttemptio.kestra.plugin.core.condition.HasRetryAttemptConditionio.kestra.core.models.conditions.types.HasRetryAttemptCondition

inarray

SubTypestring

Possible Values

CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTED

notInarray

SubTypestring

Possible Values

CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTED

Run a flow if the list of preconditions is met in a time window.

conditions*object

id*string

Validation RegExp^[a-zA-Z0-9][a-zA-Z0-9_-]*

Min length1

type*object

Possible Values

io.kestra.plugin.core.condition.MultipleConditionio.kestra.core.models.conditions.types.MultipleCondition

resetOnSuccessboolean

Defaulttrue

timeWindow

Default

{
  "type": "DURATION_WINDOW"
}

io.kestra.core.models.triggers.TimeWindow

deadlinestring

Formatpartial-time

endTimestring

Formatpartial-time

startTimestring

Formatpartial-time

typestring

DefaultDURATION_WINDOW

Possible Values

DAILY_TIME_DEADLINEDAILY_TIME_WINDOWDURATION_WINDOWSLIDING_WINDOW

windowstring

Formatduration

windowAdvancestring

Formatduration

windowstring

Formatduration

windowAdvancestring

Formatduration

Condition to exclude other conditions.

conditions*

Min items1

type*object

Possible Values

io.kestra.plugin.core.condition.Notio.kestra.core.models.conditions.types.NotConditionio.kestra.plugin.core.condition.NotCondition

Condition to have at least one condition validated.

conditions*

Min items1

type*object

Possible Values

io.kestra.plugin.core.condition.Orio.kestra.core.models.conditions.types.OrConditionio.kestra.plugin.core.condition.OrCondition

Condition to allow events on public holidays.

type*object

Possible Values

io.kestra.plugin.core.condition.PublicHolidayio.kestra.plugin.core.condition.PublicHolidayConditionio.kestra.core.models.conditions.types.PublicHolidayCondition

countrystring

datestring

Default{{ trigger.date}}

subDivisionstring

Condition to allow events between two specific times.

type*object

Possible Values

io.kestra.plugin.core.condition.TimeBetweenio.kestra.plugin.core.condition.TimeBetweenConditionio.kestra.core.models.conditions.types.TimeBetweenCondition

afterstring

beforestring

datestring

Default{{ trigger.date }}

Condition to allow events on weekend.

type*object

Possible Values

io.kestra.plugin.core.condition.Weekendio.kestra.plugin.core.condition.WeekendConditionio.kestra.core.models.conditions.types.WeekendCondition

datestring

Default{{ trigger.date }}

delimiter string

encodingType string

endpointOverride string

expectedBucketOwner string

filter string

DefaultBOTH

Possible Values

FILESDIRECTORYBOTH

forcePathStyle booleanstring

Defaultfalse

interval string

DefaultPT1M

Formatduration

marker string

maxFiles integerstring

Default25

maxKeys integerstring

Default1000

moveTo

Definitions

io.kestra.plugin.aws.s3.Copy-CopyObject

bucket*string

key*string

kmsKeyIdstring

serverSideEncryptionstring

Possible Values

NONEAES256AWS_KMS

on string

DefaultCREATE_OR_UPDATE

Possible Values

CREATEUPDATECREATE_OR_UPDATE

prefix string

regexp string

region string

requestPayer string

secretKeyId string

sessionToken string

stateKey string

stateTtl string

stopAfter array

SubTypestring

Possible Values

CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTED

stsEndpointOverride string

stsRoleArn string

stsRoleExternalId string

stsRoleSessionDuration string

DefaultPT15M

stsRoleSessionName string

Outputs

objects array

Definitions

io.kestra.plugin.aws.s3.Trigger-TriggeredObject

changeTypestring

Possible Values

CREATEUPDATE

etagstring

keystring

lastModifiedstring

Formatdate-time

owner

io.kestra.plugin.aws.s3.models.Owner

displayNamestring

idstring

sizeinteger

uristring

Formaturi

Event-driven data ingestion to AWS S3 data lake managed by Apache Iceberg, AWS Glue and Amazon Athena

Anomaly detection using DuckDB SQL query and S3 file event trigger, sending a CSV file attachment via email if anomalies are detected

Detect New Files in S3 and process them in Python

Core Plugins and tasks

Tasks that provide Kestra's built-in orchestration, I/O, and observability capabilities.

CoreStorageScript

PostgreSQL

Tasks that connect to PostgreSQL via JDBC to run queries and loads.

Database

SurrealDB

Tasks that run SurrealQL queries against SurrealDB.

Database

Trigger

Create automations with AWS S3 Trigger

Event-driven data ingestion to AWS S3 data lake managed by Apache Iceberg, AWS Glue and Amazon Athena

Anomaly detection using DuckDB SQL query and S3 file event trigger, sending a CSV file attachment via email if anomalies are detected

Detect New Files in S3 and process them in Python

More Plugins in this Category

Core Plugins and tasks

PostgreSQL

SurrealDB

2.1.0