AWS StartServerlessJobRun

AWS StartServerlessJobRun
Certified
Start a job on an EMR Serverless app

Submits a job run to an existing EMR Serverless application using the Spark driver with the provided entry point. Returns jobRunId only; does not wait for job completion.

yaml
type: io.kestra.plugin.aws.emr.StartServerlessJobRun

Examples

Start EMR Serverless job

yaml
id: start_emr_job
namespace: company.team

tasks:
  - id: start_job
    type: io.kestra.plugin.aws.emr.StartServerlessJobRun
    accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
    secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
    region: "eu-central-1"
    applicationId: "00f123abc456xyz"
    executionRoleArn: "arn:aws:iam::123456789012:role/EMRServerlessRole"
    jobName: "sample-spark-job"
    entryPoint: "s3://my-bucket/scripts/spark-app.py"
    jobDriver:
      sparkSubmit:
        entryPointArguments:
          - "--arg1"
          - "value1"

Properties

applicationId *string

Application ID

Existing EMR Serverless application to run the job on.

entryPoint *string

Job entry point

Path to the Spark application file; passed to spark-submit.

executionRoleArn *string

Execution role ARN

IAM role assumed for this job run.

jobName *string

Job name

Displayed name for the job run.

accessKeyId string

Access Key Id in order to connect to AWS

If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

compatibilityMode booleanstring

Enable compatibility mode

Use it to connect to S3 bucket with S3 compatible services that don't support the new transport client.

endpointOverride string

The endpoint with which the SDK should communicate

This property allows you to use a different S3 compatible storage backend.

forcePathStyle booleanstring

Force path style access

Must only be used when compatibilityMode is enabled.

pluginDefaultsRef string

Reference (ref) of the pluginDefaults to apply to this task.

region string

AWS region with which the SDK should communicate

secretKeyId string

Secret Key Id in order to connect to AWS

If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

sessionToken string

AWS session token, retrieved from an AWS token service, used for authenticating that this user has received temporary permissions to access a given resource

If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

stsEndpointOverride string

The AWS STS endpoint with which the SDKClient should communicate

stsRoleArn string

AWS STS Role

The Amazon Resource Name (ARN) of the role to assume. If set the task will use the StsAssumeRoleCredentialsProvider. If no credentials are defined, we will use the default credentials provider chain to fetch credentials.

stsRoleExternalId string

AWS STS External Id

A unique identifier that might be required when you assume a role in another account. This property is only used when an stsRoleArn is defined.

stsRoleSessionDuration string

DefaultPT15M

AWS STS Session duration

The duration of the role session (default: 15 minutes, i.e., PT15M). This property is only used when an stsRoleArn is defined.

stsRoleSessionName string

AWS STS Session name

This property is only used when an stsRoleArn is defined.

Outputs

jobRunId string