Execute Kestra Tasks on AWS Lambda MicroVMs

Run tasks natively on AWS Lambda MicroVMs — no container runtime required, with built-in image management and S3-based file staging.

When to use the Lambda MicroVM task runner

Use the Lambda MicroVM task runner when your workload needs lightweight VM isolation on ARM64 without the overhead of container scheduling or EC2 instance provisioning:

• Fast-start serverless workloads that need VM-level isolation but can tolerate a one-time per-region image build (a few minutes on first run)
• Long-running non-containerized tasks that exceed Lambda’s 15-minute function timeout — runs can last up to 8 hours
• ARM64-native workloads built for Amazon Linux 2023 on ARM64 that do not require a custom container image
• No infrastructure to maintain: the runner builds and manages its own image, with no AMIs or job queues to maintain

For containerized workloads, use the AWS Batch task runner. For direct EC2 control with Spot instances, GPU support, or custom AMIs, use the AWS EC2 task runner.

How it works

For each task run, the Lambda MicroVM task runner:

1. On the first run in an AWS account and region, builds a managed ARM64 image named kestra-task-runner-<hash> from an Amazon Linux 2023 minimal base. This one-time build takes a few minutes. Later runs reuse it. A plugin upgrade, a different memory, or a different baseImageArn builds a new image next to the old one.
2. Uploads the commands, environment variables, inputFiles and namespaceFiles to S3 under the kestra-lambda-microvm/run/ prefix. The MicroVM reads and writes them through short-lived presigned URLs, so it needs no IAM credentials for staging.
3. Starts the MicroVM and runs the task’s commands directly in it, with no container.
4. Streams command output to the log group at /aws/lambda-microvm/kestra (default) via CloudWatch Logs.
5. Downloads outputFiles from S3 to Kestra internal storage once the command completes.
6. Terminates the MicroVM unconditionally on success, failure, or worker interruption.

The MicroVM accesses S3 via presigned URLs generated by the Kestra worker, so the execution role does not need S3 permissions for file staging.

The following Pebble expressions and environment variables are available inside the task:

IAM permissions

The Lambda MicroVM task runner involves three IAM principals: the runner credentials (used by the Kestra worker to build the image, start and monitor runs, and stage files) and two roles assumed by Lambda: a build role (for image construction) and an execution role (for the task at runtime). For a quick setup, one role can serve as both the build and the execution role (same trust policy, merged permissions). Separate roles are cleaner, because the build role then grants nothing your commands could use.

Runner credentials

The IAM principal used by Kestra (the user or role whose credentials are passed via accessKeyId/secretKeyId, or the one resolved by the AWS default credentials provider chain) needs the following minimum permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["lambda:RunMicrovm", "lambda:GetMicrovm", "lambda:TerminateMicrovm"],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "lambda:CreateMicrovmImage",
        "lambda:GetMicrovmImage",
        "lambda:ListMicrovmImages",
        "lambda:GetMicrovmImageVersion",
        "lambda:ListManagedMicrovmImages",
        "lambda:ListManagedMicrovmImageVersions",
        "lambda:TagResource"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "iam:PassRole",
      "Resource": ["<buildRoleArn>", "<executionRoleArn>"],
      "Condition": {"StringEquals": {"iam:PassedToService": "lambda.amazonaws.com"}}
    },
    {
      "Effect": "Allow",
      "Action": "lambda:PassNetworkConnector",
      "Resource": "arn:aws:lambda:*:aws:network-connector:*"
    },
    {
      "Effect": "Allow",
      "Action": ["s3:PutObject", "s3:GetObject", "s3:DeleteObject", "s3:AbortMultipartUpload"],
      "Resource": "arn:aws:s3:::<bucket>/kestra-lambda-microvm/*"
    },
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket", "s3:GetBucketLocation"],
      "Resource": "arn:aws:s3:::<bucket>"
    },
    {
      "Effect": "Allow",
      "Action": "logs:DescribeLogGroups",
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": ["logs:CreateLogGroup", "logs:FilterLogEvents"],
      "Resource": "arn:aws:logs:<region>:<account>:log-group:<logGroup>:*"
    }
  ]
}

Replace <buildRoleArn> and <executionRoleArn> with the ARNs of the two roles described below.

The runner needs lambda:PassNetworkConnector because, when no network connector is set, AWS attaches its managed INTERNET_EGRESS and HTTP_INGRESS connectors to the MicroVM, and passing them requires this permission. If you set custom connectors in ingressNetworkConnectors or egressNetworkConnectors, add their ARNs to that statement.

Build role

The build role is assumed by Lambda MicroVM to construct the managed image on first run. Create a role with the following trust policy and permissions:

Trust policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": "arn:aws:s3:::<bucket>/kestra-lambda-microvm/bootstrap/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:<region>:<account>:log-group:<logGroup>:*"
    }
  ]
}

Replace <bucket> with your staging bucket name.

Execution role

The execution role is assumed by the MicroVM at runtime when running your task commands. At minimum it needs permission to write logs:

Trust policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:<region>:<account>:log-group:<logGroup>:*"
    }
  ]
}

Add any permissions your task commands need (for example, S3, DynamoDB, or other AWS service access) to this role.

KMS-encrypted buckets

To encrypt staged objects with KMS, set serverSideEncryption to aws:kms or aws:kms:dsse. Set kmsKeyId to use a specific key. When it’s unset, S3 uses the AWS managed key aws/s3. The runner rejects kmsKeyId when serverSideEncryption is AES256.

serverSideEncryption only applies to the objects the runner uploads: the bundle, the input archive, and the agent’s code artifact. The MicroVM writes output.zip and status.json through presigned links, so those objects use the bucket’s default encryption.

When you use aws:kms, or your bucket’s default encryption is SSE-KMS, add the following to the runner credentials:

{
  "Effect": "Allow",
  "Action": [
    "kms:GenerateDataKey",
    "kms:Decrypt"
  ],
  "Resource": "<kmsKeyArn>"
}

The runner credentials need kms:GenerateDataKey to upload objects and kms:Decrypt to read the result back. They need both for the MicroVM too, because they sign the presigned links it uses. With an SSE-KMS bucket default, kms:GenerateDataKey on that key also covers the output.zip and status.json objects the MicroVM writes. Add kms:Decrypt to the build role as well, since Lambda assumes it to read the agent’s code artifact from S3.

S3 lifecycle rule

The runner stages files under kestra-lambda-microvm/run/ in your bucket and removes them when the run ends. If a run is killed before it can clean up, the objects stay behind. As a safety net, add a lifecycle rule that expires objects under this prefix after 1 day.

Example flows

Basic shell command

id: lambda_microvm_shell
namespace: company.team

tasks:
  - id: shell
    type: io.kestra.plugin.scripts.shell.Commands
    taskRunner:
      type: io.kestra.plugin.ee.aws.runner.LambdaMicroVm
      region: us-east-1
      bucket: my-kestra-staging-bucket
      buildRoleArn: "arn:aws:iam::123456789012:role/kestra-lambda-build-role"
      executionRoleArn: "arn:aws:iam::123456789012:role/kestra-lambda-execution-role"
    commands:
      - echo "Hello from Lambda MicroVM"

Input and output files

id: lambda_microvm_with_files
namespace: company.team

inputs:
  - id: file
    type: FILE

tasks:
  - id: process
    type: io.kestra.plugin.scripts.shell.Commands
    inputFiles:
      data.txt: "{{ inputs.file }}"
    outputFiles:
      - out.txt
    taskRunner:
      type: io.kestra.plugin.ee.aws.runner.LambdaMicroVm
      region: us-east-1
      bucket: my-kestra-staging-bucket
      buildRoleArn: "arn:aws:iam::123456789012:role/kestra-lambda-build-role"
      executionRoleArn: "arn:aws:iam::123456789012:role/kestra-lambda-execution-role"
      memory: 4096
    commands:
      - cp "{{ workingDir }}/data.txt" "{{ workingDir }}/out.txt"

With STS role assumption

Instead of static accessKeyId / secretKeyId credentials, authenticate via AWS STS AssumeRole:

id: lambda_microvm_sts
namespace: company.team

tasks:
  - id: shell
    type: io.kestra.plugin.scripts.shell.Commands
    taskRunner:
      type: io.kestra.plugin.ee.aws.runner.LambdaMicroVm
      region: us-east-1
      bucket: my-kestra-staging-bucket
      buildRoleArn: "arn:aws:iam::123456789012:role/kestra-lambda-build-role"
      executionRoleArn: "arn:aws:iam::123456789012:role/kestra-lambda-execution-role"
      stsRoleArn: "arn:aws:iam::123456789012:role/kestra-runner-role"
      stsRoleExternalId: "{{ secret('STS_EXTERNAL_ID') }}"
      stsRoleSessionName: kestra-session
    commands:
      - echo "Hello from Lambda MicroVM with STS"

The runner credentials sign the presigned links the MicroVM uses to write its result, so temporary credentials must stay valid for the whole run. With stsRoleArn, the session defaults to the run’s wait (waitUntilCompletion, or the task timeout when set) plus 15 minutes, capped at 1 hour. For a longer wait, set stsRoleSessionDuration explicitly. Otherwise the runner fails the task before it starts. The role’s maximum session duration must allow the value you set.

Security

Use ingressNetworkConnectors and egressNetworkConnectors to control the MicroVM’s network access when your account has Lambda Network Connectors configured.

Monitoring

Set monitoring.enabled: true to run the task command under kotlp, a portable observability wrapper Kestra stages into the task’s working directory. No image change is needed.

kotlp reports the command’s resource usage as process.* task metrics, sampled every monitoring.metricsInterval (default PT1S):

The three sum metrics (process.cpu.time, process.disk.io, process.network.io) are cumulative and reflect totals for the entire run. The gauge metrics reflect the last sample taken before the command exits, not a peak.

kotlp also runs an embedded OTLP receiver, so traces the command exports are captured as task traces.

id: lambda_microvm_with_monitoring
namespace: company.team

tasks:
  - id: run
    type: io.kestra.plugin.scripts.shell.Commands
    taskRunner:
      type: io.kestra.plugin.ee.aws.runner.LambdaMicroVm
      region: us-east-1
      bucket: my-kestra-staging-bucket
      buildRoleArn: "arn:aws:iam::123456789012:role/kestra-lambda-build-role"
      executionRoleArn: "arn:aws:iam::123456789012:role/kestra-lambda-execution-role"
      monitoring:
        enabled: true
    commands:
      - echo "stdout line"
      - echo "stderr line" >&2

Monitoring constraints

An explicit commands list is required when monitoring is enabled: kotlp wraps the commands you supply.

Key properties

For the full property reference, see the plugin documentation.