AWS LambdaMicroVm

Enterprise Edition only.

This task runner starts an AWS Lambda MicroVM, runs your task's commands in it, then terminates it. The microVM is isolated and starts fast, so it suits short and bursty jobs that should not share a machine. The containerImage property is ignored: a microVM runs a Lambda microVM image, not a container image.

Minimal setup. Set region, bucket, buildRoleArn and executionRoleArn. The first run builds a microVM image holding the Kestra agent in your account (a few minutes, once). Later runs reuse it. The image is built from a Dockerfile on public.ecr.aws/lambda/microvms: al2023-minimal, so commands get the Amazon Linux 2023 minimal userland (arm64). The image is named kestra-task-runner-<hash> and never changes, so a plugin upgrade or a different memory builds a new image next to the old one. If the build fails, the error carries AWS's reason and the build log location: fix the cause, delete the failed image and run again.

How a run works. The runner uploads your input files and the commands to bucket (under kestra-lambda-microvm/), starts the microVM with short-lived links to those objects, and waits for the agent to write the result back. The microVM needs no IAM role for this. The staged objects, which include the task's environment variables, are deleted when the run ends, so use a bucket you trust and, ideally, with default encryption. Command output appears in the Kestra logs, and the exit code is the real exit code of your commands. Access the working directory with {{ workingDir }} or the WORKING_DIR environment variable. To pass inputFiles, outputFiles, or namespaceFiles, nothing more is needed: bucket is always used.

Differences from Batch and Ec2.

• There is no containerImage, no vCPU and no disk setting: the Lambda MicroVMs API only exposes memory, set with memory, which applies to the image.
• microVMs run on arm64 only: binaries and native Python wheels your commands use must be arm64.
• A run lasts at most 8 hours (waitUntilCompletion).
• The first run in a region builds the image.
• File permissions are not kept when files go through the staging bucket, as with Ec2.
• On a failed run, Kestra core drops the taskRunner outputs, so the runtime details (microvmId, state, stateReason, startedAt, stoppedAt, imageArn, imageVersion, logGroup, logStream, region) are available in {{ outputs.<taskId>.vars.kestraTaskRunner }} instead.

Runner credentials. The credentials need (action names follow the API operation names):

{
  "Version": "2012-10-17",
  "Statement": [
    {"Effect": "Allow", "Action": ["lambda: RunMicrovm", "lambda: GetMicrovm", "lambda: TerminateMicrovm"], "Resource": "*"},
    {"Effect": "Allow", "Action": ["lambda: CreateMicrovmImage", "lambda: GetMicrovmImage", "lambda: ListMicrovmImages",
      "lambda: GetMicrovmImageVersion", "lambda: ListManagedMicrovmImages", "lambda: ListManagedMicrovmImageVersions", "lambda: TagResource"], "Resource": "*"},
    {"Effect": "Allow", "Action": "iam: PassRole", "Resource": ["<buildRoleArn>", "<executionRoleArn>"],
      "Condition": {"StringEquals": {"iam: PassedToService": "lambda.amazonaws.com"}}},
    {"Effect": "Allow", "Action": "lambda: PassNetworkConnector", "Resource": "arn: aws: lambda: *: aws: network-connector: *"},
    {"Effect": "Allow", "Action": ["s3: PutObject", "s3: GetObject", "s3: DeleteObject", "s3: AbortMultipartUpload"], "Resource": "arn: aws: s3: : : <bucket>/kestra-lambda-microvm/*"},
    {"Effect": "Allow", "Action": ["s3: ListBucket", "s3: GetBucketLocation"], "Resource": "arn: aws: s3: : : <bucket>"},
    {"Effect": "Allow", "Action": "logs: DescribeLogGroups", "Resource": "*"},
    {"Effect": "Allow", "Action": ["logs: CreateLogGroup", "logs: FilterLogEvents"], "Resource": "arn: aws: logs: <region>: <account>: log-group: <logGroup>: *"}
  ]
}

lambda: PassNetworkConnector is needed because, with no network connector set, AWS applies its managed INTERNET_EGRESS and HTTP_INGRESS connectors to the microVM, and passing them requires this permission. Custom connectors set in ingressNetworkConnectors or egressNetworkConnectors need their own ARNs added to that statement. The runner credentials must also be valid for the whole run, since the links are signed with them: with temporary credentials, make sure they outlive waitUntilCompletion.

Encryption with KMS. With serverSideEncryption: aws: kms (or an SSE-KMS bucket default), add a statement for the key to the runner credentials: kms: GenerateDataKey to upload the bundle, the input archive and the agent's code artifact, and kms: Decrypt because the microVM reads the bundle and the input archive through presigned links, which act with these credentials, and the runner reads the result back. The build role needs kms: Decrypt on the same key to read the agent's code artifact. Objects the microVM writes through presigned links (output.zip, status.json) use the bucket's default encryption: an SSE-KMS bucket default covers them, and the runner credentials then need kms: GenerateDataKey on that key too, as they sign those links.

{"Effect": "Allow", "Action": ["kms: GenerateDataKey", "kms: Decrypt"], "Resource": "arn: aws: kms: <region>: <account>: key/<key id>"}

Build role (buildRoleArn). Lambda assumes this role to build the image. Its trust policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {"Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts: AssumeRole"}
  ]
}

Its permissions policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {"Effect": "Allow", "Action": "s3: GetObject", "Resource": "arn: aws: s3: : : <bucket>/kestra-lambda-microvm/bootstrap/*"},
    {"Effect": "Allow", "Action": ["logs: CreateLogStream", "logs: PutLogEvents"], "Resource": "arn: aws: logs: <region>: <account>: log-group: <logGroup>: *"}
  ]
}

Execution role (executionRoleArn). Required. The runner has the microVM write its logs to CloudWatch, which it reads back to get the command output, and AWS writes those logs as this role. It uses the same trust policy (lambda.amazonaws.com). Its permissions policy needs the log permissions below, plus whatever your own commands need (for example s3: GetObject for aws s3 cp):

{
  "Version": "2012-10-17",
  "Statement": [
    {"Effect": "Allow", "Action": ["logs: CreateLogStream", "logs: PutLogEvents"], "Resource": "arn: aws: logs: <region>: <account>: log-group: <logGroup>: *"}
  ]
}

For a quick setup, one role can serve as both the build and the execution role (same trust policy, merged permissions); separate roles are cleaner, as the build role then carries no permission your commands could use.

In the IAM console, create the roles with the trusted entity "AWS service" and the use case "Lambda".

To use your own image instead, set imageIdentifier (and imageVersion). It must embed the Kestra agent, which is the bootstrap executable listening on 0.0.0.0: 8080 and serving POST /aws/lambda-microvms/runtime/v1/{run,terminate,validate,ready}.

Security note. The agent's hooks are not authenticated. With no ingressNetworkConnectors set, AWS attaches its managed HTTP_INGRESS connector to the microVM. According to the AWS samples, reaching a microVM through that ingress requires a MicroVM auth token (lambda: CreateMicrovmAuthToken), so keep that permission restricted to trusted principals. Do not add other ingress connectors that would expose port 8080: any peer that can reach it could start a job or stop the running one.