Parallel icon
Invoke icon
Commands icon
Process icon

Microservice orchestration: invoke multiple AWS Lambda functions in parallel

Orchestrate AWS Lambda with Kestra. Invoke functions in parallel by ARN, target versions and aliases, pass custom payloads, and parse JSON results with jq.

Categories
Cloud

Stitch serverless functions into a real pipeline. This blueprint invokes several AWS Lambda functions concurrently, addresses them by ARN (including a published version and a named alias), passes a custom event payload to each, and then reads the returned JSON to extract the data you actually care about. It solves the common problem of Lambda functions that exist as isolated, event driven endpoints: instead of chaining them by hand or wiring up bespoke glue code, you coordinate them declaratively and capture their outputs in one place.

How it works

  • A io.kestra.plugin.core.flow.Parallel task fans out three Lambda invocations so they run at the same time.
  • io.kestra.plugin.aws.lambda.Invoke (lambda) calls a function by its functionArn.
  • A second Invoke (lambda_version) targets a pinned function version via an ARN suffix and sends a functionPayload dictionary (your_event_input: hello).
  • A third Invoke (lambda_alias) targets a named alias and sends its own functionPayload.
  • A final io.kestra.plugin.scripts.shell.Commands task (lambda_result) runs on the io.kestra.plugin.core.runner.Process runner and uses cat plus jq -r '.body' to read {{ outputs.lambda.uri }} and pull the response body out of the JSON.

What you get

  • Parallel, low latency invocation of several functions in a single run.
  • Precise targeting of a Lambda version or alias, not just $LATEST.
  • Typed, declarative event payloads passed as simple dictionaries.
  • Captured JSON outputs you can parse, branch on, or feed downstream.

Who it's for

  • Backend and platform engineers orchestrating serverless microservices.
  • Data and ML teams triggering Lambda based processing steps.
  • DevOps teams that need auditable, repeatable Lambda runs.

Why orchestrate this with Kestra

AWS Lambda has no native cross function workflow engine: each function fires on its own event with no shared run history. Kestra adds event triggers, automatic retries, full execution lineage across every invocation, and a single declarative YAML definition for the whole sequence. You get one place to schedule, observe, retry, and audit calls that the Lambda console alone cannot coordinate.

Prerequisites

  • An AWS account with one or more deployed Lambda functions.
  • IAM credentials with lambda:InvokeFunction permission for the target ARNs.
  • The function ARNs (and any version or alias suffixes) you want to call.

Secrets

This blueprint uses inline placeholder ARNs and does not reference any {{ secret('NAME') }} values. For production, store AWS credentials as secrets and reference them from the task (for example {{ secret('AWS_ACCESS_KEY_ID') }}, {{ secret('AWS_SECRET_KEY_ID') }}, and {{ secret('AWS_DEFAULT_REGION') }}) instead of hardcoding them.

Quick start

  1. Deploy a demo function and a ResultHandler function in AWS (sample Python handlers below).
  2. Optionally publish a version and create an alias (for example kestra) on ResultHandler.
  3. Replace the placeholder functionArn values with your real ARNs, including region and account id.
  4. Run the flow and inspect the lambda_result logs for the parsed response body.

Sample demo handler:

import json

def lambda_handler(event, context):
    print(event)
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

Sample ResultHandler handler:

import json

def lambda_handler(event, context):
    function_result = event['your_event_input']
    return {
        'body': json.dumps(function_result + ' from Kestra')
    }

How to extend

  • Add more Invoke tasks inside the Parallel block to fan out to additional functions.
  • Swap the Parallel block for sequential tasks if one function depends on another's output.
  • Attach a Schedule or webhook trigger to invoke your functions on a cadence or in reaction to an event.
  • Feed the parsed body into downstream tasks such as a database load, a notification, or another Lambda call.

Links

Orchestrate with Kestra
Orchestrate AWS with Kestra
Share this Blueprint
See How

New to Kestra?

Use blueprints to kickstart your first workflows.