DocsUse Cases

​Coordinate ​Microservices

Data and engineering teams often rely on microservices to keep their systems modular, fault-tolerant, and flexible. Each microservice handles a single task, making it easier to develop, scale, and maintain. The main challenge is orchestrating these independent components so they run in the correct order, automatically recover from failures, and scale as needed.

What is Microservices Orchestration?

Microservices orchestration is the automated coordination of services, often triggered by events such as a file upload to S3/SFTP, a message in Kafka/PubSub, or a new database row. A platform like Kestra defines the sequence of these services, manages their dependencies, and ensures reliable execution—whether triggered manually, via API calls, webhooks, schedules, or by completion of upstream workflows.

Kestra can:

  • Trigger your microservices from any event, schedule, or flow dependency
  • Pass data of any size between services thanks to built-in internal storage
  • Dynamically provision task runner environments, so your services have enough compute resources
  • Retry failed services, keeping workflows robust and fault-tolerant
  • Send alerts or notifications on success or failure
  • Track logs, metrics, inputs, and outputs of each service execution
  • Roll back to earlier workflow revisions as needed.

Why Use Kestra for Microservices Orchestration?

  1. Visibility – View dependencies, see which service failed or succeeded, then restart or roll back as needed.
  2. Simplicity – Declare dependencies in YAML or use Kestra’s UI with no-code/low-code options.
  3. Scalability – Run microservices in parallel and scale compute resources based on workload.
  4. Resilience – If one service fails, Kestra can retry just that part instead of re-running the entire workflow.
  5. Zero Code Changes – Keep your existing code as-is and add minimal YAML on top to orchestrate it.
  6. Extensibility – Add new triggers, tasks, runners, or notifications through Kestra’s plugin system.
  7. Security and Compliance – Manage secrets, access, encryption, and audit logs within Kestra.
  8. Version Control – Keep orchestration configurations in Git and revert to previous versions if needed.
  9. Multi-Tenancy – Use separate tenants/namespaces for different teams or projects, each with its own variables and secrets.
  10. Open-Source Core – Ask questions in our Slack community, report issues on GitHub, contribute to the codebase — all with no vendor lock-in.

Example: Microservices Orchestration in Kestra

Below is a minimal Kestra flow for an e-commerce order-processing workflow. It checks inventory, processes payment, confirms the order, arranges shipping, and updates delivery status. Each task waits for a successful response before triggering the next service, passing data and determining the next step based on the status code of the prior API call.

yaml
id: orderProcessing
namespace: ecommerce
description: E-commerce Order Processing Workflow

inputs:
  - id: orderId
    type: STRING
    defaults: myorder

tasks:
  - id: checkInventory
    type: io.kestra.plugin.core.http.Request
    description: Check inventory for the order items
    uri: https://reqres.in/api/inventory-service/check

  - id: processPayment
    type: io.kestra.plugin.core.http.Request
    runIf: "{{ outputs.checkInventory.code == 201 }}"
    description: Process payment for the order
    uri: https://reqres.in/api/payment-service/process

  - id: orderConfirmation
    type: io.kestra.plugin.core.http.Request
    runIf: "{{ outputs.processPayment.code == 201 }}"
    description: Confirm the order and notify the customer
    uri: https://reqres.in/api/order-service/confirm

  - id: arrangeShipping
    type: io.kestra.plugin.core.http.Request
    runIf: "{{ outputs.orderConfirmation.code == 201 }}"
    description: Arrange shipping for the order
    uri: https://reqres.in/api/shipping-service/arrange

  - id: updateDeliveryStatus
    type: io.kestra.plugin.core.http.Request
    runIf: "{{ outputs.arrangeShipping.code == 201 }}"
    description: Update the delivery status of the order
    uri: https://reqres.in/api/delivery-service/updateStatus

pluginDefaults:
  - type: io.kestra.plugin.core.http.Request
    values:
      contentType: multipart/form-data
      method: POST
      formData:
        orderId: "{{inputs.orderId}}"

Getting Started with Microservice Orchestration in Kestra

  1. Install Kestra – Follow the quick start guide or the full installation instructions for production environments.
  2. Write Your Workflows – Configure your flow in YAML. Each task can invoke an API, run scripts, or call any existing service.
  3. Add Triggers – Use scheduled or event-based triggers to start microservice workflows.
  4. Observe and Manage – Use Kestra’s UI to monitor states, logs, and metrics. Rerun failed workflow executions or roll back with one click.

Next Steps

Was this page helpful?