​Server components

Detailed breakdown of the server components behind Kestra.

Kestra consists of multiple server components that can be scaled independently.

Each server component interacts with internal components (internal storage, queues, and repositories).

Executor

The Executor oversees all executions and certain types of tasks, such as Flowable tasks or Flow Triggers. It requires minimal resources as it doesn't perform heavy computations.

Generally speaking, the Executor never touches your data. It orchestrates workflows based on the information it receives from the Scheduler and the Queue, and it defers the execution of tasks to Workers.

If you have a large number of Executions, you can scale the Executor horizontally. However, this is rarely necessary as the Executor is very lightweight — all heavy computations are performed by Workers.

Worker

Workers execute tasks (from the Executor) and polling triggers (from the Scheduler).

Workers are highly configurable and scalable, accommodating a wide range of tasks from simple API calls to complex computations. Workers are the only server components that need access to external services in order to connect to databases, APIs, or other services that your tasks interact with.

Worker Group (EE)

In the Enterprise Edition, Worker Groups allow tasks and polling triggers to be executed on specific worker sets. They can be beneficial in various scenarios, such as using compute instances with GPUs, executing tasks on a specific OS, restricting backend access, and region-specific execution. A default worker group is recommended per tenant or namespace.

To specify a worker group for a task, use the workerGroup.key property in the task definition to point the task to a specific worker group key. If no worker group is specified, the task will be executed on the default worker group.

Scheduler

The Scheduler manages all triggers except for Flow triggers handled by the Executor. It determines when to start a flow based on trigger conditions.

Indexer

The Indexer indexes content from Kafka topics (such as the flows and executions topics) to Elasticsearch repositories.

Webserver

The Webserver is the entry point for all external communications with Kestra. It handles all REST API calls made to Kestra and serves the Kestra UI.