​Main components

Technical overview of Kestra’s main components: internal storage, queue, repository, and plugins.

Kestra relies on the following internal components:

• Internal storage: stores flow data such as task outputs and flow inputs.
• Queue: enables internal communication between Kestra server components.
• Repository: persists flows, templates, executions, logs, and all other internal objects.
• Plugins: extend Kestra’s core with additional task and trigger types, storage implementations, and data transformations.

Each component has multiple implementations depending on deployment architecture. Some require additional plugins.

Internal storage

The internal storage is a dedicated system that handles files of any size during flow executions. It manages both inputs and outputs, enabling scalable file sharing between tasks.

Purpose

Internal storage is used to:

• Save files generated during a flow’s execution and pass them between tasks via outputs.
• Automatically persist flow inputs of type FILE.
• Provide download links for stored files in the Outputs tab of an execution.

Files can be retrieved in the execution context using {{ outputs.task_id.output_attribute }} (often the uri property). Kestra fetches the file automatically when referenced.

Execution metadata — including storage file paths — is recorded in the repository.

Storage types

By default, Kestra uses local storage, which stores files on the host filesystem. This option is simple but not scalable and is not recommended for production.

Scalable alternatives are available as plugins:

• Storage MinIO — supports MinIO, AWS S3, and other S3-compatible systems.
• Storage GCS — for Google Cloud Storage.
• Storage Azure — for Azure Blob Storage.

For details, see Internal storage configuration.

Queue

The queue is used internally for communication between Kestra’s server components. Each repository type has a matching queue implementation:

• In-memory queue — must be used with the in-memory repository.
• Database queue — must be used with the database repository.
• Kafka queue — must be used with the Elasticsearch repository.
  Only available in the Enterprise Edition.

Repository

The repository persists all internal objects, including flows, executions, logs, and templates. Each repository type must be paired with its corresponding queue:

• In-memory repository — must be used with the in-memory queue.
• Database repository — must be used with the database queue.
• Elasticsearch repository — must be used with the Kafka queue.
  Only available in the Enterprise Edition.

Plugins

Kestra’s core only provides basic functionality. A plugin ecosystem extends the platform with:

• New task and trigger types.
• Alternative implementations of core components (e.g., storage backends).
• Integrations with external systems and data transformation utilities.

A wide range of plugins is already available, and the ecosystem continues to expand.