DocsArchitectureMain components

Main components​Main components

Was this page helpful?

Technical description of Kestra's main components, including the internal storage, queue, repository, and plugins.

Kestra has three internal components:

  • The Internal Storage stores flow data like task outputs and flow inputs.
  • The Queue is used for internal communication between Kestra server components.
  • The Repository is used to store flows, templates, executions, logs, etc. The repository stores every internal object.
  • The Plugins extend the core of Kestra with new task and trigger types, storage implementations, etc.

These internal components are provided on multiple implementations depending on your needs and deployment architecture. You may need to install additional plugins to use some implementations.

Internal Storage

Kestra uses the concept of Internal Storage for storing input and output data. Multiple storage options are available, including local storage (default), S3 and Minio, Google Cloud Storage, and Azure Blobs Storage.

Check the Internal Storage documentation for more information.

Queue

The Queue, or more precisely, queues, are used internally for communication between the different Kestra server components. Kestra provides multiple queue types that must be used with their repository counterparts.

There are three types of queues:

  • In-Memory that must be used with the In-Memory Repository.
  • Database that must be used with the Database Repository.
  • Kafka that must be used with the Elasticsearch Repository. Only available in the Enterprise Edition.

Repository

The Repository, or more precisely, repositories, are the internal way to store data. Kestra provides multiple repository types that must be used with their queue counterparts.

There exist three types of repositories:

  • In-Memory that must be used with the In-Memory Queue.
  • Database that must be used with the Database Queue.
  • Elasticsearch that must be used with the Kafka Queue. Only available in the Enterprise Edition.

Plugins

Kestra's core is not able to handle a lot of task types on its own. We have therefore, included a Plugins' ecosystem that allows developing as many task types as you need. A wide range of plugins are already available, and many more will be delivered by the Kestra team!

Plugins are also used to provide different implementations for Kestra's internal components like its Internal Storage.