Versioned Plugins
Available on: Enterprise EditionBETA0.22.0
Use multiple versions of a plugin depending on your instance requirements and upgrade path.
Versioned plugins are an enterprise feature that simplifies the upgrade process. They allow you to pin older plugin versions to your production and legacy flows while using the latest version for newer flows, enabling granular version management in your Kestra instance.
Configuration
Versioned plugins support several properties that can be modified in your Kestra configuration:
remoteStorageEnabled
: Specifies whether remote storage is enabled (i.e., plugins are stored on the internal storage).localRepositoryPath
: The local path where managed plugins will be synced.autoReloadEnabled
: The interval at which the Kestra server checks for new or removed plugins.autoReloadInterval
: The default version to be used when no version is specified for a plugin.defaultVersion
: Accepted are: 'latest', 'current', 'oldest', 'none', or a specific version (e.g., 0.20.0)
An example configuration looks as follows:
kestra:
plugins:
management:
enabled: true # setting to false will make Versioned plugin tab disappear + API will return an error
remoteStorageEnabled: true
customPluginsEnabled: true # setting to false will disable installing or uploading custom plugins
localRepositoryPath: /tmp/kestra/plugins-repository
autoReloadEnabled: true
autoReloadInterval: 60s
defaultVersion: LATEST
With remote storage enabled, installed plugins are stored in a plugins repository in the _plugins/repository
path. For example, the below paths show the storage for 0.19.0 and 0.20.0 versions of the Shell script plugin:
_plugins/repository/io_kestra_plugin__plugin-script-shell__0_19_0
_plugins/repository/io_kestra_plugin__plugin-script-shell__0_19_0.jar
_plugins/repository/io_kestra_plugin__plugin-script-shell__0_20_0
_plugins/repository/io_kestra_plugin__plugin-script-shell__0_20_0.jar
Artifact files are renamed using the format: <groupId>__<artifactId>__<version>
to be easily parseable (dots .
are replaced with _
for groupId
and version
).
For locally stored plugins configured by the localRepositoryPath
attribute, the file path looks like /tmp/kestra/plugins-repository
. For example, the following plugins are stored locally, where the local repository contains a JSON plugins.meta
file that contains metadata about remote plugins. This file is used for synchronization, where only plugins with detected changes are synchronized.
├── io_kestra_plugin__plugin-kafka__0_20_0.jar
├── io_kestra_plugin__plugin-script-shell__0_20_0.jar
├── io_kestra_plugin__plugin-terraform__0_20_0.jar
├── io_kestra_plugin__plugin-transform-grok__0_20_0.jar
└── plugins.meta
Install versioned plugins
Versioned plugins can be installed from the Kestra UI as well as programmatically.
From the UI
Both Kestra official plugins and custom plugins can be installed from the UI. Navigate to the Administration > Instance section and then Versioned Plugins. You can click + Install and open up the full library of available plugins.
From the list, search and select the plugin to install and select the version.
After installing plugins, the full list of versioned plugins is displayed. Kestra alerts you that a newer version of your plugin is available and allows you to upgrade by installing the latest version. When upgrading, the previous version of the plugin is preserved, and a separate, fresh installation of the latest version is added.
For a custom plugin, after clicking + Install, switch from Official plugin to Custom plugin. You need to specify two identifiers for each custom plugin installation:
- Group ID: The group identifier of the plugin to be installed.
- Artifact ID: The artifact identifier of the plugin to be installed.
Instead of installing a new plugin, you can Upload a plugin by choosing a valid Java archive file (.jar
).
From the API
Only Super Admin users can install versioned plugins with the API. To install a versioned plugin, you can use the API POST request with your username and password with -u
or an API token.
With Kestra username and password:
curl -X POST http://0.0.0.0:8080/api/v1/cluster/versioned-plugins/install \
-u '[email protected]:kestra' \
-H "Content-Type: application/json" \
-d '{"plugins":["io.kestra.plugin:plugin-airbyte:0.21.0"]}'
With API Token:
curl -X POST http://0.0.0.0:8080/api/v1/cluster/versioned-plugins/install /
-H "Authorization: Bearer YOUR-API-TOKEN" \
-H "Content-Type: application/json" \
-d '{"plugins":["io.kestra.plugin:plugin-airbyte:0.21.0"]}'
To uninstall a versioned plugin, use the following DELETE request:
curl -X DELETE http://0.0.0.0:8080/api/v1/cluster/versioned-plugins/uninstall \
-u '[email protected]:kestra' \
-H "Content-Type: application/json" \
-d '{"plugins":["io.kestra.plugin:plugin-airbyte:0.21.0"]}'
To check for all available versions of a plugin, you can use the following API request to resolve:
curl -X POST http://0.0.0.0:8080/api/v1/cluster/versioned-plugins/resolve \
-u '[email protected]:kestra' \
-H "Content-Type: application/json" \
-d '{"plugins":["io.kestra.plugin:plugin-airbyte:0.21.0"]}'
If you want to install a newer plugin version, use the install request with the specified version or use LATEST
instead of the version number. This creates a second, separate installation of the plugin, so you can keep using an old version in production flows and test using the newer version in development.
curl -X POST http://0.0.0.0:8080/api/v1/cluster/versioned-plugins/install \
-u '[email protected]:kestra' \
-H "Content-Type: application/json" \
-d '{"plugins":["io.kestra.plugin:plugin-airbyte:LATEST"]}'
From the CLI
To install versioned plugins from the Kestra CLI, you can use the following command:
./kestra plugins install --locally=false io.kestra.plugin:plugin-jdbc-duckdb:0.21.2
The --locally
flag specifies whether the plugin should be installed locally or according to your Kestra configuration, where remote storage can be enabled.
--locally=true
installs the plugin locally.--locally=false
checks ifremoteStorageEnabled
is enabled and then plugins are downloaded and pushed to the configured internal storage directly.
version
property in a Flow
In Flow tasks or triggers, you can specify the version of the plugin to use with the version
property. For example, if the instance has both 0.22.0 and 0.21.0 versions installed of the Shell script plugin, the version to use can be specified in the flow as follows:
id: legacy_shell_script
namespace: company.team
tasks:
-id: script
type: io.kestra.plugin.scripts.shell.Script
version: 0.21.0
The version
property also accepts specific, non-case-sensitive values like in the configuration file:
LATEST
(orlatest
): To use the latest available version of a Kestra plugin.OLDEST
(oroldest
): To use the oldest available version of a Kestra plugin.
When there are multiple versions of a plugin available, Kestra resolves the version of a plugin by following this priority order:
- Task-Level: Using the version specified in the
version
property. - Flow-Level: Using the plugin’s default version.
- Namespace-Level: Using the plugin’s default version for the namespace.
- Instance-Level: Using the value set in
kestra.plugins.management.defaultVersion
(default:LATEST
).- This property can be configured to
NONE
to enforce that a version is always explicitly defined.
- This property can be configured to
Note: By default, Kestra defaults to LATEST
for core plugins if no version can be resolved. For other plugins, if no version can be resolved, the Flow will be considered invalid.
The version is resolved both at flow creation time and execution time to ensure the correct plugin version is used during both stages. This means that a Task/Trigger can only be deserialized after ensuring that all default versions are properly resolved.
Was this page helpful?