​Document ​Your ​Plugin

Here is how you can document your plugin.

First, let us remember the organization of a plugin project:

• The Gradle project can contain several plugins, we call it a group of plugins.
• The package in which a plugin is written in is called a sub-group of plugins. Sometimes, there is only one sub-group, in which case the group and the sub-group are the same.
• Each class is a plugin that provides one task, trigger, condition, etc.

The plugin documentation will be used on the Kestra website and the Kestra UI.

We provide a way to document each level of a plugin project.

Document the plugin group

Manifest attributes

Kestra uses custom manifest attributes to provide top-level group documentation.

The following manifest attributes are used to document the group of plugins:

• X-Kestra-Title: by default, the Gradle project.name property is used.
• X-Kestra-Group: by default, the Gradle group.id property with an additional group name is used.
• X-Kestra-Description: by default, the Gradle project.description property is used.
• X-Kestra-Version: by default, the Gradle project.version property is used.

If you follow the plugin structure of the template on GitHub, you should have something like this:

As you can see, the most important documentation attribute is the description, which should be a short sentence describing the plugins.

Additional markdown files

You can add additional markdown files in the src/main/resources/doc directory.

If there is a file src/main/resources/doc/<plugin-group>, it will be inlined inside the main documentation page as the long description for the group of plugins.

For example, for the GCP group of plugins, the file is src/main/resources/doc/io.kestra.plugin.gcp, and it contains authentication information that applies to all tasks.

If there are files inside the src/main/resources/doc/guides directory, we will list them in a Guides section on the documentation for the group of plugins.

Group Icon

It is possible to provide an icon representing the whole plugin group. If there is a SVG file src/main/resources/icons/plugin-icon.svg, it will be used as the group icon.

Document the plugin sub-groups

Each sub-group can be documented via the io.kestra.core.models.annotations.PluginSubGroup annotation that must be defined at the package level in a package-info.java file.

The @PluginSubGroup annotation allows setting:

• The sub-group title. If not set, the name of the sub-group will be used.
• The sub-group description, which is a short sentence introducing the sub-group.
• The sub-group categories, which is a list of PluginCategory. If not set, the category MISC will be used.

For example, for the GCP BigQuery sub-group:

@PluginSubGroup(
    title = "BigQuery",
    description = "This sub-group of plugins contains tasks for accessing Google Cloud BigQuery.\n" +
        "BigQuery is a completely serverless and cost-effective enterprise data warehouse.",
    categories = { PluginSubGroup.PluginCategory.DATABASE, PluginSubGroup.PluginCategory.CLOUD }
)
package io.kestra.plugin.gcp.bigquery;

import io.kestra.core.models.annotations.PluginSubGroup;

Sub-Group Icon

Each plugin sub-group can define an icon representing plugins contained in the sub-group. If there is a SVG file src/main/resources/icons/<plugin-sub-group>.svg, it will be used as the icon for the corresponding plugins.

For example, for the GCP BigQuery sub-group, the src/main/resources/icons/io.kestra.plugin.gcp.bigquery.svg file is used.

Document each plugin

Plugin documentation will generate a JSON Schema that will be used to validate flows. It also generates documentation for both the UI and the website (see the kestra plugins doc command).

Document the plugin class

Each plugin class must be documented via the following:

• The io.kestra.core.models.annotations.Plugin annotation allows providing examples.
• The io.swagger.v3.oas.annotations.media.Schema annotation, which the title attribute will use as the plugin description.

For example, the Query task of the PostgreSQL group of plugins is documented as follows:

@Schema(
    title = "Query a PostgresSQL server."
)
@Plugin(
    examples = {
        @Example(
            full = true,
            title = "Execute a query.",
            code = """
                id: query_postgres
                namespace: company.team

                tasks:
                  - id: query
                    type: io.kestra.plugin.jdbc.postgresql.Query
                    url: jdbc:postgresql://127.0.0.1:56982/
                    username: pg_user
                    password: pg_password
                    sql: |
                      select concert_id, available, a, b, c, d, play_time, library_record, floatn_test, double_test, real_test, numeric_test, date_type, time_type, timez_type, timestamp_type, timestampz_type, interval_type, pay_by_quarter, schedule, json_type, blob_type from pgsql_types
                    fetchType: FETCH
                """
        )
    }
)

For convenience, the code attribute of the @Example annotation is a list of strings. Each string will be a line of the example. That avoids concatenating multi-line strings in a single attribute.

You can add multiple examples if needed.

Document the plugin properties

In a plugin, all properties must be annotated by io.kestra.core.models.annotations.PluginProperty and should provide documentation via the io.swagger.v3.oas.annotations.media.Schema annotation and validation rules via javax.validation.constraints.*.

The @PluginProperty annotation contains two attributes:

• dynamic: set it to true if the property will be rendered dynamically.
• additionalProperties: you can use it to denote the sub-type of the property. For example, when using a Map<String, Message>, you can set it to Message.class.

The Swagger @Schema annotation contains a lot of attributes that can be used to document the plugin properties. The most useful are:

• title: a short description of a property.
• description: long description of a property.
• anyOf: a list of allowed sub-types of a property. Use it when the property type is an interface, an abstract class, or a class inside a hierarchy of classes to denote possible sub-types. This should be set when the property type is Object.

The @Schema and @PluginProperty annotations can be used on fields, methods, or classes.

Many tasks can take input from multiple sources on the same property. They usually have a single from property, a string representing a file in the Kestra Storage, a single object, or a list of objects. To document such property, you can use anyOf this way:

@PluginProperty(dynamic = true)
@NotNull
@Schema(
    title = "The source of the published data.",
    description = "Can be an internal storage URI, a list of Pub/Sub messages, or a single Pub/Sub message.",
    anyOf = {String.class, Message[].class, Message.class}
)
private Object from;

Document the plugin outputs

Outputs should be documented with the io.swagger.v3.oas.annotations.media.Schema annotation in the same way as plugin properties. Please read the section above for more information.

Only use the annotation mentioned above. Never use @PluginProperty on an output.

Document the plugin metrics

Tasks can expose metrics; to document those you must add a @Metric annotation instance for each metric in the @Plugin annotation instance of the task.

For example, to document two metrics: a length metric of type counter and a duration metric of type timer, you can use the following:

@Plugin(
    metrics = {
        @Metric(name = "length", type = Counter.TYPE),
        @Metric(name = "duration", type = Timer.TYPE)
    }
)

JSON Schema Usage for Flow Validation

Kestra provides a JSON Schema to validate your flow definitions. This ensures that your flows are correctly structured and helps catch errors early in the development process.

JSON Schema in VSCode

To use the JSON Schema in Visual Studio Code (VSCode), follow these steps:

1. Install the YAML extension by Red Hat.
2. Open your VSCode settings (Ctrl+, or Cmd+,).
3. Search for YAML: Schemas and click on Edit in settings.json.
4. Add the following configuration to associate the Kestra JSON Schema with your flow files:

{
  "yaml.schemas": {
    "https://your-kestra-instance.com/api/v1/schemas/flow.json": "/*.yaml"
  }
}

Replace https://your-kestra-instance.com/api/v1/schemas/flow.json with the actual URL of your Kestra JSON Schema.

Example of Using JSON Schema in Flow Editor

Here is an example of how to use the JSON Schema in the flow editor:

id: example_flow
namespace: example_namespace
tasks:
  - id: example_task
    type: io.kestra.core.tasks.log.Log
    message: "Hello, World!"

When you open this flow in the editor, the JSON Schema will validate the structure and provide autocompletion and error checking.

Globally Available Location for JSON Schema

The JSON Schema for Kestra flows is available at the following URL:

https://your-kestra-instance.com/api/v1/schemas/flow.json

Replace https://your-kestra-instance.com with the actual URL of your Kestra instance.

Generating and Using JSON Schema for Plugins

To generate a JSON Schema for your plugin, you can use the kestra plugins doc command. This command will generate a JSON Schema file for your plugin, which can be used to validate your plugin's configuration.

Here is an example of how to use the kestra plugins doc command:

kestra plugins doc --plugin io.kestra.plugin.yourplugin --output yourplugin-schema.json

This will generate a JSON Schema file named yourplugin-schema.json for your plugin.

You can then use this JSON Schema file to validate your plugin's configuration in the same way as the flow JSON Schema.