Document your plugin
First, let us remember the organization of a plugin project:
- The Gradle project can contain several plugins, we call it a group of plugins.
- The package in which a plugin is is called a sub-group of plugins. Sometimes there is only one sub-group, so the group and the sub-group are the same.
- Each class is a plugin that provides one task, trigger, condition, etc.
The plugin documentation will be used on the Kestra website and the Kestra UI.
We provide a way to document each level of a plugin project.
Document the plugin group
Manifest attributes
Kestra uses custom manifest attributes to provide top-level group documentation.
The following manifest attributes are used to document the group of plugins:
X-Kestra-Title
: by default, the Gradleproject.name
property is used.X-Kestra-Group
: by default, the Gradlegroup.id
property with an additional group name is used.X-Kestra-Description
: by default, the Gradleproject.description
property is used.X-Kestra-Version
: by default, the Gradleproject.version
property is used.
If you follow the plugin structure of the GitHub template, you should have something like this:
group "io.kestra.plugin"
description 'Google Cloud Platform (GCP) plugins for Kestra.'
// [...]
jar {
manifest {
attributes(
"X-Kestra-Title": project.name,
"X-Kestra-Group": project.group + ".gcp",
"X-Kestra-Description": project.description,
"X-Kestra-Version": project.version
)
}
}
As you can see, the most important documentation attribute is the description
, which should be a short sentence describing the plugins.
Additional markdown files
You can add additional markdown files in the src/main/resources/doc
directory.
If there is a file src/main/resources/doc/<plugin-group>.md
, it will be inlined inside the main documentation page as the long description for the group of plugins.
For example, for the GCP group of plugins, the file is src/main/resources/doc/io.kestra.plugin.gcp.md
, and it contains authentication information that applies to all tasks.
If there are files inside the src/main/resources/doc/guides
directory, we will list them in a Guides
section on the documentation for the group of plugins.
Document the plugin sub-groups
Each sub-group can be documented via the io.kestra.core.models.annotations.PluginSubGroup
annotation that must be defined at the package level in a package-info.java
file.
The @PluginSubGroup
annotation allows setting:
- The sub-group
title
. If not set, the name of the sub-group will be used. - The sub-group
description
, which is a short sentence introducing the sub-group. - The sub-group
categories
, which is a list ofPluginCategory
. If not set, the categoryMISC
will be used.
For example, for the GCP BigQuery sub-group:
@PluginSubGroup(
title = "BigQuery",
description = "This sub-group of plugins contains tasks for accessing Google Cloud BigQuery.\n" +
"BigQuery is a completely serverless and cost-effective enterprise data warehouse.",
categories = { PluginSubGroup.PluginCategory.DATABASE, PluginSubGroup.PluginCategory.CLOUD }
)
package io.kestra.plugin.gcp.bigquery;
import io.kestra.core.models.annotations.PluginSubGroup;
Document each plugin
Plugin documentation will generate a JSON Schema that will be used to validate flows. It also generates documentation for both the UI and the website (see the kestra plugins doc
command).
Document the plugin class
Each plugin class must be documented via the following:
- The
io.kestra.core.models.annotations.Plugin
annotation allows providing examples. - The
io.swagger.v3.oas.annotations.media.Schema
annotation, which thetitle
attribute will use as the plugin description.
For example, the Query
task of the PostgreSQL group of plugins is documented as follows:
@Schema(
title = "Query a PostgresSQL server"
)
@Plugin(
examples = {
@Example(
full = true,
title = "Execute a query",
code = {
"tasks:",
"- id: update",
" type: io.kestra.plugin.jdbc.postgresql.Query",
" url: jdbc:postgresql://127.0.0.1:56982/",
" username: postgres",
" password: pg_passwd",
" sql: select concert_id, available, a, b, c, d, play_time, library_record, floatn_test, double_test, real_test, numeric_test, date_type, time_type, timez_type, timestamp_type, timestampz_type, interval_type, pay_by_quarter, schedule, json_type, blob_type from pgsql_types",
" fetch: true"}
)
}
)
For convenience, the code
attribute of the @Example
annotation is a list of strings. Each string will be a line of the example. That avoids concatenating multi-line strings in a single attribute.
You can add multiple examples if needed.
Document the plugin properties
In a plugin, all properties must be annotated by io.kestra.core.models.annotations.PluginProperty
and should provide documentation via the io.swagger.v3.oas.annotations.media.Schema
annotation and validation rules via javax.validation.constraints.*
.
Please check the Properties validation documentation page for validation rules.
The @PluginProperty
annotation contains two attributes:
dynamic
: set it to true if the property will be rendered dynamically. See Render Variable.additionalProperties
: you can use it to denote the sub-type of the property. For example, when using aMap<String, Message>
, you can set it toMessage.class
.
The Swagger @Schema
annotation contains a lot of attributes that can be used to document the plugin properties. The most useful are:
title
: a short description of a property.description
: long description of a property.anyOf
: a list of allowed sub-types of a property. Use it when the property type is an interface, an abstract class, or a class inside a hierarchy of classes to denote possible sub-types. This should be set when the property type isObject
.
The @Schema
and @PluginProperty
annotations can be used on fields, methods, or classes.
Many tasks can take input from multiple sources on the same property. They usually have a single from
property, a string representing a file in the Kestra Storage, a single object, or a list of objects. To document such property, you can use anyOf
this way:
@PluginProperty(dynamic = true)
@NotNull
@Schema(
title = "The source of the published data.",
description = "Can be an internal storage URI, a list of Pub/Sub messages, or a single Pub/Sub message.",
anyOf = {String.class, Message[].class, Message.class}
)
private Object from;
::alert{type="info"} NOTE
Due to limitations on how JSON schema works, you cannot add @Schema
on a Java enum type and the plugin property that uses this type. We advise avoiding using @Schema
on enumerations.
::
Document the plugin outputs
Outputs should be documented with the io.swagger.v3.oas.annotations.media.Schema
annotation in the same way as plugin properties. Please read the section above for more information.
Only use the annotation mentioned above. Never use @PluginProperty
on an output.
Document the plugin metrics
Tasks can produce metrics; to document those you must add a @Metric
annotation instance for each metric in the @Plugin
annotation instance of the task.
For example, to document two metrics: a length metric of type counter
and a duration metric of type
timer, you can use the following:
@Plugin(
metrics = {
@Metric(name = "length", type = Counter.TYPE),
@Metric(name = "duration", type = Timer.TYPE)
}
)