Apache Kafka ConnectorCreate

Apache Kafka ConnectorCreate
Certified
Create a Kafka Connect connector

Submits a new connector to the Kafka Connect REST API. Fails with the API's error body verbatim (HTTP 409) if a connector with the same name already exists, and (HTTP 400/500) if config fails the connector plugin's own validation.

yaml
type: io.kestra.plugin.kafka.ConnectorCreate

Examples

Create a JDBC sink connector

yaml
id: kafka_connector_create
namespace: company.team

tasks:
  - id: create_connector
    type: io.kestra.plugin.kafka.ConnectorCreate
    connectUrl: http://connect:8083
    connectorName: orders_jdbc_sink
    config:
      connector.class: io.confluent.connect.jdbc.JdbcSinkConnector
      tasks.max: "1"
      topics: orders
      connection.url: jdbc:postgresql://postgres:5432/orders
      connection.user: "{{ secret('POSTGRES_USER') }}"
      connection.password: "{{ secret('POSTGRES_PASSWORD') }}"
      insert.mode: upsert
      pk.mode: record_key

Properties

config *object

Connector configuration

Must include connector.class and any property required by that connector plugin (e.g. topics, tasks.max, connection settings). Left empty, Connect rejects the request with its own validation error.

connectUrl *string

Kafka Connect REST API base URL

For example http://connect: 8083. Kafka Connect has no dedicated Java admin client — every operation goes through this REST API.

connectorName *string

Connector name

headers object

Additional HTTP headers

Sent on every request to the Connect REST API. Useful when the worker sits behind a reverse proxy or expects a bearer token, e.g. Authorization: Bearer ....

password string

Basic auth password

pluginDefaultsRef string

Reference (ref) of the pluginDefaults to apply to this task.

username string

Basic auth username

Required together with password when the Connect REST API is protected with HTTP basic auth. Leave both unset to call an unauthenticated worker — no Authorization header is sent in that case.

Outputs

config object

SubTypestring

Effective connector configuration, as accepted by Connect

connectorName string

Connector name

taskIds array

SubTypeinteger

Task ids provisioned by Connect for this connector