Apache Kafka ConnectorUpdateConfig

Apache Kafka ConnectorUpdateConfig

Certified

Update a Kafka Connect connector's configuration

Replaces the entire configuration of an existing connector, restarting its tasks with the new config. Connect creates the connector if it doesn't already exist. Fails with the API's error body verbatim if config fails the connector plugin's own validation.

yaml
type: io.kestra.plugin.kafka.ConnectorUpdateConfig

Bump the task count of an existing connector

yaml
id: kafka_connector_update_config
namespace: company.team

tasks:
  - id: update_connector
    type: io.kestra.plugin.kafka.ConnectorUpdateConfig
    connectUrl: http://connect:8083
    connectorName: orders_jdbc_sink
    config:
      connector.class: io.confluent.connect.jdbc.JdbcSinkConnector
      tasks.max: "3"
      topics: orders
      connection.url: jdbc:postgresql://postgres:5432/orders
      connection.user: "{{ secret('POSTGRES_USER') }}"
      connection.password: "{{ secret('POSTGRES_PASSWORD') }}"
      insert.mode: upsert
      pk.mode: record_key
Properties

Connector configuration

Replaces the connector's entire configuration; must include connector.class and any property required by that connector plugin.

Kafka Connect REST API base URL

For example http://connect: 8083. Kafka Connect has no dedicated Java admin client — every operation goes through this REST API.

Connector name

Assets this task consumes as inputs or produces as outputs, for lineage tracking and the asset graph (Enterprise Edition). A flow declaring this property on a task is rejected in the open-source edition.

Definitions
assetFailureBehaviorstring
Possible Values
IGNOREFAILWARN

Asset failure behavior

Behavior applied to the task state when a declared asset fails to render, emit, or be persisted (e.g. a lock conflict): FAIL escalates it to FAILED, WARN (default) warns it if it would otherwise succeed, IGNORE leaves the state untouched.

enableAutobooleanstring

Whether to auto-register assets referenced dynamically at runtime that are not statically declared in inputs or outputs.

inputsarray

The assets consumed as inputs.

id*string
Min length1
typestring
outputs

The assets produced as outputs.

id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*object
descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150
id*string
Min length1
Max length150
type*string
Min length1

Custom asset type

descriptionstring
displayNamestring
metadataobject
Default{}
namespacestring
Min length1
Max length150

Additional HTTP headers

Sent on every request to the Connect REST API. Useful when the worker sits behind a reverse proxy or expects a bearer token, e.g. Authorization: Bearer ....

Basic auth password

Basic auth username

Required together with password when the Connect REST API is protected with HTTP basic auth. Leave both unset to call an unauthenticated worker — no Authorization header is sent in that case.

SubTypestring

Effective connector configuration, as accepted by Connect

Connector name

SubTypeinteger

Task ids provisioned by Connect for this connector