Apache Kafka TopicUpdate

Apache Kafka TopicUpdate

Certified

Update a Kafka topic's configuration

Incrementally alters an existing topic's configuration, typically retention.ms or retention.bytes to enforce a per-tenant data retention policy. Only the provided configs are changed; other configs are left untouched. Fails with UnknownTopicOrPartitionException if the topic does not exist.

yaml
type: io.kestra.plugin.kafka.TopicUpdate

Shrink retention for a tenant topic to 3 days

yaml
id: kafka_topic_update
namespace: company.team

tasks:
  - id: update_topic
    type: io.kestra.plugin.kafka.TopicUpdate
    properties:
      bootstrap.servers: localhost:9092
    topic: tenant_acme_orders
    retentionMs: 259200000
Properties

Kafka AdminClient properties

Must include bootstrap.servers; accepts any Kafka AdminClient config. Provide base64-encoded content for ssl.keystore.location and ssl.truststore.location when using SSL.

Topic name

Default{}

Additional topic-level configs to set

Any other Kafka topic config, for example cleanup.policy or min.insync.replicas.

Reference (ref) of the pluginDefaults to apply to this task.

Retention size in bytes

Maps to the topic-level retention.bytes config.

Retention duration in milliseconds

Maps to the topic-level retention.ms config.

DefaultPT30S

AdminClient call timeout

Maximum duration to wait for each AdminClient operation to complete before failing the task. Defaults to PT30S (30 seconds).

Updated topic name

SubTypestring

Configs that were set on the topic