Apache Flink MonitorJob

Apache Flink MonitorJob

Certified

Trigger on Flink job state

Polls a Flink job over REST and triggers a flow when it reaches a terminal state. Uses a 10s polling default and can fail the flow on FAILED states.

yaml
type: io.kestra.plugin.flink.MonitorJob

Monitor Flink job and trigger on completion

yaml
id: flink-job-monitor
namespace: company.team

inputs:
  - id: jobId
    type: STRING
    required: true

triggers:
  - id: job-monitor
    type: io.kestra.plugin.flink.MonitorJob
    restUrl: "http://flink-jobmanager:8081"
    jobId: "{{ inputs.jobId }}"
    interval: "PT30S"
    expectedTerminalStates:
      - "FINISHED"
      - "CANCELED"
    failOnError: true

tasks:
  - id: notify-completion
    type: io.kestra.plugin.core.log.Log
    message: "Flink job {{ trigger.jobId }} reached state: {{ trigger.finalState }}"

# jobId must be a valid hexadecimal Flink JobID for an existing job.
# The Flink JobManager must be reachable by this trigger.
Installation

Our standard Docker images already include this plugin. If you use a -slim image or another setup, install it with the following command:

kestractl plugins get io.kestra.plugin:plugin-flink --compatible-for $KESTRA_VERSION

For kestractl, use your Kestra version with --compatible-for, or specify an exact plugin version.

For Kestra CLI, replace LATEST with a specific version or version range to pin the plugin version.

Properties

Job ID

ID of the Flink job to monitor.

Flink REST API URL

Base URL of the Flink REST API (e.g., http://flink-jobmanager: 8081).

Defaultfalse

Specifies whether a trigger is allowed to start a new execution even if a previous run is still in progress.

SubTypestring

Expected terminal states

Job states treated as success. Defaults to ['FINISHED'].

Defaulttrue

Fail on error

Fail the trigger when the job reaches FAILED. If false, trigger still fires with success=false; defaults to true.

DefaultPT10S
Formatduration

Polling interval

Interval between job status checks (static ISO-8601 duration, e.g., PT30S). Defaults to PT10S.

SubTypestring
Possible Values
CREATEDSUBMITTEDRUNNINGPAUSEDRESTARTEDKILLINGSUCCESSWARNINGFAILEDKILLEDCANCELLEDQUEUEDRETRYINGRETRIEDSKIPPEDBREAKPOINTRESUBMITTED

List of execution states after which a trigger should be stopped (a.k.a. disabled).

Defaulttrue

A condition that determines whether the trigger should run.

A Pebble expression evaluated at trigger time. The trigger fires only when the expression evaluates to a truthy value (true, a non-empty string, a non-zero number). Use this to gate trigger execution on dynamic runtime values such as execution labels, flow variables, or environment conditions.