New to Kestra?
Use blueprints to kickstart your first workflows.
Probe an Arrow Flight SQL endpoint on a schedule and alert Slack when it answers without the TLS encryption your policy requires.
Apache Arrow Flight SQL is a gRPC-based protocol that backends like Dremio and Apache DataFusion expose, usually as a shared analytics gateway serving several teams. Because it is shared infrastructure, nobody necessarily owns watching whether every client still connects over an encrypted channel. A connection string with encryption quietly flipped off can sit unnoticed for a long time. This blueprint probes the endpoint with a trivial query and separately confirms the way it was reached matches your own TLS policy, alerting Slack with the real host and port the moment it does not.
probe_arrow_flight_endpoint (io.kestra.plugin.jdbc.arrowflight.Query) runs
SELECT 1 AS probe, a trivial, backend-agnostic query, with allowFailure: true so a
bad host, port, or credential does not stop the flow before it can be classified as
PROBE_FAILED below.classify_endpoint_status (io.kestra.plugin.core.debug.Return) returns
PROBE_FAILED when the probe did not return at all, TLS_POLICY_VIOLATION when it
succeeded but use_encryption is false while require_tls is true, HEALTHY
otherwise.route_by_endpoint_status (io.kestra.plugin.core.flow.Switch) branches on that
status: HEALTHY logs it, the other two alert Slack with the real host, port, and
encryption setting. A defaults branch logs anything unexpected.log_audit_summary always runs last. The errors block posts to Slack if the flow
itself fails outside the gate logic above.Arrow Flight SQL has no concept of alerting on how it was reached, that is a property of each client connection, not something the protocol surfaces on its own. Kestra adds the schedule, the policy comparison, and the Slack alert as one declarative, auditable flow.
For local testing, the official reference Arrow Flight SQL server, which ships a small TPC-H dataset by default:
docker run --name flight-sql -d -p 31337:31337 \
-e TLS_ENABLED=1 \
-e FLIGHT_PASSWORD=flight_password \
voltrondata/flight-sql:latest
With TLS_ENABLED=1 (this image's own default), the server serves a self-signed
certificate, which is why disable_certificate_verification defaults to true here.
Set it false against any real endpoint with a certificate from a trusted CA. The
image hardcodes its username to flight_username; only the password is configurable
via FLIGHT_PASSWORD. Note: this image is marked deprecated on Docker Hub in favor of
voltrondata/sqlflite, but remains published and pullable as of this writing.
A Slack incoming webhook for alerts.
ARROWFLIGHT_USERNAME, ARROWFLIGHT_PASSWORD: credentials for the endpoint. Against
the reference server, the username is fixed to flight_username; the password is
whatever FLIGHT_PASSWORD it was started with.
SLACK_WEBHOOK: Slack incoming webhook URL used by the alert tasks and the errors
block.
In Kestra OSS, set these as base64-encoded environment variables, for example
SECRET_ARROWFLIGHT_USERNAME, SECRET_SLACK_WEBHOOK.
| Input | Type | Default | Allowed values |
|---|---|---|---|
arrow_flight_host |
STRING | localhost |
any reachable Arrow Flight SQL host |
arrow_flight_port |
INT | 31337 |
any valid port |
use_encryption |
BOOL | true |
true, false |
disable_certificate_verification |
BOOL | true |
true, false |
require_tls |
SELECT | "true" |
"true", "false" |
{{ outputs.probe_arrow_flight_endpoint.row }}: the single probe row, when the query
succeeded.{{ outputs.classify_endpoint_status.value }}: HEALTHY, TLS_POLICY_VIOLATION, or
PROBE_FAILED.use_encryption: true,
require_tls: "true"), it should report HEALTHY.use_encryption: false, it should report TLS_POLICY_VIOLATION.every_30_minutes trigger.disable_certificate_verification: false against your own
internal CA to also catch an expired or wrongly-issued certificate, not just encryption
left off entirely.io.kestra.plugin.core.flow.Loop across a list of host:port
endpoints to audit an entire fleet of Flight SQL gateways from one flow.disable_certificate_verification: true is meant for the local test server's
self-signed certificate. Left true against a real endpoint, it silently defeats
certificate validation, which this gate reports the same as a correctly validated
connection whenever use_encryption is true.TLS_POLICY_VIOLATION only fires when use_encryption is false. A connection that
is encrypted but unauthenticated (disable_certificate_verification: true against an
untrusted cert) is not currently distinguished from one that is both, see How to
extend.SELECT 1 AS probe is deliberately trivial so this gate works against any backend
behind the protocol, confirming the endpoint answers and authenticates, not that any
dataset behind it is healthy.