Apache SeaTunnel Job Failure and Zeta Cluster Health Sentinel
Audits Apache SeaTunnel Zeta Engine REST API for failed streaming or batch jobs and alerts on pipeline stalls.
Build heating, cooling, and growing degree days from NASA POWER daily weather for three sites, and fail when fill values remain.
id: nasa-power-daily-degree-day-mart
namespace: company.climate
description: |
Load NASA POWER daily temperature, precipitation, and solar irradiance for three sites, derive heating, cooling, and growing degree days in DuckDB, and fail the run when the archive still contains fill values. The Daily API needs no key. A schedule runs on the 8th; manual Execute uses the same tasks.
inputs:
- id: site_1_name
type: STRING
defaults: berlin
displayName: Site 1 name
description: Short name stored on every row. Lowercase letters, digits, and
underscores. The three names must differ.
- id: site_1_latitude
type: FLOAT
defaults: 52.52
min: -90
max: 90
displayName: Site 1 latitude
description: Decimal degrees. Default is Berlin.
- id: site_1_longitude
type: FLOAT
defaults: 13.405
min: -180
max: 180
displayName: Site 1 longitude
description: Decimal degrees. Default is Berlin.
- id: site_2_name
type: STRING
defaults: madrid
displayName: Site 2 name
- id: site_2_latitude
type: FLOAT
defaults: 40.4168
min: -90
max: 90
displayName: Site 2 latitude
description: Default is Madrid.
- id: site_2_longitude
type: FLOAT
defaults: -3.7038
min: -180
max: 180
displayName: Site 2 longitude
- id: site_3_name
type: STRING
defaults: nairobi
displayName: Site 3 name
- id: site_3_latitude
type: FLOAT
defaults: -1.2921
min: -90
max: 90
displayName: Site 3 latitude
description: Default is Nairobi.
- id: site_3_longitude
type: FLOAT
defaults: 36.8219
min: -180
max: 180
displayName: Site 3 longitude
- id: start_date
type: STRING
defaults: auto
displayName: First day
description: "`auto`, or `YYYY-MM-DD`. `auto` is 49 days before the run, which
with the default end date is a 30-day window."
- id: end_date
type: STRING
defaults: auto
displayName: Last day
description: "`auto`, or `YYYY-MM-DD`. `auto` is 20 days before the run so the
request stays inside the published POWER archive. POWER lags the
calendar."
- id: community
type: STRING
defaults: AG
displayName: POWER community
description: Agroclimatology (`AG`). The parameters below are requested from
that community.
- id: hdd_base_c
type: FLOAT
defaults: 18
min: 0
max: 40
displayName: Heating degree-day base (°C)
description: Daily heating degree days are `max(0, base - T2M)`. 18°C is the
usual energy base.
- id: cdd_base_c
type: FLOAT
defaults: 18
min: 0
max: 40
displayName: Cooling degree-day base (°C)
description: Daily cooling degree days are `max(0, T2M - base)`.
- id: gdd_base_c
type: FLOAT
defaults: 10
min: 0
max: 40
displayName: Growing degree-day base (°C)
description: Daily growing degree days are `max(0, (T2M_MAX + T2M_MIN) / 2 -
base)`. 10°C is a common crop base.
- id: max_missing_days
type: INT
defaults: 0
min: 0
displayName: Missing-day budget
description: Largest number of gap days allowed across all three sites. A gap
day is a date POWER skipped or a date still equal to the header fill
value.
triggers:
- id: monthly_power_refresh
type: io.kestra.plugin.core.trigger.Schedule
cron: "20 7 8 * *"
timezone: UTC
recoverMissedSchedules: NONE
description: |
07:20 UTC on the 8th. The default window ends 20 days before the run, so the 8th is not treated as a POWER release date. Missed runs are not replayed, so importing the flow does not backfill.
concurrency:
limit: 1
behavior: QUEUE
tasks:
- id: inputs_ok
type: io.kestra.plugin.core.flow.If
description: Refuse names and dates that would change the request or the SQL
before any call is sent.
condition: "{{ (inputs.site_1_name | regexMatch(regex='^[a-z][a-z0-9_]{1,31}$'))
and (inputs.site_2_name | regexMatch(regex='^[a-z][a-z0-9_]{1,31}$')) and
(inputs.site_3_name | regexMatch(regex='^[a-z][a-z0-9_]{1,31}$')) and
inputs.site_1_name != inputs.site_2_name and inputs.site_1_name !=
inputs.site_3_name and inputs.site_2_name != inputs.site_3_name and
(inputs.community | regexMatch(regex='^AG$')) and (inputs.start_date |
regexMatch(regex='^(auto|[0-9]{4}-(0[1-9]|1[0-2])-(0[1-9]|[12][0-9]|3[01]\
))$')) and (inputs.end_date |
regexMatch(regex='^(auto|[0-9]{4}-(0[1-9]|1[0-2])-(0[1-9]|[12][0-9]|3[01]\
))$')) and inputs.site_1_latitude >= -90 and inputs.site_1_latitude <= 90
and inputs.site_1_longitude >= -180 and inputs.site_1_longitude <= 180 and
inputs.site_2_latitude >= -90 and inputs.site_2_latitude <= 90 and
inputs.site_2_longitude >= -180 and inputs.site_2_longitude <= 180 and
inputs.site_3_latitude >= -90 and inputs.site_3_latitude <= 90 and
inputs.site_3_longitude >= -180 and inputs.site_3_longitude <= 180 and
inputs.hdd_base_c >= 0 and inputs.hdd_base_c <= 40 and inputs.cdd_base_c
>= 0 and inputs.cdd_base_c <= 40 and inputs.gdd_base_c >= 0 and
inputs.gdd_base_c <= 40 and inputs.max_missing_days >= 0 }}"
then:
- id: previous_window
type: io.kestra.plugin.core.kv.Get
description: Last window accepted for this trio of site names. Absent on the
first pass.
key: "nasa_power_degree_days_{{ inputs.site_1_name }}_{{ inputs.site_2_name
}}_{{ inputs.site_3_name }}"
errorOnMissing: false
- id: fetch_sites
type: io.kestra.plugin.core.flow.Parallel
description: One POWER Daily point request per site. The API has no multi-point
call for this product.
tasks:
- id: fetch_site_1
type: io.kestra.plugin.core.http.Download
description: |
Daily T2M, T2M_MAX, T2M_MIN (°C), PRECTOTCORR (mm/day), and ALLSKY_SFC_SW_DWN (MJ/m²/day) for site 1. `auto` dates are rendered as YYYYMMDD. No API key.
uri: "https://power.larc.nasa.gov/api/temporal/daily/point?parameters=T2M,T2M_M\
AX,T2M_MIN,PRECTOTCORR,ALLSKY_SFC_SW_DWN&community={{
inputs.community }}&longitude={{ inputs.site_1_longitude
}}&latitude={{ inputs.site_1_latitude }}&start={{
inputs.start_date == 'auto' ? (now() | dateAdd(-49, 'DAYS') |
date('yyyyMMdd')) : (inputs.start_date | replace({'-': ''}))
}}&end={{ inputs.end_date == 'auto' ? (now() | dateAdd(-20,
'DAYS') | date('yyyyMMdd')) : (inputs.end_date | replace({'-':
''})) }}&format=JSON"
headers:
Accept: application/json
User-Agent: kestra-nasa-power-daily-degree-day-mart/1.0 (+https://kestra.io)
options:
timeout:
readIdleTimeout: PT3M
retry:
type: exponential
interval: PT15S
maxInterval: PT2M
maxAttempts: 4
- id: fetch_site_2
type: io.kestra.plugin.core.http.Download
description: Same daily parameters for site 2.
uri: "https://power.larc.nasa.gov/api/temporal/daily/point?parameters=T2M,T2M_M\
AX,T2M_MIN,PRECTOTCORR,ALLSKY_SFC_SW_DWN&community={{
inputs.community }}&longitude={{ inputs.site_2_longitude
}}&latitude={{ inputs.site_2_latitude }}&start={{
inputs.start_date == 'auto' ? (now() | dateAdd(-49, 'DAYS') |
date('yyyyMMdd')) : (inputs.start_date | replace({'-': ''}))
}}&end={{ inputs.end_date == 'auto' ? (now() | dateAdd(-20,
'DAYS') | date('yyyyMMdd')) : (inputs.end_date | replace({'-':
''})) }}&format=JSON"
headers:
Accept: application/json
User-Agent: kestra-nasa-power-daily-degree-day-mart/1.0 (+https://kestra.io)
options:
timeout:
readIdleTimeout: PT3M
retry:
type: exponential
interval: PT15S
maxInterval: PT2M
maxAttempts: 4
- id: fetch_site_3
type: io.kestra.plugin.core.http.Download
description: Same daily parameters for site 3.
uri: "https://power.larc.nasa.gov/api/temporal/daily/point?parameters=T2M,T2M_M\
AX,T2M_MIN,PRECTOTCORR,ALLSKY_SFC_SW_DWN&community={{
inputs.community }}&longitude={{ inputs.site_3_longitude
}}&latitude={{ inputs.site_3_latitude }}&start={{
inputs.start_date == 'auto' ? (now() | dateAdd(-49, 'DAYS') |
date('yyyyMMdd')) : (inputs.start_date | replace({'-': ''}))
}}&end={{ inputs.end_date == 'auto' ? (now() | dateAdd(-20,
'DAYS') | date('yyyyMMdd')) : (inputs.end_date | replace({'-':
''})) }}&format=JSON"
headers:
Accept: application/json
User-Agent: kestra-nasa-power-daily-degree-day-mart/1.0 (+https://kestra.io)
options:
timeout:
readIdleTimeout: PT3M
retry:
type: exponential
interval: PT15S
maxInterval: PT2M
maxAttempts: 4
- id: build_mart
type: io.kestra.plugin.jdbc.duckdb.Queries
description: |
Unnest each parameter map, replace the header fill value with null, and compute degree days.
Days are local solar time, which is what POWER labels `time_standard`. Coordinates and elevation
come from the response geometry, not from the inputs, so the mart records the grid cell POWER used.
url: "jdbc:duckdb:"
fetchType: FETCH_ONE
inputFiles:
site_1.json: "{{ outputs.fetch_site_1.uri }}"
site_2.json: "{{ outputs.fetch_site_2.uri }}"
site_3.json: "{{ outputs.fetch_site_3.uri }}"
outputFiles:
- daily
- site_summary
sql: |
CREATE TABLE meta AS
SELECT '{{ inputs.site_1_name }}' AS site_id, content FROM read_text('site_1.json')
UNION ALL
SELECT '{{ inputs.site_2_name }}', content FROM read_text('site_2.json')
UNION ALL
SELECT '{{ inputs.site_3_name }}', content FROM read_text('site_3.json');
CREATE TABLE meta_cols AS
SELECT
site_id,
json_extract_string(content, '$.header.api.version') AS api_version,
json_extract_string(content, '$.header.title') AS source_title,
json_extract_string(content, '$.header.time_standard') AS time_standard,
CAST(json_extract(content, '$.header.fill_value') AS DOUBLE) AS fill_value,
json_extract_string(content, '$.header.start') AS header_start,
json_extract_string(content, '$.header.end') AS header_end,
CAST(json_extract(content, '$.geometry.coordinates[0]') AS DOUBLE) AS longitude,
CAST(json_extract(content, '$.geometry.coordinates[1]') AS DOUBLE) AS latitude,
CAST(json_extract(content, '$.geometry.coordinates[2]') AS DOUBLE) AS elevation_m,
content
FROM meta;
CREATE TABLE daily AS
WITH exploded AS (
SELECT m.site_id, m.api_version, m.source_title, m.time_standard, m.fill_value, m.header_start, m.header_end,
m.longitude, m.latitude, m.elevation_m,
'T2M' AS parameter, j.key AS ymd, TRY_CAST(j.value AS DOUBLE) AS raw_value
FROM meta_cols m, json_each(json_extract(m.content, '$.properties.parameter.T2M')) j
UNION ALL
SELECT m.site_id, m.api_version, m.source_title, m.time_standard, m.fill_value, m.header_start, m.header_end,
m.longitude, m.latitude, m.elevation_m,
'T2M_MAX', j.key, TRY_CAST(j.value AS DOUBLE)
FROM meta_cols m, json_each(json_extract(m.content, '$.properties.parameter.T2M_MAX')) j
UNION ALL
SELECT m.site_id, m.api_version, m.source_title, m.time_standard, m.fill_value, m.header_start, m.header_end,
m.longitude, m.latitude, m.elevation_m,
'T2M_MIN', j.key, TRY_CAST(j.value AS DOUBLE)
FROM meta_cols m, json_each(json_extract(m.content, '$.properties.parameter.T2M_MIN')) j
UNION ALL
SELECT m.site_id, m.api_version, m.source_title, m.time_standard, m.fill_value, m.header_start, m.header_end,
m.longitude, m.latitude, m.elevation_m,
'PRECTOTCORR', j.key, TRY_CAST(j.value AS DOUBLE)
FROM meta_cols m, json_each(json_extract(m.content, '$.properties.parameter.PRECTOTCORR')) j
UNION ALL
SELECT m.site_id, m.api_version, m.source_title, m.time_standard, m.fill_value, m.header_start, m.header_end,
m.longitude, m.latitude, m.elevation_m,
'ALLSKY_SFC_SW_DWN', j.key, TRY_CAST(j.value AS DOUBLE)
FROM meta_cols m, json_each(json_extract(m.content, '$.properties.parameter.ALLSKY_SFC_SW_DWN')) j
),
clean AS (
SELECT site_id, api_version, source_title, time_standard, header_start, header_end,
longitude, latitude, elevation_m, parameter, ymd,
CASE WHEN raw_value IS NULL OR raw_value = fill_value THEN NULL ELSE raw_value END AS value
FROM exploded
),
wide AS (
SELECT
site_id,
any_value(api_version) AS api_version,
any_value(source_title) AS source_title,
any_value(time_standard) AS time_standard,
any_value(header_start) AS header_start,
any_value(header_end) AS header_end,
any_value(longitude) AS longitude,
any_value(latitude) AS latitude,
any_value(elevation_m) AS elevation_m,
strptime(ymd, '%Y%m%d')::DATE AS obs_date,
max(value) FILTER (WHERE parameter = 'T2M') AS t2m_c,
max(value) FILTER (WHERE parameter = 'T2M_MAX') AS t2m_max_c,
max(value) FILTER (WHERE parameter = 'T2M_MIN') AS t2m_min_c,
max(value) FILTER (WHERE parameter = 'PRECTOTCORR') AS precip_mm,
max(value) FILTER (WHERE parameter = 'ALLSKY_SFC_SW_DWN') AS solar_mj_m2
FROM clean
GROUP BY site_id, ymd
)
SELECT
site_id,
obs_date,
latitude,
longitude,
elevation_m,
t2m_c,
t2m_max_c,
t2m_min_c,
precip_mm,
solar_mj_m2,
CASE WHEN t2m_c IS NULL THEN NULL ELSE greatest(0, {{ inputs.hdd_base_c }} - t2m_c) END AS hdd_c,
CASE WHEN t2m_c IS NULL THEN NULL ELSE greatest(0, t2m_c - {{ inputs.cdd_base_c }}) END AS cdd_c,
CASE WHEN t2m_max_c IS NULL OR t2m_min_c IS NULL THEN NULL ELSE greatest(0, (t2m_max_c + t2m_min_c) / 2.0 - {{ inputs.gdd_base_c }}) END AS gdd_c,
(t2m_c IS NULL OR t2m_max_c IS NULL OR t2m_min_c IS NULL OR precip_mm IS NULL OR solar_mj_m2 IS NULL) AS incomplete,
api_version,
source_title,
time_standard,
header_start,
header_end
FROM wide;
COPY (
SELECT site_id, obs_date, latitude, longitude, round(elevation_m, 2) AS elevation_m,
t2m_c, t2m_max_c, t2m_min_c, precip_mm, solar_mj_m2,
round(hdd_c, 4) AS hdd_c, round(cdd_c, 4) AS cdd_c, round(gdd_c, 4) AS gdd_c
FROM daily
ORDER BY site_id, obs_date
) TO '{{ outputFiles.daily }}' (FORMAT PARQUET);
COPY (
SELECT site_id,
count(*) AS days,
count(*) FILTER (WHERE incomplete) AS missing_days,
round(avg(t2m_c), 2) AS t2m_mean_c,
round(sum(precip_mm), 2) AS precip_mm,
round(sum(solar_mj_m2), 2) AS solar_mj_m2,
round(sum(hdd_c), 2) AS hdd,
round(sum(cdd_c), 2) AS cdd,
round(sum(gdd_c), 2) AS gdd,
any_value(latitude) AS latitude,
any_value(longitude) AS longitude,
round(any_value(elevation_m), 1) AS elevation_m,
any_value(header_start) AS window_start,
any_value(header_end) AS window_end,
any_value(api_version) AS api_version,
any_value(time_standard) AS time_standard
FROM daily
GROUP BY site_id
ORDER BY site_id
) TO '{{ outputFiles.site_summary }}' (HEADER, DELIMITER ',');
WITH bounds AS (
SELECT
count(*) AS day_rows,
count(DISTINCT site_id) AS sites,
count(*) FILTER (WHERE incomplete) AS filled_days,
count(DISTINCT header_start || header_end) AS windows,
min(header_start) AS window_start,
max(header_end) AS window_end,
any_value(source_title) AS source_title,
any_value(api_version) AS api_version,
any_value(time_standard) AS time_standard,
round(sum(hdd_c), 1) AS hdd_total,
round(sum(cdd_c), 1) AS cdd_total,
round(sum(gdd_c), 1) AS gdd_total,
round(sum(precip_mm), 1) AS precip_mm,
round(sum(solar_mj_m2), 1) AS solar_mj_m2
FROM daily
),
expected AS (
SELECT coalesce(sum(expected_days), 0) AS expected_rows
FROM (
SELECT site_id,
date_diff('day', strptime(any_value(header_start), '%Y%m%d')::DATE, strptime(any_value(header_end), '%Y%m%d')::DATE) + 1 AS expected_days
FROM daily
GROUP BY site_id
)
),
roll AS (
SELECT coalesce(string_agg(site_id || '=' || t2m_mean_c || 'C hdd=' || hdd || ' cdd=' || cdd || ' gdd=' || gdd, ', ' ORDER BY site_id), '') AS site_rollups
FROM (
SELECT site_id,
round(avg(t2m_c), 1) AS t2m_mean_c,
round(sum(hdd_c), 1) AS hdd,
round(sum(cdd_c), 1) AS cdd,
round(sum(gdd_c), 1) AS gdd
FROM daily
GROUP BY site_id
)
)
SELECT
b.source_title,
b.api_version,
b.time_standard,
b.window_start,
b.window_end,
b.sites,
b.day_rows,
e.expected_rows,
b.filled_days,
CASE
WHEN b.day_rows = 0 THEN 1
WHEN b.windows <> 1 THEN b.day_rows
ELSE e.expected_rows - b.day_rows + b.filled_days
END AS gap_days,
CASE
WHEN b.day_rows = 0 OR b.sites <> 3 OR b.windows <> 1
OR (e.expected_rows - b.day_rows + b.filled_days) > {{ inputs.max_missing_days }}
THEN 'FAIL'
ELSE 'PASS'
END AS gate,
b.hdd_total,
b.cdd_total,
b.gdd_total,
b.precip_mm,
b.solar_mj_m2,
r.site_rollups
FROM bounds b, expected e, roll r;
- id: coverage_ok
type: io.kestra.plugin.core.flow.If
description: Fail when DuckDB sets gate to FAIL. That covers a missing site,
mismatched windows, or gap days above the budget. A failed window does
not move the KV marker.
condition: "{{ outputs.build_mart.outputs[0].row.gate == 'FAIL' }}"
then:
- id: fail_coverage
type: io.kestra.plugin.core.execution.Fail
errorMessage: "POWER coverage failed. window={{
outputs.build_mart.outputs[0].row.window_start }}-{{
outputs.build_mart.outputs[0].row.window_end }} sites={{
outputs.build_mart.outputs[0].row.sites }} days={{
outputs.build_mart.outputs[0].row.day_rows }} expected={{
outputs.build_mart.outputs[0].row.expected_rows }} filled={{
outputs.build_mart.outputs[0].row.filled_days }} gaps={{
outputs.build_mart.outputs[0].row.gap_days }} budget={{
inputs.max_missing_days }} api={{
outputs.build_mart.outputs[0].row.api_version }}."
else:
- id: remember_window
type: io.kestra.plugin.core.kv.Set
key: "nasa_power_degree_days_{{ inputs.site_1_name }}_{{ inputs.site_2_name
}}_{{ inputs.site_3_name }}"
value: "{{ outputs.build_mart.outputs[0].row.window_start }}:{{
outputs.build_mart.outputs[0].row.window_end }}:{{
outputs.build_mart.outputs[0].row.api_version }}"
description: Marker for the next run. Written only after the coverage gate
passes.
- id: log_pass
type: io.kestra.plugin.core.log.Log
message: "POWER degree-day mart passed. window={{
outputs.build_mart.outputs[0].row.window_start }}-{{
outputs.build_mart.outputs[0].row.window_end }} previous={{
outputs.previous_window.value ?? 'none' }} time={{
outputs.build_mart.outputs[0].row.time_standard }} sites={{
outputs.build_mart.outputs[0].row.sites }} days={{
outputs.build_mart.outputs[0].row.day_rows }} gaps={{
outputs.build_mart.outputs[0].row.gap_days }} api={{
outputs.build_mart.outputs[0].row.api_version }} hdd={{
outputs.build_mart.outputs[0].row.hdd_total }} cdd={{
outputs.build_mart.outputs[0].row.cdd_total }} gdd={{
outputs.build_mart.outputs[0].row.gdd_total }} precip_mm={{
outputs.build_mart.outputs[0].row.precip_mm }} rollup={{
outputs.build_mart.outputs[0].row.site_rollups }}"
else:
- id: fail_inputs
type: io.kestra.plugin.core.execution.Fail
errorMessage: "Site names must be distinct lowercase slugs, community must be
AG, and dates must be auto or YYYY-MM-DD. Bases must be 0 to 40. Got
names={{ inputs.site_1_name }},{{ inputs.site_2_name }},{{
inputs.site_3_name }} community={{ inputs.community }} start={{
inputs.start_date }} end={{ inputs.end_date }}."
NASA POWER publishes daily temperature, rain, and solar data for any land point. No API key. This flow loads three sites, builds heating, cooling, and growing degree days in DuckDB, and fails when fill values remain.
monthly_power_refresh (io.kestra.plugin.core.trigger.Schedule) runs at 07:20 UTC on the 8th. Missed runs are not replayed.inputs_ok (io.kestra.plugin.core.flow.If) checks three distinct slugs, coordinates, community AG, bases, and auto or YYYY-MM-DD dates.previous_window (io.kestra.plugin.core.kv.Get) reads the last good window.fetch_sites (io.kestra.plugin.core.flow.Parallel) downloads temperature, rain, and solar for each site. auto is 30 days ending 20 days ago.build_mart (io.kestra.plugin.jdbc.duckdb.Queries) nulls the header fill value and writes degree days to Parquet and CSV. Days are local solar time.coverage_ok (io.kestra.plugin.core.flow.If) fails when gaps exceed max_missing_days. Otherwise remember_window stores the window and log_pass prints the rollup.power.larc.nasa.gov.auto dates.daily and site_summary. The schedule waits for the 8th.log_pass.UNION ALL branch, or alert from fail_coverage.Audits Apache SeaTunnel Zeta Engine REST API for failed streaming or batch jobs and alerts on pipeline stalls.
When a fund manager you follow files its quarterly 13F, compare its holdings with the previous quarter and post the new positions, exits and largest changes to Slack.
Extract CSV data, mask PII columns with DuckDB SQL, then load the anonymized result into BigQuery.
Turn real Sentinel-2 COG windows into a calibrated, cloud-masked NDVI raster with provenance and read-back checks.
Load bounded JSON batches with a transactional PostgreSQL ledger, then archive the SFTP source. Retry an archive failure without inserting duplicate records.
Trigger a Segment Reverse ETL sync, classify it OK, PARTIAL_FAILURE, or FAILED by status and delivery count, alert Slack, and log a governance audit trail.
