ClickHouse Cold Storage S3 Partition Tiering
Audit ClickHouse system.parts for partitions older than retention limits on local disk, recommending migration to S3 cold storage.
Read Brevo transactional email stats every 4 hours, change nothing, and alert Slack when hard bounces or spam complaints cross your limits.
Order confirmations, OTPs and password resets are transactional emails, and they only work while the sending domain keeps a good reputation. A bad import that adds dead addresses, or a template that makes people press "Report spam", raises the hard bounce and complaint rates, and mailbox providers start throttling or junking every email from the domain. Brevo shows the numbers on its statistics page, but nothing pages the team when they move. This blueprint turns a scheduled, read-only call to the Brevo SMTP statistics API into a deliverability gate: io.kestra.plugin.core.http.Request reads the aggregated report, one jq query turns the counters into rates, and the result routes to a quiet log, a Slack note, or a Slack page.
get_email_report (io.kestra.plugin.core.http.Request) calls GET /v3/smtp/statistics/aggregatedReport?days=N with the api-key header. It is a read: no email is sent and no contact is changed. Retries cover short network blips, and allowFailed plus allowFailure keep an API error from aborting the flow so it can be reported instead.summarize_deliverability (io.kestra.plugin.core.debug.Return) first guards for a missing or non-200 response (UNREACHABLE). Otherwise one jq query computes the hard bounce rate (hardBounces / requests), the spam complaint rate (spamReports / delivered) and the blocked rate, and classifies the window: fewer than min_requests sends is LOW_VOLUME; a bounce or complaint rate at or above its critical line is CRITICAL; at or above its warning line is WARNING; otherwise HEALTHY.route_by_deliverability (io.kestra.plugin.core.flow.Switch) branches on that classification: HEALTHY and LOW_VOLUME log quietly, WARNING posts a Slack note, CRITICAL and UNREACHABLE page Slack. A defaults fallback logs anything unexpected.log_audit_result (io.kestra.plugin.core.log.Log) writes one audit line per run with the HTTP status and the full summary.errors block pages Slack if the flow itself fails, and the deliverability_check trigger runs every 4 hours, shipped disabled: true.status output (HEALTHY / WARNING / CRITICAL / LOW_VOLUME / UNREACHABLE) other flows can gate on, for example to pause a bulk campaign while complaints are high.Brevo's statistics are a dashboard, and its webhooks report single events, not a rate over a window. Kestra turns one read-only API call into a scheduled, typed classification with retries, a failure path and an execution history, and the status output lets your sending flows hold back bulk mail automatically when deliverability is in trouble.
BREVO_API_KEY: a Brevo API v3 key.SLACK_WEBHOOK: Slack incoming webhook used for every alert branch and the flow-failure alert.In Kestra OSS, secrets are supplied as environment variables prefixed SECRET_ and base64-encoded, for example SECRET_BREVO_API_KEY=$(echo -n 'xkeysib-xxxxxxxx' | base64), and read back with {{ secret('BREVO_API_KEY') }}. Use the Enterprise secrets backend for encryption at rest and access control.
BREVO_API_KEY and SLACK_WEBHOOK as secrets.min_requests and check status and the rates in the logs against Brevo's statistics page.deliverability_check.tag to check one email type (for example only OTPs) against its own limits.GET /v3/smtp/statistics/events?event=hardBounces to list the bouncing addresses and remove them from your own database.status != CRITICAL with a Flow trigger.lookback_days: type INT, default 1 (Brevo allows up to 90).warning_bounce_rate / critical_bounce_rate: type FLOAT, defaults 2.0 / 5.0 (percent of requests).warning_spam_rate / critical_spam_rate: type FLOAT, defaults 0.1 / 0.3 (percent of delivered).min_requests: type INT, default 100.status: one of HEALTHY, WARNING, CRITICAL, LOW_VOLUME, UNREACHABLE.{{ fromJson(outputs.summarize_deliverability.value).spam_rate }} for the complaint rate.range in the output to see the exact span Brevo used for lookback_days.