Cube.js Pre-Aggregation Freshness & Build Sentinel

id: cubejs-pre-aggregation-table-freshness-and-build-sentinel
namespace: company.analytics

description: |
  Continuously audits Cube.js universal semantic layer pre-aggregation partitions and build statuses, detects refresh stalls and build failures, and dispatches actionable Slack incident alerts to prevent analytical query degradation.

inputs:
  - id: cube_host
    type: STRING
    displayName: Cube.js Endpoint Host
    defaults: "http://cube.internal:4000"
    description: "Base HTTP URL of the target Cube.js semantic layer API server."

  - id: target_cube
    type: STRING
    displayName: Target Semantic Cube
    defaults: "orders"
    description: "Name of the semantic cube model to inspect for pre-aggregation
      partition freshness."

  - id: max_allowed_failed_builds
    type: INT
    displayName: Maximum Allowed Failed Builds
    defaults: 0
    description: "Maximum tolerated count of failed pre-aggregation partitions
      before triggering operational alerts."

  - id: min_expected_pre_aggregations
    type: INT
    displayName: Minimum Expected Pre-Aggregations
    defaults: 1
    description: "Minimum count of active pre-aggregation tables expected for the
      target cube model."

  - id: slack_channel
    type: STRING
    displayName: Operational Slack Channel
    defaults: "#data-ops"
    description: "Target Slack channel or webhook destination for pre-aggregation
      build and freshness alerts."

triggers:
  - id: scheduled_sentinel_audit
    type: io.kestra.plugin.core.trigger.Schedule
    cron: "0 */4 * * *"
    disabled: true
    inputs:
      cube_host: "http://cube.internal:4000"
      target_cube: "orders"
      max_allowed_failed_builds: 0
      min_expected_pre_aggregations: 1
      slack_channel: "#data-ops"

tasks:
  - id: probe_cube_readiness
    type: io.kestra.plugin.core.http.Request
    uri: "{{ inputs.cube_host }}/readyz"
    method: GET
    headers:
      Accept: application/json
    timeout: PT15S
    description: "Queries the Cube.js readyz endpoint to verify HTTP service
      readiness and background worker availability."

  - id: probe_pre_aggregations_status
    type: io.kestra.plugin.core.http.Request
    uri: "{{ inputs.cube_host }}/cubejs-api/v1/pre-aggregations"
    method: GET
    headers:
      Accept: application/json
      Authorization: "{{ secret('CUBEJS_API_TOKEN') }}"
    timeout: PT30S
    description: "Queries the Cube.js pre-aggregations management API to inspect
      partition build statuses, table names, and refresh timestamps."

  - id: evaluate_pre_aggregation_health
    type: io.kestra.plugin.core.flow.Switch
    value: >-
      {% set readiness = outputs.probe_cube_readiness.body %} {% set
      ready_status = readiness.status | default('unknown') %} {% set
      preaggs_payload = outputs.probe_pre_aggregations_status.body %} {% set
      preaggs = preaggs_payload.preAggregations | default(preaggs_payload |
      default([])) %} {% set target = inputs.target_cube | lower %} {% set
      matched = [] %} {% set failed_count = 0 %} {% if preaggs is iterable %}
        {% for item in preaggs %}
          {% set item_name = item.tableName | default(item.id | default('')) | lower %}
          {% if target in item_name %}
            {% set _ = matched.add(item) %}
            {% if item.status == 'failed' or item.buildStatus == 'failed' or item.lastBuildError is defined %}
              {% set failed_count = failed_count + 1 %}
            {% endif %}
          {% endif %}
        {% endfor %}
      {% endif %} {% if ready_status != 'ok' %}
        SERVICE_UNHEALTHY
      {% elseif matched | length < inputs.min_expected_pre_aggregations %}
        PRE_AGGREGATION_MISSING
      {% elseif failed_count > inputs.max_allowed_failed_builds %}
        BUILD_FAILED
      {% else %}
        COMPLIANT
      {% endif %}
    cases:
      COMPLIANT:
        - id: log_compliant_status
          type: io.kestra.plugin.core.log.Log
          message: "Cube.js semantic layer on '{{ inputs.cube_host }}' is healthy. Target
            cube '{{ inputs.target_cube }}' has healthy pre-aggregation tables
            with zero build errors."

      BUILD_FAILED:
        - id: alert_build_failure_slack
          type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
          url: "{{ secret('SLACK_WEBHOOK_URL') }}"
          payload: |
            {
              "attachments": [
                {
                  "color": "#FFA726",
                  "title": "⚠️ Cube.js Pre-Aggregation Build Failure Detected",
                  "text": "One or more pre-aggregation tables for cube `{{ inputs.target_cube }}` failed to build on `{{ inputs.cube_host }}`.\n*Failure Threshold:* `{{ inputs.max_allowed_failed_builds }}`\n*Dashboard queries may fall back to expensive raw table scans.*",
                  "footer": "Kestra Sentinel | Semantic Layer Freshness Guard"
                }
              ]
            }

        - id: log_build_failure
          type: io.kestra.plugin.core.log.Log
          message: "Pre-aggregation build failures detected for cube '{{
            inputs.target_cube }}' on '{{ inputs.cube_host }}'. Investigation
            required."

      PRE_AGGREGATION_MISSING:
        - id: alert_missing_preagg_slack
          type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
          url: "{{ secret('SLACK_WEBHOOK_URL') }}"
          payload: |
            {
              "attachments": [
                {
                  "color": "#E53935",
                  "title": "❌ Cube.js Pre-Aggregation Table Missing",
                  "text": "Expected pre-aggregation tables for cube `{{ inputs.target_cube }}` could not be found on `{{ inputs.cube_host }}`.\n*Expected Minimum:* `{{ inputs.min_expected_pre_aggregations }}`\nVerify pre-aggregation definitions in schema files or Cube Store connectivity.",
                  "footer": "Kestra Sentinel | Schema Partition Integrity Gate"
                }
              ]
            }

        - id: log_missing_preagg
          type: io.kestra.plugin.core.log.Log
          message: "Missing pre-aggregation tables for cube '{{ inputs.target_cube }}' on
            '{{ inputs.cube_host }}'."

      SERVICE_UNHEALTHY:
        - id: alert_service_unhealthy_slack
          type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
          url: "{{ secret('SLACK_WEBHOOK_URL') }}"
          payload: |
            {
              "attachments": [
                {
                  "color": "#D32F2F",
                  "title": "🚨 Cube.js Service Readiness Failure",
                  "text": "Cube.js readiness probe returned status `{{ outputs.probe_cube_readiness.body.status | default('unhealthy') }}` on `{{ inputs.cube_host }}`.\nService is unreachable or background build queue is blocked.",
                  "footer": "Kestra Sentinel | Semantic Layer Availability Alert"
                }
              ]
            }

  - id: log_audit_summary
    type: io.kestra.plugin.core.log.Log
    message: "Completed Cube.js pre-aggregation audit for cube '{{
      inputs.target_cube }}' on {{ inputs.cube_host }} at {{ now() }}. Service
      status: {{ outputs.probe_cube_readiness.body.status | default('unknown')
      }}."

errors:
  - id: alert_probe_failure_slack
    type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
    url: "{{ secret('SLACK_WEBHOOK_URL') }}"
    payload: |
      {
        "attachments": [
          {
            "color": "#D32F2F",
            "title": "🚨 Cube.js Sentinel Probe Failure",
            "text": "The Cube.js compliance probe encountered an unhandled network error or connection timeout connecting to `{{ inputs.cube_host }}`.\n*Execution ID:* `{{ flow.executionId }}`\n*Error Message:* `{{ flow.errorMessage | default('Unknown execution error') }}`",
            "footer": "Kestra Orchestrator | Automated Incident Management"
          }
        ]
      }

Automated drift auditor and health gate that continuously monitors Cube.js universal semantic layer pre-aggregation tables and partition build statuses. Dispatches structured Slack incident notifications upon build failures or missing pre-aggregation models, preventing expensive raw table scans and degraded analytical dashboard response times.

How it works

A recurrent scheduled trigger initiates the pre-aggregation audit on a 4-hour interval, or triggers on-demand via webhook after upstream dbt or ETL warehouse runs.
The probe_cube_readiness task queries GET /readyz on the target Cube.js server to confirm HTTP service status and background orchestration readiness.
The probe_pre_aggregations_status task queries GET /cubejs-api/v1/pre-aggregations using an authenticated API token to extract partition metadata, build statuses, and error details across registered cubes.
Pebble expressions inspect the returned pre-aggregation catalog, isolating tables belonging to the specified target_cube, counting build failures, and checking partition availability against min_expected_pre_aggregations.
The evaluate_pre_aggregation_health flow-control Switch evaluates the classified operational state:
- COMPLIANT: Logs clean audit metrics when all expected pre-aggregation tables are built and healthy.
- BUILD_FAILED: Formats and dispatches a rich Slack warning containing target cube details and failed partition alerts.
- PRE_AGGREGATION_MISSING: Posts a high-priority Slack incident alert notifying data engineers that pre-aggregation definitions or storage tables are missing.
- SERVICE_UNHEALTHY: Alerts on-call teams immediately if the Cube.js readiness probe returns non-ok status.
A final log_audit_summary task records execution audit telemetry, and an errors block intercepts connection timeouts or authentication failures with automated Slack escalations.

What you get

Proactive pre-aggregation partition build failure detection before business users experience broken BI dashboards.
Continuous surveillance of semantic layer acceleration tables preventing unexpected cloud warehouse cost spikes.
Automated Slack incident routing with detailed contextual payload attachments identifying failing cube models.
Verifiable operational gate that can be chained downstream of warehouse loading pipelines before refreshing analytical models.
Centralized audit trail of semantic layer health preserved directly within Kestra's execution history.

Who it's for

Analytics Engineers and Data Platform Teams managing Cube.js semantic layers and embedded analytics.
BI Engineers responsible for executive dashboard SLAs and query performance.
Data Engineers orchestrating end-to-end ELT, dbt, and semantic caching workflows.

Why orchestrate this with Kestra

Declarative Semantic Governance: Codify pre-aggregation health policies, minimum table thresholds, and escalation paths in revision-controlled YAML workflows.
Native Secret Isolation: Securely manage Cube API tokens and Slack webhook URLs in namespace secret stores via {{ secret(...) }} without plaintext exposure.
Coordinated Data Pipeline Triggers: Trigger this sentinel automatically after upstream dbt transformations or data warehouse loading flows finish.
Resilient Multi-State Routing: Differentiate between infrastructure downtime, partition build stalls, and schema configuration errors using native Pebble logic and Switch tasks.

Prerequisites

A running Cube.js instance (accessible over HTTP/HTTPS) with the REST API and pre-aggregations management enabled.
Network connectivity from the Kestra worker or runner environment to the Cube.js server.
A Slack Incoming Webhook URL configured with permissions to post into your target operations channel.

Secrets

CUBEJS_API_TOKEN: Administrative API secret or JWT token for authenticating against the Cube.js REST API.
SLACK_WEBHOOK_URL: Slack Incoming Webhook endpoint URL used for operational alerts.

Quick start

Configure the required secrets (CUBEJS_API_TOKEN, SLACK_WEBHOOK_URL) in your Kestra namespace settings.
Import this blueprint into your Kestra instance under your desired namespace (e.g., company.analytics).
Click Execute with default inputs or provide your custom cube_host and target_cube to validate connectivity against your instance.
Review execution logs under the Executions tab to confirm that pre-aggregation partition statuses are retrieved and parsed correctly.
Enable the scheduled_sentinel_audit trigger to establish recurrent 4-hour automated freshness evaluations.

How to extend

Automated Pre-Aggregation Rebuild: Extend the build failure branch with an HTTP request to trigger a forced rebuild (POST /cubejs-api/v1/pre-aggregations/jobs).
Upstream dbt Pipeline Integration: Trigger this flow as a downstream dependency using a Kestra Flow trigger immediately following dbt model materialization.
PagerDuty Escalation: Route critical SERVICE_UNHEALTHY states directly to PagerDuty or Opsgenie on-call schedules.
Multi-Cube Sweep: Combine with a Kestra ForEach loop task to iterate across an array of critical semantic cube models.

Cube.js Pre-Aggregation Table Freshness and Build Sentinel

How it works

What you get

Who it's for

Why orchestrate this with Kestra

Prerequisites

Secrets

Quick start

How to extend

Links

Tasks

Orchestrate with Kestra

Share this Blueprint

Create a Blueprint

New to Kestra?

Cube.js Pre-Aggregation Table Freshness and Build Sentinel

How it works

What you get

Who it's for

Why orchestrate this with Kestra

Prerequisites

Secrets

Quick start

How to extend

Links

Tasks

Orchestrate with Kestra

Share this Blueprint

Create a Blueprint

More Data Blueprints

Detect lasting KPI shifts and name the release that caused them

Audit, Compact, and Optimize LanceDB Vector Tables with Stale Version Pruning

Langfuse Daily LLM Trace Quality and Cost Sentinel

Purge aged Kestra execution history behind an approval gate

LakeFS Isolated Lakehouse Branching and Atomic Merge Gate

Kubernetes Secret Exposure Sentinel

New to Kestra?