GCP Budget Forecasting with BigQuery Billing Export and Kestra

id: gcp-billing-budget-forecast-guard
namespace: company.team
description: |
  Project end-of-month GCP spend straight from the BigQuery billing export,
  extend the trailing 7-day pace across the rest of the month, and alert
  Slack the moment the forecast crosses your budget — while there is still
  time to cut something.

triggers:
  - id: daily_forecast_check
    type: io.kestra.plugin.core.trigger.Schedule
    description: Run the forecast once a day, after the billing export has landed.
      Shipped disabled; enable it after your first successful manual run.
    cron: "0 8 * * *"
    disabled: true

inputs:
  - id: monthly_budget_usd
    type: FLOAT
    defaults: 5000.0
    description: Monthly budget in USD. The forecast alerts when projected
      end-of-month spend exceeds this number.
  - id: billing_export_table
    type: STRING
    defaults: your-project.billing.gcp_billing_export_v1_SAMPLE
    description: Fully qualified BigQuery table of the standard daily GCP billing
      export, as project.dataset.gcp_billing_export_v1_<ACCOUNT>.

tasks:
  - id: fetch_spend_summary
    type: io.kestra.plugin.gcp.bigquery.Query
    description: Sum month-to-date cost and the trailing 7-day cost from the billing
      export, and pull the calendar facts the forecast needs — days elapsed and
      days in the month — in one pass.
    projectId: "{{ secret('GCP_PROJECT_ID') }}"
    serviceAccount: "{{ secret('GCP_SERVICE_ACCOUNT') }}"
    sql: |
      SELECT
        IFNULL(SUM(cost), 0) AS mtd_cost,
        IFNULL(SUM(IF(usage_start_time >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY), cost, 0)), 0) AS trailing_7d_cost,
        COUNT(DISTINCT IF(usage_start_time >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 7 DAY), DATE(usage_start_time), NULL)) AS trailing_7d_days,
        EXTRACT(DAY FROM CURRENT_DATE()) AS days_elapsed,
        DAY(LAST_DAY(CURRENT_DATE())) AS days_in_month
      FROM `{{ inputs.billing_export_table }}`
      WHERE usage_start_time >= TIMESTAMP(DATE_TRUNC(CURRENT_DATE(), MONTH))
    fetchType: FETCH_ONE

  - id: fetch_top_service
    type: io.kestra.plugin.gcp.bigquery.Query
    description: Name the service carrying the biggest share of month-to-date cost,
      so the alert points at where to look instead of only at a number.
    projectId: "{{ secret('GCP_PROJECT_ID') }}"
    serviceAccount: "{{ secret('GCP_SERVICE_ACCOUNT') }}"
    sql: |
      SELECT
        service.description AS service_name,
        IFNULL(SUM(cost), 0) AS service_cost
      FROM `{{ inputs.billing_export_table }}`
      WHERE usage_start_time >= TIMESTAMP(DATE_TRUNC(CURRENT_DATE(), MONTH))
      GROUP BY service.description
      ORDER BY service_cost DESC
      LIMIT 1
    fetchType: FETCH_ONE

  - id: assess_forecast
    type: io.kestra.plugin.scripts.python.Script
    description: Extend the trailing 7-day pace across the remaining days of the
      month, fall back to a linear month-to-date projection when the recent
      window is empty, and measure the result against the budget.
    taskRunner:
      type: io.kestra.plugin.core.runner.Process
    env:
      MTD_COST: "{{ outputs.fetch_spend_summary.row.mtd_cost }}"
      TRAILING_7D_COST: "{{ outputs.fetch_spend_summary.row.trailing_7d_cost }}"
      TRAILING_7D_DAYS: "{{ outputs.fetch_spend_summary.row.trailing_7d_days }}"
      DAYS_ELAPSED: "{{ outputs.fetch_spend_summary.row.days_elapsed }}"
      DAYS_IN_MONTH: "{{ outputs.fetch_spend_summary.row.days_in_month }}"
      MONTHLY_BUDGET: "{{ inputs.monthly_budget_usd }}"
      TOP_SERVICE: "{{ outputs.fetch_top_service.row.service_name ?? 'n/a' }}"
      TOP_COST: "{{ outputs.fetch_top_service.row.service_cost ?? 0 }}"
    script: |
      import json
      import os

      mtd = float(os.environ["MTD_COST"])
      trailing = float(os.environ["TRAILING_7D_COST"])
      trailing_days = int(float(os.environ["TRAILING_7D_DAYS"]))
      days_elapsed = int(float(os.environ["DAYS_ELAPSED"]))
      days_in_month = int(float(os.environ["DAYS_IN_MONTH"]))
      budget = float(os.environ["MONTHLY_BUDGET"])
      top_service = os.environ["TOP_SERVICE"] or "n/a"
      top_cost = round(float(os.environ["TOP_COST"]), 2)

      remaining = max(days_in_month - days_elapsed, 0)
      if trailing_days > 0:
          pace = trailing / trailing_days
          projected = round(mtd + pace * remaining, 2)
          basis = f"7-day pace (${pace:.2f}/day)"
      else:
          projected = round(mtd / max(days_elapsed, 1) * days_in_month, 2)
          basis = "linear month-to-date"

      baseline = max(budget, 0.01)
      pct_used = round(mtd / baseline * 100, 1)
      pct_projected = round(projected / baseline * 100, 1)
      headroom = round(budget - projected, 2)

      print(f"MTD ${mtd:.2f} projects to ${projected:.2f} ({pct_projected}% of ${budget:.2f}) using {basis}; top service: {top_service}")
      print("::" + json.dumps({"outputs": {
          "mtd_cost": round(mtd, 2),
          "projected_cost": projected,
          "monthly_budget": budget,
          "pct_used": pct_used,
          "pct_projected": pct_projected,
          "headroom": headroom,
          "forecast_basis": basis,
          "top_service": top_service,
          "top_service_cost": top_cost,
      }}) + "::")

  - id: check_forecast
    type: io.kestra.plugin.core.flow.If
    description: Compare the projected end-of-month cost against the budget input
      and only wake anyone when the month is tracking to overspend.
    condition: "{{ outputs.assess_forecast.vars.projected_cost >
      inputs.monthly_budget_usd }}"
    then:
      - id: alert_overspend
        type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
        description: Post the projection, the budget, month-to-date spend, the forecast
          basis, and the leading service so the channel knows the size of the
          problem and where to cut.
        url: "{{ secret('SLACK_WEBHOOK_URL') }}"
        payload: |
          {
            "text": "GCP budget guard: projected end-of-month spend is ${{ outputs.assess_forecast.vars.projected_cost }} against a ${{ inputs.monthly_budget_usd }} budget ({{ outputs.assess_forecast.vars.pct_projected }}%, headroom ${{ outputs.assess_forecast.vars.headroom }}) on a {{ outputs.assess_forecast.vars.forecast_basis }} basis. Month-to-date ${{ outputs.assess_forecast.vars.mtd_cost }} ({{ outputs.assess_forecast.vars.pct_used }}% of budget). Top service: {{ outputs.assess_forecast.vars.top_service }} (${{ outputs.assess_forecast.vars.top_service_cost }}). Execution {{ execution.id }}."
          }
    else:
      - id: log_on_track
        type: io.kestra.plugin.core.log.Log
        description: Quietly record healthy forecasts so the execution history becomes a
          day-by-day runway of the month.
        message: "GCP month-to-date ${{ outputs.assess_forecast.vars.mtd_cost }},
          projecting ${{ outputs.assess_forecast.vars.projected_cost }} ({{
          outputs.assess_forecast.vars.pct_projected }}% of the ${{
          inputs.monthly_budget_usd }} budget) — under budget."

errors:
  - id: alert_failure
    type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
    description: Alert when the guard itself fails, because a forecast that silently
      stopped running looks exactly like a calm month until the invoice lands.
    url: "{{ secret('SLACK_WEBHOOK_URL') }}"
    payload: |
      {
        "text": "GCP budget guard FAILED in flow {{ flow.id }} (execution {{ execution.id }}). End-of-month spend was NOT forecast — verify the billing export table, the GCP credentials, and the bigquery.jobs.create permission."
      }

outputs:
  - id: forecast_summary
    type: JSON
    description: 'Month-to-date cost, projected end-of-month cost, budget,
      percentages, headroom, forecast basis, and the top service — e.g.
      {"mtd_cost": 2310.44, "projected_cost": 5140.2, "monthly_budget": 5000.0,
      "pct_projected": 102.8, "top_service": "Compute Engine"}.'
    value: "{{ outputs.assess_forecast.vars | toJson }}"

A budget alert that fires at 90% of the budget has already lost the argument: by then the money is mostly spent. This blueprint moves the decision earlier by forecasting instead of measuring. It queries the BigQuery billing export for month-to-date cost and the trailing 7-day cost, extends that recent pace across the remaining days of the month, and compares the projection against a budget input — over it, Slack gets the forecast, the headroom, and the service driving the spend while the month is still young enough to do something about it.

How it works

fetch_spend_summary (io.kestra.plugin.gcp.bigquery.Query, FETCH_ONE) reads the standard daily billing export in a single pass: month-to-date cost, cost over the trailing 7 days, how many of those days actually have data, and the calendar facts (days_elapsed, days_in_month) the projection needs.
fetch_top_service (io.kestra.plugin.gcp.bigquery.Query) ranks services by month-to-date cost and returns the leader, so the alert names where to cut rather than only how much is at stake.
assess_forecast (io.kestra.plugin.scripts.python.Script on the io.kestra.plugin.core.runner.Process runner) extends the trailing 7-day daily pace across the remaining days of the month — a recent-pace forecast that reacts to a spike the same week it starts. When the recent window has no rows (a brand-new export), it falls back to a linear month-to-date projection, and it computes percent-of-budget and headroom either way.
check_forecast (io.kestra.plugin.core.flow.If) compares projected_cost with the monthly_budget_usd input. Over it, alert_overspend posts the projection, budget, month-to-date numbers, forecast basis, and top service to Slack; under it, log_on_track keeps a quiet record so the execution history shows the month's runway.
The errors block alerts Slack when the guard itself fails, and a daily Schedule trigger (shipped disabled) drives the cadence once enabled.

What you get

A projection, not a rear-view number: end-of-month cost estimated from this month's own recent pace.
A budget comparison with explicit headroom, so the question "how much room is left?" has a daily answer.
Per-service attribution in the alert, pointing at the largest line item before anyone opens the console.
A forecast_summary JSON output (month-to-date, projection, budget, percentages, basis, top service) for dashboards or downstream flows.

Who it's for

FinOps engineers who own a GCP budget and want to know where the month is heading, not where it has been.
Platform teams whose alarm at 80% of a budget always arrives too late to react.
Engineering leaders who have been surprised by a GCP invoice at least once.

Why orchestrate this with Kestra

The billing export already stores every cost row; what it cannot do is turn those rows into a scheduled decision. Kestra adds the loop: a daily schedule timed after the export lands, two BigQuery queries with retries, a Python projection step that needs no infrastructure of its own thanks to the Process runner, a branch that separates alerting from quiet record-keeping, and an execution history that doubles as a daily audit of the forecast. A cron script would run, but it would not retry, alert on its own failure, or leave a queryable trail.

Prerequisites

The standard daily BigQuery billing export enabled for your billing account, writing into the table named by billing_export_table.
A GCP service account that can run BigQuery jobs against the export dataset (for example roles/bigquery.jobUser plus a viewer role on the dataset).
Export data can lag by up to a day or two; the trailing-pace basis absorbs short gaps, but the first days of a brand-new export fall back to the linear projection.
A Slack incoming webhook for forecast and failure alerts.

Secrets

GCP_PROJECT_ID: project in which the BigQuery queries run.
GCP_SERVICE_ACCOUNT: service account key JSON with BigQuery job permissions.
SLACK_WEBHOOK_URL: Slack incoming webhook URL for overspend and failure alerts.

Quick start

Add the three secrets to your Kestra namespace.
Set billing_export_table to your export table's fully qualified name and monthly_budget_usd to your real budget.
Execute the flow once and read forecast_summary (and the logs) to see your projection.
Set disabled: false on the daily_forecast_check trigger.

How to extend

Add a second If that alerts on pct_used alone (for example 80% consumed) so slow and steady creep trips the guard even when the projection is still calm.
Group by project.id instead of service.description to attribute spend to the team whose project is growing.
Compare the projection against last month's actual spend (as the AWS blueprint in this repository does) for a month-over-month anomaly angle alongside the budget angle.
Fan the alert out to PagerDuty or a FinOps channel by adding another notification task in the then branch.

Forecast GCP Spend Against Your Budget Before the Invoice

How it works

What you get

Who it's for

Why orchestrate this with Kestra

Prerequisites

Secrets

Quick start

How to extend

Links

Tasks

Orchestrate with Kestra

Share this Blueprint

Create a Blueprint

New to Kestra?

Forecast GCP Spend Against Your Budget Before the Invoice

How it works

What you get

Who it's for

Why orchestrate this with Kestra

Prerequisites

Secrets

Quick start

How to extend

Links

Tasks

Orchestrate with Kestra

Share this Blueprint

Create a Blueprint

More Cloud Blueprints

Snowflake Warehouse Credit Usage and FinOps Cost Guard

Snowflake Warehouse Credit Rightsizing with Dry-Run Gate

Snort Packet Processing Stall and Rule Eval Timeout Sentinel

Snowflake Failed Login Attempts and Brute Force Detection Sentinel

Snowflake Task Failure and Stream Staleness Sentinel

SpinKube SpinApp Wasm Dispatch and Cold-Start Sentinel

More Business Blueprints

Monitor FastAPI Synthetic Latency SLOs and Alert SRE Teams via Slack

GDPR Data Subject Request Orchestrator for Access and Erasure

Google Cloud Storage (GCS) Bucket Lifecycle Policy Drift Sentinel

Forecast a FRED Series and Publish a Challenger Only If It Beats Seasonal-Naive

Revalue open foreign currency balances at month end with ECB rates and rate controls

Freshdesk SLA Breach Sentinel

New to Kestra?