Query icon
OutputValues icon
Log icon
If icon
SlackIncomingWebhook icon
Pause icon
Loop icon
Fail icon
Schedule icon

Oracle Materialized View Stale Refresh Guard

Detect Oracle materialized views serving old rows, compile then refresh them with DBMS_MVIEW.REFRESH behind an approval pause, then re-read dba_mviews.

Categories
DataInfrastructure

A materialized view on REFRESH ON DEMAND is exactly as fresh as the last refresh somebody ran. Nothing in Oracle refreshes it for you. When the refresh job is removed, fails quietly or was never scheduled, the view keeps answering every query out of the rows it already holds.

This is the dangerous shape of staleness, because nothing errors. A query against a stale materialized view is fast, returns a number plus raises no warning. On the test rig below, a view reported 8 orders while its base table held 11. The only place that showed was dba_mviews.staleness. Query rewrite can route a query to the same stale view without the query ever naming it.

This blueprint reads that column, carefully, because the value most people expect is not the value Oracle sets.

This blueprint was created by zkasuran.

Findings

Insert rows into a base table that has no materialized view log and the view does not go STALE. It goes NEEDS_COMPILE, which means Oracle cannot say whether the data moved until the view is compiled. Only after ALTER MATERIALIZED VIEW ... COMPILE does dba_mviews resolve it to STALE or FRESH. A detector that looks for STALE alone misses the most common case completely.

staleness what it means what this flow does
FRESH the view matches its base tables reported only when the last refresh is past max_age_hours
STALE base data changed since the last refresh refresh target
NEEDS_COMPILE a base object changed in a way Oracle could not track, so freshness is undetermined compile target, then re-audited, then refreshed only if it resolved to STALE
UNKNOWN freshness cannot be established, a remote or untracked base object refresh target
UNDEFINED the view does not track staleness at all refresh target
UNUSABLE the container table is not trustworthy, usually an interrupted refresh reported only, a rebuild is not a refresh
COMPILATION_ERROR a compile was already tried and the definition no longer holds, so a base column was dropped or retyped reported only, the definition has to change first

Compiling first is also the cheapest safety check available. A COMPILE touches no rows. If it errors, the base change broke the definition. That is exactly the case where a blind complete refresh does the most damage: it empties the container, then fails.

Safety

  • A change runs only when auto_remediate is true, dry_run is false and a human resumes the approval_gate pause with approved set to true. Three conditions, all off by default.
  • refresh_method defaults to C, a complete refresh, because it always works. F fails with ORA-23413 unless every base table has a materialized view log, so it is a per view choice rather than a default. The findings carry fast_refreshable so you can see which views F could ever work on.
  • A complete refresh is heavy. It rereads every base row, writes the whole container table again plus can run for hours on a large view. That is what the pause is for.
  • atomic_refresh defaults to true, which keeps the view queryable throughout because the delete plus insert runs as one transaction. Setting it false lets Oracle truncate the container then direct path load it, which is much faster and leaves the view empty to every reader until the load finishes. On a view behind a live dashboard that is an outage, not an optimisation.
  • Only refresh_mode = 'DEMAND' views are touched. An ON COMMIT view refreshes itself, so a flow refreshing it would be fighting the database.
  • The re-read requires a last_refresh_date at or after a clock reading taken from the database before the pass, so a view that was already FRESH cannot be counted as a refresh that happened.

How it works

  1. totals_before (io.kestra.plugin.jdbc.oracle.Query, read-only) counts the on demand materialized views in the checked schemas by staleness state plus reads the oldest last refresh.
  2. audit (read-only) lists the views that are not FRESH or whose last_refresh_date is older than max_age_hours. For each one it returns the staleness value, the compile state, fast_refreshable, the last refresh type, stale_since plus the exact COMPILE and DBMS_MVIEW.REFRESH statements.
  3. targets (io.kestra.plugin.core.output.OutputValues) splits the findings into refresh targets, compile targets and reported only. summary logs all three groups.
  4. route (io.kestra.plugin.core.flow.If):
    • clean: every view is fresh inside the policy, so it logs the counts and stops.
    • findings: alert posts the three groups to Slack. state_route explains the NEEDS_COMPILE plus UNUSABLE views when there are any, otherwise it says so.
      • remediate_gate runs only when auto_remediate is true, dry_run is false and there is at least one compile or refresh target.
      • approval_gate (io.kestra.plugin.core.flow.Pause) holds the run. apply checks the resume answer, so an unanswered pause never reads as approval.
      • mark_time reads the database clock before anything changes.
      • compile_stage runs compile_pass (io.kestra.plugin.core.flow.Loop, concurrencyLimit: 1), one ALTER MATERIALIZED VIEW ... COMPILE per undetermined view, then compile_check re-reads dba_mviews and compile_verify fails the run if any of them is still unresolved.
      • reaudit re-runs the audit. resolved_targets rebuilds the refresh list from it, so a view that compiled back to FRESH is never refreshed.
      • refresh_stage runs remediate (Loop, concurrencyLimit: 1), one DBMS_MVIEW.REFRESH per view. reverify re-reads dba_mviews for those exact views, totals_after re-reads the counts. verify_result fails the run through io.kestra.plugin.core.execution.Fail if any target did not come back fresh with a newer last_refresh_date.
  5. errors posts to Slack if the flow fails for any reason.

What you get

  • A detector that catches the common case, not just the textbook one, because it treats NEEDS_COMPILE as a finding and resolves it instead of ignoring it.
  • A freshness policy in hours, so a view that still reads FRESH but has not refreshed since yesterday is a finding rather than a surprise.
  • A compile pass that is both the way to learn the real staleness and a free check that the definition still holds.
  • fast_refreshable in the findings, so the choice between F and C is informed.
  • Two independent failure gates. A view that will not compile fails the run. A view that did not actually refresh fails the run too.
  • Slack alerts on the findings and on any failure.

Who it's for

  • Data teams whose dashboards read from materialized views and whose numbers quietly stopped matching the source tables.
  • DBAs who inherited an instance with materialized views nobody owns, where the refresh jobs may or may not still exist.
  • Platform teams who want the freshness promise written down as a schedule plus a threshold, rather than living in a scheduler entry somebody deleted.

Why orchestrate this with Kestra

The usual answer is a DBMS_SCHEDULER job calling DBMS_MVIEW.REFRESH, which is the thing that goes missing in the first place. When it fails it fails inside the database, with the error in a scheduler table nobody watches. Kestra puts the check outside the database: it reads dba_mviews on a schedule, posts what is stale to Slack, holds a heavy complete refresh behind a human approval, then proves the refresh landed by reading the catalog again plus failing the run if it did not. Every run keeps its logs, so you can show which view was stale, since when and what the refresh changed.

Tested end to end

On Kestra 2.0.5 OSS against Oracle Database Free 23ai (gvenzl/oracle-free, service FREEPDB1), with a local HTTP sink standing in for Slack. Schema APP held SALES with 8 rows plus REGION_DIM with 3. Two materialized views were built REFRESH COMPLETE ON DEMAND: MV_SALES_BY_REGION over SALES and MV_REGION_DIM over REGION_DIM. Both were refreshed once so both read FRESH, then 3 rows were inserted into SALES only.

The wrong answer was measured, not assumed. With MV_SALES_BY_REGION not fresh, SELECT SUM(order_count) FROM app.mv_sales_by_region returned 8 while SELECT COUNT(*) FROM app.sales returned 11. The amounts were 1475 against 2435. After the flow compiled plus refreshed it, the same two queries both returned 11 and both returned 2435. MV_REGION_DIM stayed FRESH throughout, which is the precision check: the detector returned exactly one row, not two.

Run Setup and inputs Result
1 both views FRESH, defaults 0 findings, clean branch, SUCCESS
2 3 rows inserted into SALES, defaults 1 finding, staleness NEEDS_COMPILE, posted to Slack, no change, SUCCESS
3 same, auto_remediate: true, dry_run: true Same report, Slack said no change applied, remediate_gate never opened. The dry run gate held
4 same, armed, pause resumed approved: false Declined, state unchanged, SUCCESS
5 same, armed, pause resumed approved: true Compiled, resolved to STALE, refreshed, reverify found 0, SUCCESS
6 base column the view aggregates dropped then compiled by hand, defaults 1 COMPILATION_ERROR finding, 0 refresh targets, 0 compile targets, reported only, SUCCESS
7 stale view, refresh_method: F, no materialized view log, armed, approved Refresh failed with ORA-23413, reverify reported refresh did not run, run FAILED, errors block alerted Slack
8 base column dropped with the view left NEEDS_COMPILE, armed, approved Compile failed, compile_verify fired, no refresh was attempted, run FAILED, errors block alerted Slack
9 max_age_hours: 0, both views FRESH Both detected by age alone with staleness FRESH, SUCCESS

Runs 1 and 2 are the proof the check can fail: the same query returned 0 rows against two FRESH views and exactly 1 row after 3 inserts into one base table. Run 9 proves the age predicate works on its own, independently of the staleness column. Run 8 is the one that matters most for safety, because the broken view was never handed to a refresh. Every staleness value plus last refresh timestamp was read back from dba_mviews with sqlplus, not only from the Kestra log.

Prerequisites

  • An Oracle Database reachable over the thin JDBC driver (jdbc:oracle:thin:@//host:1521/service).
  • A database user that can read dba_mviews (SELECT_CATALOG_ROLE or SELECT ANY DICTIONARY). Compiling or refreshing another schema's materialized view needs ALTER ANY MATERIALIZED VIEW. A user working only on its own views needs no extra privilege.
  • At least one materialized view with REFRESH ON DEMAND. ON COMMIT views are skipped by design.
  • For refresh_method: F, a materialized view log on every base table, created before the view was built.

Secrets

  • ORACLE_PASSWORD: the database password. It is the default for the password input.
  • SLACK_WEBHOOK_URL: used by alert and the errors block.
  • In Kestra OSS, provide each one as a base64-encoded environment variable prefixed SECRET_.

Inputs

Input Default Purpose
jdbc_url jdbc:oracle:thin:@//localhost:1521/FREEPDB1 The Oracle instance to check
username SYSTEM The database user
password {{ secret('ORACLE_PASSWORD') }} The database password
mview_owners [APP] Schemas whose materialized views are checked
max_age_hours 24.0 A FRESH view last refreshed longer ago than this is also a finding
refresh_method C DBMS_MVIEW.REFRESH method code, one of C, F, ?, A
atomic_refresh true True keeps the view readable during the refresh
auto_remediate false Allow COMPILE plus DBMS_MVIEW.REFRESH
dry_run true Must be false for any change to run

Quick start

  1. Add the ORACLE_PASSWORD and SLACK_WEBHOOK_URL secrets.
  2. Import the flow, set jdbc_url plus username, then list the schemas in mview_owners.
  3. Run it with the defaults and read which views are stale or undetermined.
  4. Set max_age_hours to the freshness your reports actually promise.
  5. When you want it to fix things, run with auto_remediate: true plus dry_run: false, then resume the pause with approved: true.
  6. Enable the daily trigger, which always runs in safe mode.

How to extend

  • Fail the run as soon as the audit finds anything, for a read-only freshness gate in front of a reporting job.
  • Call DBMS_MVIEW.EXPLAIN_MVIEW on the fast_refreshable: NO views to get the exact reason a fast refresh is impossible, then create the missing materialized view logs.
  • Refresh a dependent set in one call by passing a comma separated list to DBMS_MVIEW.REFRESH, which keeps those views consistent with each other.
  • Add a per view freshness policy from a table instead of one max_age_hours for the whole schema.
  • Switch the age predicate to stale_since, so the measure becomes how long the view has been wrong rather than how long since it was refreshed.
  • Compare the view's own aggregate against the base table after the refresh, for a value level check on top of the catalog check.
  • Loop over several instances to check every database from one flow.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.