Webhook icon
Schedule icon
If icon
Fail icon
Request icon
Script icon
Process icon
Queries icon
Query icon
SlackIncomingWebhook icon
Pause icon
LoopUntil icon

Catch Stale, Broken or Drifting Chainlink Price Feeds Before Your Protocol Uses Them

Detect stale, zero, negative or drifting Chainlink price feeds and down L2 sequencers, then pause your protocol with approval.

Categories
BusinessInfrastructure

A lending protocol, a perp DEX or a stablecoin trusts whatever latestRoundData() returns. When a feed stops updating, its last price stays on-chain and looks perfectly valid: borrowers can borrow against a collateral price that is hours old, and liquidations fire, or fail to fire, at the wrong price. Chainlink documents the checks consumers should make (an answer above zero, an update within the heartbeat, the L2 sequencer being up), but many contracts skip some of them, and nobody watches the feeds from the outside.

This blueprint watches every feed your protocol reads, on any EVM chain, with plain JSON-RPC eth_calls. It reports each problem once, compares each feed with a second feed or CoinGecko, and on a critical problem lets on-call pause your own contract through a guardian signer. Kestra never holds keys.

How it works

  1. every_five_minutes (Schedule) is the backstop: a stale feed emits no event, so only polling catches it. incident_webhook (Webhook) runs the same check at once when your alerting sees a consumer revert or a feed go quiet.
  2. validate_inputs (If) rejects malformed Label=0xfeed=heartbeat[=reference] entries, duplicates and bad pause settings before any RPC call.
  3. read_feeds sends one JSON-RPC batch: the chain id, the latest block, and latestRoundData(), decimals() and description() of every feed and reference feed, plus the L2 sequencer uptime feed and your contract's paused() when configured.
  4. fetch_reference_prices (inside has_api_references, an If) gets CoinGecko prices for coingecko: references. It may fail without stopping the run.
  5. evaluate (Python, standard library) decodes the ABI results and checks each feed against the latest block's timestamp, which is the clock your contract sees:
    • stale: last update older than the heartbeat plus grace_seconds;
    • invalid_answer: an answer of zero or below;
    • carried_over: answeredInRound < roundId;
    • unreadable: latestRoundData() reverts (a deprecated feed) or returns nothing (no feed at that address on this chain);
    • deviation: the price is more than deviation_warn_percent / deviation_critical_percent away from the reference;
    • wrong_feed: description() does not match the label, which catches a copy-pasted address;
    • sequencer_down / sequencer_grace: on an L2, the sequencer is down or came back less than sequencer_grace_seconds ago.
  6. track (Postgres) compares each feed's set of problems with the last one in price_feed_watch, logs every change to price_feed_watch_log, and decides what to report and propose.
  7. report (If) posts new problems and recoveries to Slack.
  8. propose (If) asks on-call, through a Pause of up to 1 hour, whether to Pause the consumer. The answer:
    • check_not_paused reads paused() again, since someone may have paused it while the question waited;
    • send_pause sends pause() from the guardian through your signer;
    • wait_for_receipt (LoopUntil) waits for the receipt, and check_paused reads paused() to confirm.

Error handling

  • One alert per change, not one per run. A feed is reported when its set of problems changes and when it recovers. A conditional UPDATE ... RETURNING makes this exactly-once even when a burst of webhook calls starts several runs at the same moment. A critical feed is proposed again only every remind_minutes.
  • A broken reference never hides a broken feed. If CoinGecko is down or rate-limited, the run ends WARNING, each affected feed says "deviation not checked", and the staleness and answer checks still run. A reference feed that is itself stale or invalid is not used for comparison.
  • A pause is verified, not assumed.
    • If the contract was paused while the question waited, nothing is sent (a second pause() would revert).
    • If the signer refuses (for example "No Signer available" for the guardian), the run fails with that message and says the consumer is NOT paused.
    • If the transaction reverts, or is mined but paused() is still false, the run fails the same way.
  • No silent gaps. Malformed entries fail before any RPC call. An address with no feed is reported as CRITICAL, not skipped. The RPC call retries with backoff; if it still fails, the errors alert names the task and cause, and says the feeds were not checked in this run.

What this blueprint teaches about Kestra

  • Reading smart contracts with only the core HTTP task: a JSON-RPC batch built with a Pebble loop in flow variables, decoded by a short Python script whose ::{"outputs": ...}:: line becomes task outputs.
  • Degrading gracefully: an optional data source behind allowFailure, read with ??, so its outage downgrades the run to WARNING instead of stopping the checks that matter.
  • Change detection in one Postgres statement, exactly-once across concurrent executions, without a concurrency limit that would block checks while a Pause waits.
  • A human-approved circuit breaker: Pause, a pre-check, an on-chain action, a LoopUntil for the receipt, and a read-back that confirms the effect.

Prerequisites

  • A JSON-RPC endpoint for the chain (any provider).
  • A Postgres database and a Slack incoming webhook.
  • Optional, for the pause answer: your contract with pause() and paused(), a guardian address allowed to call it, and a signer for that guardian (Web3Signer, a KMS relayer, a Safe module relayer) on a private network.

Secrets

  • EVM_RPC_URL: JSON-RPC endpoint.
  • EVM_SIGNER_URL: signer accepting eth_sendTransaction for the guardian (only for the pause answer).
  • POSTGRES_URL, POSTGRES_USER, POSTGRES_PASSWORD: state and change log.
  • SLACK_WEBHOOK_URL: Slack incoming webhook.

Inputs

  • feeds (ARRAY): Label=0xfeed=heartbeatSeconds[=reference], where the reference is a feed address or coingecko:<coin id>. The defaults are ETH/USD, BTC/USD and USDC/USD on Ethereum mainnet.
  • grace_seconds (INT, default 120): allowed delay past the heartbeat.
  • deviation_warn_percent, deviation_critical_percent (FLOAT, default 1 / 3).
  • price_api_url (STRING): CoinGecko-compatible simple/price endpoint.
  • sequencer_uptime_feed (STRING, optional), sequencer_grace_seconds (INT, default 3600): for L2s.
  • pause_target, guardian (STRING, optional): your pausable contract and its guardian. Without them the flow only alerts.
  • remind_minutes (INT, default 60), explorer_url, kestra_url (STRING).

Outputs

  • feeds (JSON): every feed with its price, round, age, reference price, deviation, status and problems. Empty when the run fails.

Quick start

  1. Add the secrets and run the flow with the defaults against an Ethereum mainnet RPC. It stays quiet when the feeds are healthy; the feeds output shows each price, its age and its deviation (in our test run: ETH/USD 2542.30 against CoinGecko's 2542.63, 0.013%).
  2. Replace feeds with the feeds your contracts read, one flow per chain, with the heartbeats listed on data.chain.link. On an L2, set sequencer_uptime_feed.
  3. Point your consumer-revert alerts at the incident_webhook and change its key.
  4. To use the circuit breaker, set pause_target and guardian, and give the signer the guardian key.

How to extend

  • Add Fail after track to fail CI or a deployment pipeline when any feed is critical.
  • Use a Pyth or RedStone adapter that exposes latestRoundData(); the checks are the same.
  • Replace the Slack task with PagerDuty for critical feeds.

Links

See How

New to Kestra?

Use blueprints to kickstart your first workflows.