Power BI Refresh with a Retry Policy for Flaky Gateway Sources

Stop babysitting flaky refreshes. Datasets fed through an on-premises data gateway fail for reasons that fix themselves: a gateway restart, a source database under momentary load, a network blip. Without a retry policy, each blip becomes a red execution and a human clicking refresh again. This blueprint attaches a task-level retry block to io.kestra.plugin.powerbi.RefreshGroupDataset, so transient failures are absorbed silently and Slack only hears about failures that survive three attempts.

How it works

1. refresh_dataset (io.kestra.plugin.powerbi.RefreshGroupDataset) authenticates with the Azure AD service principal and triggers the refresh. With wait: true it polls every 15 seconds for up to 30 minutes, and a refresh that Power BI reports as failed fails the attempt.
2. The task-level retry block (type: constant, interval: PT5M, maxAttempt: 3) re-runs the whole attempt, trigger plus wait, up to three times with five minutes between attempts, enough for a gateway restart or a momentary source outage to clear.
3. notify posts the final status to Slack on success; whether it took one attempt or three is visible in the execution's attempt history.
4. The errors block fires only after the last attempt fails, so the alert means a persistent problem, not noise.

What you get

• Transient gateway and source failures handled without a human in the loop.
• An alert channel that stays quiet unless something is genuinely broken, keeping trust in the alerts high.
• A verified wait on every attempt, so a retry is triggered by an actual failed refresh state, not a lost request.
• Attempt history per execution, showing how flaky the source really is over time.

Who it's for

• Teams refreshing datasets through an on-premises data gateway with occasional hiccups.
• BI owners who currently re-trigger failed refreshes by hand.
• On-call engineers drowning in alerts that resolve themselves five minutes later.

Why orchestrate this with Kestra

Power BI has no retry policy for refreshes: a failed scheduled refresh stays failed until the next window. Kestra adds a declarative retry with a chosen interval and attempt cap around the unchanged refresh call, plus the execution history to distinguish a flaky source from a broken one.

Prerequisites

• An Azure AD app registration with Power BI API permissions granted, authenticating through the OAuth 2.0 client credentials flow.
• Service principal access enabled for Power BI in the admin portal, and the service principal added to the target workspace.
• The workspace id (groupId) and dataset id (datasetId) of the model to refresh.
• A Slack incoming webhook for confirmations and exhausted-retry alerts.

Secrets

• AZURE_TENANT_ID: Azure AD tenant id.
• AZURE_CLIENT_ID: Azure AD application (client) id.
• AZURE_CLIENT_SECRET: Azure AD client secret.
• SLACK_WEBHOOK_URL: Slack incoming webhook URL.

Quick start

1. Add the four secrets to your Kestra namespace.
2. Replace groupId and datasetId with your workspace and dataset ids.
3. Execute the flow and confirm the Slack success message.
4. Tune interval and maxAttempt to how long your gateway blips usually last.

How to extend

• Switch the retry type to exponential for backoff when the source needs longer to recover.
• Add a Schedule trigger to combine the retry policy with a morning run, as shown in the morning refresh blueprint.
• Apply the same retry block inside a ForEach to make a whole dataset fleet self-healing, as shown in the multi-dataset refresh blueprint.
• Route the exhausted-retry alert to a paging channel while successes go to a routine channel.

Links

• Power BI plugin documentation
• Kestra retry documentation