dltHub builds dlt, the open-source Python library that loads data from APIs, databases and files into well-structured datasets. Kestra schedules, retries and monitors those pipelines alongside the rest of the data stack.
The dlt plugin runs dlt pipelines as Kestra tasks, and any dlt script also runs in a Python script task.
Run an inline dlt script with the Run task, or call the dlt command line with the CLI task, all from a YAML flow.
dlt stores cursor state in the destination and merges on primary keys, so a Kestra retry never duplicates rows.
Freeze a dlt schema contract and the run fails before a new column or changed type reaches the destination.
Start loads on a cron or an incoming webhook, then post row counts or failures to Slack and Discord.
A declarative dlt pipeline pulling two REST endpoints into DuckDB with merge write disposition, per-table row counts, and Slack reporting.
Sync Pipedrive CRM data into BigQuery every hour with a dlt Python pipeline orchestrated by Kestra.
Extract JSON from a REST API and load it into DuckDB with dlt, fully orchestrated in a single Python task.
