About this blueprint
CLI DuckDB Git Local files dbt
This flow runs multiple dbt commands in a single container.
This approach can be useful if you want to run several dbt commands and you want to run them all in one container to minimize latency.
Firstly, the flow creates the working directory. Within the working directory, the flow contains tasks to
- clone the git repository
- create a local file packages.yml mentioning the package and its version
- run the dbt commands in a docker container
yaml
id: dbtGitDockerDuckDB
namespace: blueprint
tasks:
- id: dbt
type: io.kestra.core.tasks.flows.WorkingDirectory
tasks:
- id: cloneRepository
type: io.kestra.plugin.git.Clone
url: https://github.com/kestra-io/dbt-demo
branch: main
- id: packagesConfig
type: io.kestra.core.tasks.storages.LocalFiles
inputs:
packages.yml: |
packages:
- package: brooklyn-data/dbt_artifacts
version: 2.6.2
- id: dbtCore
type: io.kestra.plugin.dbt.cli.DbtCLI
runner: DOCKER
docker:
image: ghcr.io/kestra-io/dbt-duckdb:latest
profiles: |
jaffle_shop:
outputs:
dev:
type: duckdb
path: ":memory:"
extensions:
- parquet
fixed_retries: 1
threads: 16
timeout_seconds: 300
target: dev
commands:
- dbt deps
- dbt build