This walk-through intends to give you a detailed understanding of how to use dbtvault and the provided macros to develop a Data Vault Data Warehouse from the ground up. If you're looking to quickly experiment and learn using pre-written models, take a look at our worked example.
In this section we teach you how to use dbtvault step-by-step, explaining the use of macros and the different components of the Data Vault in detail.
- process a raw staging layer.
- create a Data Vault with hubs, links and satellites using dbtvault.
Some prior knowledge of Data Vault 2.0 architecture. Have a look at How can I get up to speed on Data Vault 2.0?
A Snowflake account, trial or otherwise. Sign up for a free 30-day trial here
You must have downloaded and installed dbt and set up a project.
Sources should be set up in dbt (see below).
We assume you already have a raw staging layer.
Our macros assume that you are only loading from one set of load dates in a single load cycle (i.e. your staging layer contains data for one
load_datetimevalue only). We will be removing this restriction in future releases.
You should read our best practices guidance.
Setting up sources (in dbt)¶
We will be using the
source feature of dbt extensively throughout the documentation to make access to source
data much easier, cleaner and more modular.
We have provided an example below which shows a configuration similar to that used for the examples in our documentation, however this feature is documented extensively in the documentation for dbt.
We recommend that you place the
schema.yml file you create for your sources,
in the root of your
models folder, however you can place it wherever needed for your specific project and models.
version: 2 sources: - name: my_source database: MY_DATABASE schema: MY_SCHEMA tables: - name: raw_orders - name: ...
Add the following to your
packages: - git: "https://github.com/Datavault-UK/dbtvault" revision: v0.6.1