Tutorials
REMOVE ME OVERVIEW OF STRUCTURE HERE - REPLACE ME
Structure¶
In general, Hubs consist of 4 columns, described below.
Column type (parameter name)¶
e.g. Primary key (src_pk)
Creating models¶
Create a new dbt model as before. We'll call this one <example structure name>
.
REMOVE ME Example structure name should be something fitting in with a CUSTOMER/ORDER model, as the rest of the tutorials are.
1 2 |
|
To create a
Materialisation¶
The recommended materialisation for <recommended materialisation>
, as we load and add new records to the existing data set.
REMOVE ME Recommended materialisation type should be incremental, generally.
Adding the metadata¶
Let's look at the metadata we need to provide to the
We provide the column names which we would like to select from the staging area (source_model
).
Using our knowledge of what columns we need in our <example structure name>
REMOVE ME Below values are left as examples. CHANGE AS REQUIRED
Parameter | Value |
---|---|
source_model | v_stg_orders |
src_pk | CUSTOMER_HK |
src_nk | CUSTOMER_ID |
src_ldts | LOAD_DATETIME |
src_source | RECORD_SOURCE |
When we provide the metadata above, our model should look like the following:
REMOVE ME: CHANGE BELOW SNIPPET AS APPROPRIATE
1 2 3 4 5 6 7 8 9 10 11 |
|
Note
See our metadata reference for more detail on how to provide metadata to
Running dbt¶
With our metadata provided and our model complete, we can run dbt to create our <example structure name>
dbt run -m +<example structure name>
The resulting
REMOVE ME Below values are left as examples. CHANGE AS REQUIRED ENSURE TABLE FORMATTING IS KEPT NEAT, IT MAKES IT EASIER TO EDIT
CUSTOMER_HK | CUSTOMER_ID | LOAD_DATETIME | SOURCE |
---|---|---|---|
B8C37E... | 1001 | 1993-01-01 00:00:00.000 | 1 |
. | . | . | 1 |
. | . | . | 1 |
FED333... | 1004 | 1993-01-01 00:00:00.000 | 1 |
REMOVE ME: OPTIONAL SECTIONS
STRUCTURE SPECIFIC GUIDANCE¶
REMOVE ME e.g. Hubs and Links have a section here about how to load hubs and links from multiple sources, and how to use the macro to do so