What is the best strategy for managing slowly changing dimensions (SCD Type 2) in dbt?
I am new to using dbt (data build tool) with my data warehouse. I need to track historical changes in our "Customer" table—specifically when their subscription tier changes. Is using the dbt 'snapshots' feature the industry standard for SCD Type 2, or is it better to write custom SQL logic to handle start and end dates manually for better control over the warehouse performance?
2025-11-21 in Software Development by William Taylor
| 13272 Views
All answers to this question.
One thing to watch out for is that snapshots are "state-based." If you miss a day of running your dbt jobs, you might miss a temporary state if the record changed twice.
Answered 2025-11-09 by Mary Higgins
-
Mary is right. For mission-critical financial data, some people prefer log-based CDC (Change Data Capture) to ensure every single micro-change is captured regardless of job frequency.
Commented 2025-12-01 by Heather White
dbt snapshots are the preferred way to handle SCD Type 2 because they are incredibly simple to set up and very robust. You just define the source table and a unique key, and dbt handles the logic of inserting new rows and updating the dbt_valid_to timestamps for you. Writing custom SQL for this is often error-prone, especially when handling "late-arriving" data or multiple changes within a single batch. Stick with snapshots unless you have a very specific performance edge-case. The code is cleaner, and it makes it much easier for other developers to understand your lineage.
Answered 2025-11-23 by Heather White
Do you need to track changes on every single column in that table, or are you only looking for changes in a specific "status" or "tier" field to trigger a new record?
Answered 2025-11-25 by Brian Foster
-
Brian, we only really care about the "Subscription_Tier" and "Region" columns. I saw that dbt allows a check_cols strategy for snapshots, which seems perfect. We can list just those two columns, and dbt will only create a new row if one of those specifically changes. This will prevent our dimension tables from bloating with millions of rows every time a customer updates a minor detail like their phone number or profile picture, which keeps our warehouse much leaner and faster.
Commented 2025-11-27 by Steven Clark
Write a Comment
Your email address will not be published. Required fields are marked (*)

