The problem I’m having
When I run dbt compile
, I have noticed that snapshots do not get compiled as a SQL file in the /target
folder thus leading to the compiled_path
field to be null. Is there a reason for this?
The context of why I’m trying to do this
I am trying to parse the manifest.json
for some lineage but am not able to obtain the full lineage due to snapshots SQL file not present in the /target
folder.
I’m not exactly sure why snapshots don’t get compiled, but my guess is that it’s because your snapshot definition (like select * from {{ source('my', 'table') }}
) is not the same as what is run against the warehouse when you execute dbt snapshot
, where dbt builds temp tables to identify new/changed records.
By contrast, if you manually query the compiled SQL for models (even incremental ones!) and tests, you’ll get the same results as what would be inserted into your table.
Regardless, you shouldn’t need the file path to generate lineage, you can use the depends_on
array inside of the manifest instead.