How can I test that a model's columns defined in YAML match a model's actual columns?

Help
, , , ,
1

I am trying to write a generic model-level test that will check that the columns a model has defined in YAML match the columns of the materialized relation.

I thought I would compare the model graph object (detailed here) with the actual relation (currently using dbt_utils.get_filtered_columns_in_relation). However, I can’t seem to get access to the model graph. It seems that when running tests, model is not the graph object but a different Python object representing the relation (in this case I’m using Snowflake so a dbt.adapters.snowflake.relation.SnowflakeRelation).

Is there some way to get access to the model graph in the test context? Is there a better way to accomplish what I’m trying to do?

Full WIP test:

-- tests that the column list that appears in YAML for a model is the same as
--     the actual columns in the generated table/view

{% test expected_columns_match(model) %}

-- if you run outside of this context, `model` is then the graph object for the generated *test* model, not the tested model
{% if execute %}
{{ log("Logging data from: " ~ model.name, True) }}
-- note: model.columns doesn't exist. `model` is not a graph object, it's a "dbt.adapters.snowflake.relation.SnowflakeRelation", which seems to be a test-specific thing.
--       until this is figured out, this test won't work!
{% set documented_column_names = model.columns.values() | map(attribute='name') | list %}
{% set actual_column_names = dbt_utils.get_filtered_columns_in_relation(from=ref(model.name)) %}
{{ log("documented_column_names: " ~ documented_column_names, True) }}
{{ log("actual_column_names: " ~ actual_column_names, True) }}

    with cols as (
        select
            ARRAY_CONSTRUCT('{{ documented_column_names | join("', '") }}') as documented,
            ARRAY_CONSTRUCT('{{ actual_column_names | join("', '") }}') as actual
    )

    select * from cols where ARRAY_SORT(documented) <> ARRAY_SORT(actual)
{% endif %}

{% endtest %}