The problem I’m having
Calling a macro in a posthook does not return the correct schema. For some reason in the posthook the schema being used is the default one instead of the custom one defined in the dbt_project.yml file.
The context of why I’m trying to do this
I am trying to set and unset masking and row policies to snowflake tables and views. I encapsulated the logic in a macro and I am calling it in a posthook.
What I’ve already tried
-
Using the custom generate_schema_name() macro, with regards to the following note:
”dbt ignores any custom generate_schema_name macros that are part of a package installed in your project.”The generate_schema_name() macro works correctly when executing custom materializations, except for the posthook part.
{% macro generate_schema_name(custom_schema_name, node) -%}
{%- set default_schema = target.schema -%}
{%- if custom_schema_name is none -%}
{{ default_schema }}
{%- else -%}
{{ custom_schema_name | trim }}
{%- endif -%}
{%- endmacro %}
- Setting the config at the top of the model, with nested curlies surrounding the macro that should be called in the posthook.
{{ config(post_hook="{{ apply_policies_test() }}") }}
- Setting the config at the top of the model, explicitly defining the schema, also with nested curlies surrounding the macro that should be called in the posthook.
{{ config(schema='NS', post_hook="{{ apply_policies_test() }}") }}
The 4 variables/attributes below all return the default schema.
target.schema
schema
this.schema
model['schema']
- I tried to wrap the macro into an {% if execute %} condition. This correctly returns the custom schema. But I get an error and a warning.
- Error: “Unhandled error while executing cannot access local variable ‘connection’ where it is not associated with a value”
-In a different thread someone mentioned that having comments in the macro definition can cause errors, thus I removed every comment but I still receive the error. - Warning: “Snowflake adapter: [WARNING]: Explicit transactional logic should be used only to wrap DML logic (MERGE, DELETE, UPDATE, etc). The keywords BEGIN; and COMMIT; should be
placed directly before and after your DML statement, rather than in separate
statement calls or run_query() macros.“
-I use the run_query() function within the macro, but this warning could possibly be ignored I assume. Please correct me if I am wrong.
dbt version used: 1.8.0