Is it possible to set hive config variables in macro?

Help
1

I’m using dbt core with dbt-hive adapter.
I used to set variables in hive as follows.

set hive.server2.enable.doAs=false;
set hive.server2.tez.default.queues=default;
set hive.server2.tez.initialize.default.sessions=false;
set hive.server2.tez.sessions.per.default.queue=1;

I would like to apply different configurations using macros.
But I got ParseException. (FAILED: ParseException line 19:0 cannot recognize input near ‘set’ ‘hive.server2.enable.doAs’ ‘=’ in create table statement)

{% macro tpch_settings(engine='mr') %}
{% if engine == 'mr' %}
set hive.server2.enable.doAs=false;
set hive.server2.tez.default.queues=default;
set hive.server2.tez.initialize.default.sessions=false;
set hive.server2.tez.sessions.per.default.queue=1;
....
{% endif %}
{% endmacro %}
2

You won’t be able to configure your database connection from inside of a dbt macro in a model definition - by the time that code is being hit, you already have a database connection and as you’ve seen, you’re inside of a create table as select block.

I’m not familiar with hive but recommend you open an issue on their adapter’s repository to describe your use case as it will need to be enabled at the adapter level, not in userspace

3

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.