Parameterized folder/model name

Help
,
1

I want to create a generic job that can execute a specific model that is located in a specific folder.

I already tried this to run all the models inside my_folder:

dbt run -s "{{var('my_folder')}}".*+

and my_folder was defined in dbt_project.yml.

However, this doesn’t work as "{{var('my_folder')}}".*+ were treated as a model name.

Is there a way to implement this or something similar?

Thanks.

2

I don’t think you can use the vars defined in your dbt_project.yml to substitute arguments in your terminal command.

Is it possible to you to use environment variables?

If so, you could do it like

export my_folder='staging'

to define the variable, and then

dbt run -s $my_folder

Then it will run

dbt run -s staging

for example

1 Like
3

Where should I put export my_folder='staging'?

But this is helpful. Thank you.

4

Do you know which shell are you using?

bash, zsh, …?

If you are in doubt just run in your terminal

echo $SHELL
5

I’m afraid I can’t define this in shell. Is there somewhere I can define this programmatically within my dbt project?

6

You can create a file such as .env inside your project root path.

Inside the file you can write export my_folder='staging'.

You just have to run

source .env

before you run your dbt run.

You just have to use the source .env once. But if you close your terminal and open it again you have to do it again.

7

If you’re using dbt cloud, you can define environment variables here: Environment variables | dbt Developer Hub

8

Any other way other than the environment variables in dbt cloud? Nothing that I can do within my dbt project?

9

You can’t use variables defined inside of your dbt project outside of your dbt project - the CLI doesn’t understand Jinja.

Can you say more about your use case? Why do you want to define the list of models you want to run inside of a variable? Doesn’t that mean that if you want to change which models are run, you have to make a change to a source-controlled file instead of just specifying a different selector in the dbt Cloud config UI?

1 Like