Hi, everyone:
I have a dbt project, and some models. The project can run properly. But one model is huge, it cause some other models run at the same time goes wrong(5 threads used). Can I specify the run order of models, so when this model run, others are waiting. Or the model run thread can be specified per model.
Thanks!
Does --models
and --threads
can be used to config it?
The command dbt run --models +my-model --threads 1
is used to run whole project, and set thread to 1 when my-model running, it’s right?
I test on it. This method doesn’t work.
check this About ref function | dbt Developer Hub . If you make the proper references, the flow should execute in order, first the dependencies and then the final code
Sorry for delay reply.
This model is one of some beginning stages, and uses lots of comput resource. When I set thread more than 1, then the parallel jobs may fail; When I set the thread to 1, all the models run one by one, that will cause a long run time. So I want to run this model first using 1 thread, then others as usual.
If the thread config can be applied to a model, all thing will be great!
Thanks
You could try force a dependency via a comment in the other models?
– Depends on: {{ ref(‘The Long Running Model’) }}
This will affect lineage, therefore (presumably) build order.
thanks, janine.rawnsley.
I search the docs,but cant find the config block. Where can I perform this config?
If it works, it’ll be the simplest and most directly way.
Thanks again.
Found it! I’ll try!
Thanks again.