Custom materialisation

Help
,
1

The problem I’m having

As a test I’ve copied an existing incremental materialisation from github:

to the macros folder in my dbt project.

I’ve Renamed the materialisation from incremental to incrementalx
{% materialization incremental, default -%}
to:
{% materialization incrementalx, default -%}

And used this in a model that is set to use incremental materialisation in the model’s config block:

{{
config(
materialized=“incrementalx”,
…
Model structure:
{% if is_incremental() %}
“incremental select”
{% else %}
“inital load select”
{% endif %}

The standard incremental materialisation worked fine, the incrementalx materialisation (only a copy from git and a rename) did not, for some reason it used the select statement that was inside the
else part (so the initial load part) and used this as a basis for the merge (and that failed).
When switching the materialisation back from incrementalx to incremental the correct incremental select was used and the merge statement completed as expected.

The context of why I’m trying to do this

In order to create a custom incremental strategy whilst not altering existing functionality.

What I’ve already tried

Described above, i wanted to make sure that everything worked as expected, i wanted to use the existing materialisation as a base (since that works and i only need a small tweak)

Some example code or error messages

Described above, the incorrect select statement was used as a tmp table.

Put code inside backticks
  to preserve indentation
    which is especially important 
      for Python and YAML!
2

The is_incremental macro checks if the model is materialized as ‘incremental’. It will fail if it is anything else

You would need to overwrite this macro, or create a copy of this macro like ‘is_incrementalx’ like you did for the materialization

3

Ah thanks, that seems to do the trick,

I assume that I can place a copy in my macros folder and that I should keep the macro name is_incremental (overloading) so it gets “picked up” by dbt, the difference would be to change:
model.config.materialized == ‘incrementalx’ and everything works as expected.

If I want to keep this “seperate” (not to interfere with existing incremental) I can’t just rename the macro as well to lets say incrementalx since dbt doesnt know that this needs to be used instead of the original incremental in the case of an incrementalx materialized model?

Oh I just realised I read your article yesterday :smiley:

4

You can have your own is_incrementalx() macro in your macros folder

Then in your model you call it like

{% if is_incrementalx() %}
“incremental select”
{% else %}
“inital load select”
{% endif %}

It should work!

And thanks for reading the post :smiley:

5

Whoops, I completely missed the fact that is_incremental() is the macro I call in my model!

Great thanks!

6

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.