on-run-end with failed models

Help
1

The problem I’m having

I’m using the dbt Artifacts package to collect and insert information about runs, but I’m only getting information about runs that complete. The upload process is triggered by an on-run-end hook in dbt_project.yml and it looks like it’s not getting triggered when runs fail.

In a related post someone said that on-end-run hooks fire on unsuccessful models, so I’m a bit confused about what’s happening.

Is there a way to use on-run-end hooks or another method to upload information about failed runs with dbt Artifacts?

2

@gyoll
on-run-end hook will fire only once i.e only at the end of the following commands

  • dbt run
  • dbt test
  • dbt seed
  • dbt snapshot
  • dbt build
  • dbt compile
  • dbt docs generate

even though ur run fails it will be triggered.

May i know the dbt version ur using

3

Your run can be completed without having succesful models, take the examples below

1 - If your model raises an error, the run will continue

2 - If you are using fail fast and your model raises an error, dbt will exit and your run will not complete

In the first case the on-run-end should work, in the second case it should not work

1 Like
4

@Surya We are running with dbt version 1.5.1 (in case it ends up being useful, we are running with dbt Artifacts version 2.4.2).

Just to be super clear about this, I’m working with dbt cloud runs here (not local jobs). So we’re running dbt build with a lot of tables but something fails. Then when I inspect the run details in dbt cloud I don’t see the end hooks in the logs, and I don’t see any entries in my artifacts tables. I do see the end hooks for successful runs and the run artifacts are appearing in the tables created by dbt Artifacts.

5

Can you show an example of a failed run? the logs

6

@brunoszdl Thank you! We are using --fail-fast so that must be precluding the on-run-end hook. I don’t suppose there’s any kind of alternative to fire off end hooks when using --fail-fast?

7

I don’t think so, because fail fast causes dbt to exit immediately, so it does not even run the on-run-end hook

Maybe you can raise an issue in the dbt core about it, explaining your problem

You can also talk to Gabriel https://www.linkedin.com/in/gabriel-rangel-8bb275108/

He was having the exact same problem, maybe he has figured something out

1 Like
8

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.