DBT Cloud webhook with AWS lambda

Help
1

The problem I’m having

I can’t seem to configure a webhook that triggers an AWS lambda

The context of why I’m trying to do this

I’m trying to trigger an AWS lambda after a dbt job ends (either successfully or failed). I was looking into creating a webhook to achieve this.

What I’ve already tried

I’ve tried enabling a Lambda endpoint but I don’t want to expose it to the public internet. the other option AWS offers is through an IAM role which would allow authentication from dbt cloud. This one doesn’t seem to work because on the webhook configuration there’s no mention to any role. Unfortunately neither work… is there any other option or workaround that you’ve found?

2

You must expose your endpoint over the public internet, because dbt Cloud is going to attempt to communicate with you over the public internet. There is no alternative to this.

dbt Cloud does not provide custom authentication options (ie, providing account credentials on AWS) but it does provide a way to validate requests https://docs.getdbt.com/docs/deploy/webhooks#validate-a-webhook

Note: @Mike Stanley originally posted this reply in Slack. It might not have transferred perfectly.

1 Like
3

Got it, that was what my hunch. Is there any setup that I need to do to allow dbt Cloud to talk to the lambda? Meaning having some security groups to be set up, etc?