Right now I’m trying to install dbt packages where we are trying to utilize packages that are sourced from our dev and prod data lake environments and then create stg reports in dev and prod data warehouse and then ultimately mart models in our dbt prod and dev data marts.
What is the best way to install dbt packages so that they target dev and prod data lakes at the same time and also how can I make sure that we take the same approach with each package we want to use going forward.
right now I’m doing this
target-path: “target”
clean-targets:
- “target”
- “dbt_packages”
vars:
fivetran__remove_empty_table_warnings: true
project_data_lake: “{{ ‘prod-data-lake’ if target.name == ‘prod’ else ‘dev-data-lake’ }}”
project_data_warehouse: “{{ ‘prod-data-warehouse’ if target.name == ‘prod’ else ‘dev-data-warehouse’ }}”
project_data_mart: “{{ ‘prod-data-mart’ if target.name == ‘prod’ else ‘dev-data-mart’ }}”
five tran ad reporting package variables turn off any source that we do not use or have
ad_reporting__amazon_ads_enabled: False
ad_reporting__apple_search_ads_enabled: False
ad_reporting__facebook_ads_enabled: True
ad_reporting__google_ads_enabled: True
ad_reporting__linkedin_ads_enabled: False
ad_reporting__microsoft_ads_enabled: False
ad_reporting__pinterest_ads_enabled: False
ad_reporting__reddit_ads_enabled: False
ad_reporting__snapchat_ads_enabled: False
ad_reporting__tiktok_ads_enabled: False
ad_reporting__twitter_ads_enabled: False
ad_reporting__account_passthrough_metrics:
- name: view_through_conversions
ad_reporting__campaign_passthrough_metrics:
- name: total_shares
ad_reporting__ad_group_passthrough_metrics:
- name: interactions
ad_reporting__ad_passthrough_metrics: ## For both Ad and URL reports
- name: video_views_captured
ad_reporting__keyword_passthrough_metrics:
- name: interactions
ad_reporting__search_passthrough_metrics:
- name: local_spend_amount
ad_reporting__url_report__using_null_filter: False
facebook_ads_schema: fb_ads_data
facebook_ads_database: “{{ var(‘project_data_lake’, ‘dev-data-lake’) }}”
google_ads_schema: google_ads_data
google_ads_database: “{{ var(‘project_data_lake’, ‘dev-data-lake’) }}”
though in the past when I’ve had to do our own proprietary models this usually works
Any help with this, is much appreciated.