Hi all,
For data warehouse ETL pipeline, which single target table can came from multiple source systems. For example, Payment Transaction table which come from 2 source: source Loan payment and source Credit Card payment system and Payment Transaction table has field ‘Payment_Type’ for separate two type of transaction. Two sources have difference data-ready time, then should be design 2 dbt jobs for each of source, and write into same Target table.
Could dbt jobs can run independently, and sometimes 2 jobs run parallel, and how to design effective pipeline on this case?
Hope your guideline.