Hi,
I’m working on a small project and I need to build a continuous data pipeline with fivetran and dbt. Here is a high level detail on the project.
Get the data from SFTP location to load it to snowflake. Use dbt for transforming the SFTP file and post the curated data in snowflake tables.
I understand Fivetran is used for Extract and Load and dbt is used for transformation. We have license for dbt cloud and fivetran.
Question 1 : What is the difference between using dbt transformation in fivetran and having the project in dbt cloud. How are are they different and what are the benefits and use cases?. We don’t have a whole lot of documentation related to this.
Question 2 : How do we build a continuous data pipeline without having any data orchestration tools. is there is a way to call fivetran connectors from dbt or call dbt jobs immediately after fivetran data sync?
Question 3 : Using Fivetran and dbt is more of a ELT model. Is there is a way to design a ETL model with these tools? what i’m trying to understand is to extract data using fivetran and transform the data using dbt without loading the data to destination and then load the curated data to destination from dbt.
Any help is much appreciated. Thanks.