I want to start a small hobby project. My goal is to collect financial data from a few APIs in a database and then make them evaluable with dbt and Lightdash.
So far, I have no experience with dbt and for databases I have always been on relational databases. The JSON data from the API is nested.
From everything I have read so far, I have to transform the JSON objects into a relational DB to make them evaluable with dbt. Is that correct? Or can I also store the JSON data in a MongoDB for example and then use dbt?
If I should transform data: Which Python libraries are suitable for this? Is it worth taking a closer look at Apache Airflow? I would like to avoid having to set up a relational data model myself. That can be very time-consuming.
I know the questions are very general. But I’m looking for a certain direction from practitioners. When googling for keywords, it is always overwhelming.