As discussed in the dbt Slack I am moving this to the discourse:
I have a very general question for you and it is something i’ve been wondering a lot recently.
According to classical business intelligence theory a Data Warehouse should include some kind of transformation/data modelling like normalization, star-/snowflake-/galaxy- and/or data vault schema. Often you hear a mixture of some of those system being applied, most commonly normalization or data vault and then e.g. a star schema for end user access to the data via the BI Tool…
However, I feel like this is something mostly huge corporations are doing as the need for such a transformation is much higher because of much bigger data / way more different data sources / a much bigger data analysis department and other reasons.
To me it seems as if a lot of “younger” companies that don’t have that much data / different sources / not as many data people are skipping these classical transformations or use it only partly (e.g. when giving access to colleagues transform the data from raw to star schema right away).
I am wondering now… is this something that is actually happening? Or are most of you actually following that “classical” way?
How are you approaching that? Are you using these modelling techniques in your company? And if so how does your data stack look like (like how many data people, how many sources, small or big data - of course only if you are happy to share it )
I would be very interested in getting an idea about how other companies are approaching this and how commonly people are following the standard way or deviate from it - looking forward to your experiences!
PS: I know that there is already a great discussion about Kimball modeling here: Is Kimball dimensional modeling still relevant in a modern data warehouse? . However I am more interested in how people are actually setting up their warehouses in their companies and what factors are influencing which decisions. Also I am not necessarily talking about Kimball only.
Thanks in advance