Hi everyone!
Our DE team wants to find a better tool to give to the data transformer (e.g. analytics engineers, PAs) to grant access to some datasets. Example, I have created a transformation and at the end have an aggregated data that is joined from more than 2 different tables. It is kind of not a semantic table any more, rather than my own table. How can I, independently of DE team grant a query-only access to it t the data consumers? The first thing that comes to mind is Terraform. But it is too complicated. Any suggestions here. We are using dbt core.
We ran into a similar challenge — giving query-only access to transformed datasets without involving the full infra/DevOps team every time.
Terraform is powerful but definitely overkill for data teams just needing to manage dataset access.
A few lighter alternatives you could explore:
dbt + dbt-permissions (if using Snowflake or BigQuery): handles role-based access logic within dbt runs
Dataform (now part of Google Cloud) – similar to dbt but with more UI-based access controls
Apache Ranger or Immuta – more enterprise-level, but lets you manage fine-grained access policies
Ansible + Attune – this one’s more for admin purposes, but we’ve used it to automate access grant scripts via SQL for specific datasets and users; works across Linux/Windows nodes
Manual GRANT logic in dbt models or post-hooks – can be effective if your team wants to keep everything in Git but avoid a full Terraform setup
In our case, we used dbt post-hooks to issue GRANTs at the end of relevant models and tied that into GitOps flows. Keeps things version-controlled but still light enough for analytics teams to manage.
Hope that helps!