# Sampled development environments in dbt

**URL:** <https://discourse.getdbt.com/t/sampled-development-environments-in-dbt/353>\
**Category:** Archive\
**Created:** [May 8, 2019, 12:42pm UTC](https://discourse.getdbt.com/t/sampled-development-environments-in-dbt/353 "2019-05-08T12:42:28Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![sjwhitworth](https://avatars.discourse-cdn.com/v4/letter/s/6a8cbe/32.png) [@sjwhitworth](https://discourse.getdbt.com/u/sjwhitworth)\
**Post date:** [May 8, 2019, 12:42pm UTC](https://discourse.getdbt.com/t/sampled-development-environments-in-dbt/353/1 "2019-05-08T12:42:28Z")

</div>

Does anyone have any design patterns for building sampled development environments in dbt? This is so analysts can test things out quickly against production, without being slowed down by the large size of the dataset in prod. I’m currently just doing something like the following (sample a random 10%, up to a variable amount), but it looks a little messy:

```auto
{% if target.name == "dev" %}
    AND MOD(CAST(RAND()*10 AS INT64),10)=0
    LIMIT {{ var("max_results_in_dev") }}
{% endif %}

```

In general, I’d love to see any guidance or wisdom people have around the intricacies of introducing dbt into teams / production settings, et al.

---

<div class="post-metadata">

**Author:** ![tristan](https://sea2.discourse-cdn.com/flex020/user_avatar/discourse.getdbt.com/tristan/32/15_2.png) [@tristan](https://discourse.getdbt.com/u/tristan)\
**Post date:** [May 8, 2019, 2:02pm UTC](https://discourse.getdbt.com/t/sampled-development-environments-in-dbt/353/2 "2019-05-08T14:02:29Z")

</div>

I’ve seen companies effectively deploy your strategy @sjwhitworth, but it can require a lot of work. Overall here are my suggestions (feel free to ignore if you find another way that works for you!):

1. keep all dimension data. don’t leave out certain customers, etc. this is onerous both to write and maintain: it requires a lot of sprawl-y code to make things consistent everywhere. it’s also hard to test.
2. limit large fact tables to a configurable # of days prior to `current_timestamp()`. configure via `var()`.
3. apply all filtering within base models, and probably materialize these base models as tables so that you’re not executing the where clause over and over again. this depends on what your DAG looks like though.
4. try to implement filtering on top of efficient keys depending on your data store. partitions in bigquery, sorts in redshift, etc.

Our just-released [best practices guide](https://docs.getdbt.com/docs/best-practices#section-limit-the-data-processed-when-in-development) shares code on how to implement #2, above.

Hope that’s helpful!
