# Tips and Tricks about working with dbt

**URL:** <https://discourse.getdbt.com/t/tips-and-tricks-about-working-with-dbt/287>\
**Category:** In-Depth Discussions\
**Tags:** developer-ergonomics\
**Created:** [March 27, 2019, 12:50pm UTC](https://discourse.getdbt.com/t/tips-and-tricks-about-working-with-dbt/287 "2019-03-27T12:50:07Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![emilie](https://sea2.discourse-cdn.com/flex020/user_avatar/discourse.getdbt.com/emilie/32/23_2.png) [@emilie](https://discourse.getdbt.com/u/emilie)\
**Post date:** [March 27, 2019, 12:50pm UTC](https://discourse.getdbt.com/t/tips-and-tricks-about-working-with-dbt/287/1 "2019-03-27T12:50:07Z")

</div>

Working with dbt for a while, one can start to develop workflows that are really useful. It would be great to collect these somewhere, so that beginner dbt-ers can benefit from those experiences.

Here is my tip:

- When I was really into R, I would use the beepr package all the time because then I could switch onto a new task without losing site of my priority. Many moons ago, I asked about a built-in dbt beep and Drew taught me the `dbt run .. && say beep`.

What is something you have in your workflow that might make it easier for someone to work in/around dbt?

---

<div class="post-metadata">

**Author:** ![drew](https://sea2.discourse-cdn.com/flex020/user_avatar/discourse.getdbt.com/drew/32/9_2.png) [@drew](https://discourse.getdbt.com/u/drew)\
**Post date:** [March 27, 2019, 1:21pm UTC](https://discourse.getdbt.com/t/tips-and-tricks-about-working-with-dbt/287/2 "2019-03-27T13:21:32Z")

</div>

Nice! The one trick I really like involves using some bash magic to run all of the changed files on a branch.

Snippet:

```auto
dbt run --models $(git diff --name-only | grep '\.sql$' | awk -F '/' '{ print $NF }' | sed 's/\.sql$/+/g')

```

You can save this in your .bashrc with a function, eg:

```auto
function dbt_run_changed() {
    children=$1
    models=$(git diff --name-only | grep '\.sql$' | awk -F '/' '{ print $NF }' | sed "s/\.sql$/${children}/g" | tr '\n' ' ')
    echo "Running models: ${models}"
    dbt run --models $models
}

```

This function takes an optional argument, `+`, that will also run the children of the changed models!

Usage:

```auto
$ dbt_run_changed
$ dbt_run_changed +

```

---

<div class="post-metadata">

**Author:** ![claire](https://sea2.discourse-cdn.com/flex020/user_avatar/discourse.getdbt.com/claire/32/522_2.png) [@claire](https://discourse.getdbt.com/u/claire)\
**Post date:** [March 27, 2019, 6:02pm UTC](https://discourse.getdbt.com/t/tips-and-tricks-about-working-with-dbt/287/3 "2019-03-27T18:02:38Z")

</div>

Here’s my hot tip!

I’m a huge fan of keeping my directories organized in a tree structure. Since I do client work, my tree structure ends up looking like:

```auto
.
└── fishtown
    ├── client
    │ ├── stark-industries
    │ │ ├── stark-industries-dbt
    │ │ └── stark-industries-lookml
    │ └── wayne-enterprises
    │ └── wayne-enterprises-dbt
    ├── dbt
    └── packages
        ├── segment
        └── utils

```

This can be annoying to `cd` into the right folder, so I use the [goto](https://github.com/iridakos/goto) utility to set up aliases for my folders.  
Then I can do things like:

```bash
$ goto stark
$ pwd
./fishtown/client/stark-industries/stark-industries-dbt

```

I also recently set up a bash alias to open GitHub for my current directory (not tested on GitLab! Sorry @emilie) – instructions [here](https://gist.github.com/igrigorik/6666860).

---

<div class="post-metadata">

**Author:** ![claire](https://sea2.discourse-cdn.com/flex020/user_avatar/discourse.getdbt.com/claire/32/522_2.png) [@claire](https://discourse.getdbt.com/u/claire)\
**Post date:** [March 28, 2019, 2:12pm UTC](https://discourse.getdbt.com/t/tips-and-tricks-about-working-with-dbt/287/4 "2019-03-28T14:12:46Z")

</div>

Oh and another one…

Sometimes if my `dbt run` is resulting in an error, I like to cycle my `logs/dbt.log` file before trying to run it again, to make debugging easier.

I have a pretty rudimentary bash function that does this for me:

```bash
function cycle_logs() {
  suffix=$(date '+%Y-%m-%dT%H:%M:%S')
  mv -v logs/dbt.log logs/dbt.log.${suffix}
}

```

---

<div class="post-metadata">

**Author:** ![emilie](https://sea2.discourse-cdn.com/flex020/user_avatar/discourse.getdbt.com/emilie/32/23_2.png) [@emilie](https://discourse.getdbt.com/u/emilie)\
**Post date:** [April 1, 2019, 10:40pm UTC](https://discourse.getdbt.com/t/tips-and-tricks-about-working-with-dbt/287/5 "2019-04-01T22:40:05Z")

</div>

I just set up goto! It’s great! +1 this tip!

---

<div class="post-metadata">

**Author:** ![joshtemple](https://sea2.discourse-cdn.com/flex020/user_avatar/discourse.getdbt.com/joshtemple/32/62_2.png) [@joshtemple](https://discourse.getdbt.com/u/joshtemple)\
**Post date:** [April 3, 2019, 4:54pm UTC](https://discourse.getdbt.com/t/tips-and-tricks-about-working-with-dbt/287/6 "2019-04-03T16:54:11Z")

</div>

Drew, do you think this approach could be used in a CI framework to only build/test/deploy only the models changed by the commits being merged into master?

I’m imagining diffing the feature branch against master and only running those models (with @ and +) in production. Could speed up build/test/deploy times a _lot_ by leaving out the models that aren’t affected.

---

<div class="post-metadata">

**Author:** ![drew](https://sea2.discourse-cdn.com/flex020/user_avatar/discourse.getdbt.com/drew/32/9_2.png) [@drew](https://discourse.getdbt.com/u/drew)\
**Post date:** [April 3, 2019, 5:08pm UTC](https://discourse.getdbt.com/t/tips-and-tricks-about-working-with-dbt/287/7 "2019-04-03T17:08:11Z")

</div>

Cool idea! This would definitely be a good starting point, but it won’t hold up for changes to macros or `dbt_project.yml`, for instance. dbt doesn’t currently provide a rock-solid way of understanding which models are impacted by changes to a given set of _files_, but that’s certainly something it could do some day!

---

<div class="post-metadata">

**Author:** ![ryantuck](https://sea2.discourse-cdn.com/flex020/user_avatar/discourse.getdbt.com/ryantuck/32/196_2.png) [@ryantuck](https://discourse.getdbt.com/u/ryantuck)\
**Post date:** [July 2, 2020, 5:53pm UTC](https://discourse.getdbt.com/t/tips-and-tricks-about-working-with-dbt/287/8 "2020-07-02T17:53:53Z")

</div>

Here’s a basic run-and-test function I’ve got in my `~/.bash_profile`:

```auto
dbtrt() {
    echo "Running and testing: $@"
    dbt run -m "$@" && dbt test -m "$@"
}

```

---

<div class="post-metadata">

**Author:** ![thoren](https://sea2.discourse-cdn.com/flex020/user_avatar/discourse.getdbt.com/thoren/32/444_2.png) [@thoren](https://discourse.getdbt.com/u/thoren)\
**Post date:** [July 2, 2020, 6:02pm UTC](https://discourse.getdbt.com/t/tips-and-tricks-about-working-with-dbt/287/9 "2020-07-02T18:02:10Z")

</div>

Here’s another bash function:

```auto
# find and open in emacs
find_and_open() {
    find . -type f -iname "*$1*" | xargs emacsclient -nw -a ''
}

```

This can be configured for other editors too. E.g. For Sublime just change the ending to `... sublime -w`

---

<div class="post-metadata">

**Author:** ![anton.sol](https://avatars.discourse-cdn.com/v4/letter/a/bbce88/32.png) [@anton.sol](https://discourse.getdbt.com/u/anton.sol)\
**Post date:** [May 8, 2023, 12:37pm UTC](https://discourse.getdbt.com/t/tips-and-tricks-about-working-with-dbt/287/10 "2023-05-08T12:37:00Z")

</div>

> [@drew](#):
>
> The one trick I really like involves using some bash magic to run all of the changed files on a branch.

That’s neat! I noticed that this does not run newly added models, so I modified the code to include all modified **and newly created** (but not yet commited) models.

```auto
function dbt_run_changed() {
    children=$1
    models=$(git status --short | grep -E '^( M|\?\?).*\.sql$' | awk '{print $2}' | xargs -n 1 basename | sed "s/\.sql$/${children}/g"| tr '\n' ' ')
    echo "Running models: ${models}"
    dbt run --models $models
}

```

> This function takes an optional argument, `+`, that will also run the children of the changed models!