If you’re using a dbt package, and make improvements to it, we’d love for you to contribute that back to the dbt community.
This document runs through the workflow I would use to do this. I’m going to use a specific example here, but this workflow applies to all packages.
Let’s say I’m trying to use the
segment package in a project named
jaffle_shop, but have found that the package needs some work to make it compatible with my warehouse. I decide that I’m going to make the package work on my warehouse, and want to contribute that back to the wider dbt community (for the purpose of this example, we’re imagining that I’m an external community member without edit access on the Fishtown GitHub organization).
0. Install the existing package in my dbt project
This is sort of a baked in assumption – since I want to use the Segment package, I’ve already installed it in my project, tried to use it, and then gotten some sort of error back, which is why I have decided to make changes to the package.
At the moment, the
packages.yml file (docs) looks like this:
# packages.yml packages: - package: fishtown-analytics/segment version: 0.2.3
1. Fork the dbt package I want to work on
To work on a dbt package, I need to have edit access to the repository. As an external contributor, this means that I need to fork Fishtown Analytics’ repository for the Segment package to my own GitHub account.
2. Clone the repository
I clone the repository into a directory, for example:
3. Install the local version of the package in my project
I update the
packages.yml file to point to the clone of the segment repository.
packages: - local: /Users/claire/clrcrl/packages/segment
Use whichever path you cloned your repo into! I also like to check out a new branch in my project, like
scratch/local-segment-package to make sure that I don’t commit the code.
Then I run
dbt clean (to remove the published package) and
dbt deps (to install the local package) from the command line.
4. Make changes to the package
I make any changes to the repo, following a normal git flow (open a new branch, committing, etc.)
To test my changes, I run my dbt project and see if the changes work! Often I have two terminal tabs open – one open for my project, and one for the package.
I also have both the package and my project open in my code editor (Atom):
Note that as I develop, I don’t need to re-run
dbt depsevery time I make updates to the Segment package – when I use a
localpackage, dbt will always use the current files on my computer.
5. Test the changes
Some packages (e.g. dbt-utils and snowplow) have integration tests set up. If this is the case, I add tests for my changes to ensure they work.
( This is sort of a vague statement because this really depends on the package you’re working on)
For most packages, testing of models really just involves ensuring the code runs, which I do by running my project (as described above).
5. Open up a PR
Once I’ve made my changes, I push the branch to GitHub, and open up a PR against the main repository! From here, one of the team members from Fishtown will review the PR, give feedback (if required), and merge it!