Apply row Limits in Model runs in CI run

In-Depth Discussions
,
1

Hello I’ve implemented Slim CI with dbt-core. I have been running modified models whne PR is created or updated with following command.

dbt run --defer --models state:modified+1 --state state_file_path

I want to limit no of records in model runs to speed up CI. I have done it through custom source and ref macros. For example : [Feature] "Slim CI" but with `LIMIT 0` · Issue #4201 · dbt-labs/dbt-core · GitHub

Some of my models use dbt_utils methods where it expects Relation object.
Question : How would I apply limit in this case? Currently I pass relation parameter in my custom macro to return Relation object but I can’t limit the records.

Thanks!

2

can you send us the dbt_utils methods you are using in your model files and are causing issues

3

Hello @Surya,

For example : dbt_utils.deduplicate dbt-utils/deduplicate.sql at main · dbt-labs/dbt-utils · GitHub

dbt_utils.deduplicate(
            relation=source('schema', 'table'),
            group_by="col1",
            order_by="col2 desc",
        )