ASokol
April 17, 2024, 12:53am
1
Hi Community!
I am having come weird issue with dbt project:
dbt compile
Running with dbt=1.7.10
Registered adapter: postgres=1.7.10
Unable to do partial parsing because saved manifest not found. Starting full parse.
Encountered an error:
Compilation Error
dbt found two macros named “test_daily_count_decrease” in the project “xxx”.
To fix this error, rename or remove one of the following macros:
- tests/_default\generic\test_daily_count_decrease.sql
- tests/_default\generic\test_daily_count_decrease.sql
This is pointing to one file and there are no duplicated macroses.
Maybe you have some suggestions?
Additional info:
dbt --version
Core:
installed: 1.7.10
latest: 1.7.12 - Update available!
Your version of dbt-core is out of date!
You can find instructions for upgrading here:
About dbt Core and installation | dbt Developer Hub
Plugins:
postgres: 1.7.10 - Update available!
At least one plugin is out of date or incompatible with dbt-core.
You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation
Could you paste the file’s content?
ASokol
April 17, 2024, 7:40pm
3
sure
{% test daily_count_decrease(model, grouping_columns, date_column ) %}
{% set unique_date_count %}
SELECT count(distinct {{ date_column }}) AS date_count FROM {{ model }}
HAVING count(distinct {{ date_column }}) > 1
{% endset %}
{% set date_count = run_query(unique_date_count) %}
{% if date_count | length > 0 %}
WITH grouped_counts AS (
SELECT
{{ grouping_columns | join(', ') }},
{{ date_column }},
COUNT(*) AS daily_count
FROM {{ model }}
GROUP BY {{ grouping_columns | join(', ') }}, {{ date_column }}
)
,ordered_counts AS (
SELECT
*,
LAG(daily_count) over (PARTITION BY {{ grouping_columns | join(', ') }}
ORDER BY {{ date_column }}) AS previous_count
FROM grouped_counts
),
validation_logic AS (
SELECT
SUM(CASE WHEN daily_count < previous_count THEN 1 ELSE 0 END) as count_case
FROM ordered_counts
)
SELECT * FROM validation_logic
WHERE count_case > 0
{% else %}
SELECT FALSE AS always_pass_case
WHERE FALSE
{% endif %}
{% endtest %}
ASokol
April 17, 2024, 7:42pm
4
The thing is that if I drop this file i will get same error with other one…and so on…and so on
It seems the same issue reported here
opened 04:35PM - 21 Feb 24 UTC
closed 11:52PM - 19 Mar 24 UTC
bug
duplicate
partial_parsing
### Is this a new bug in dbt-core?
- [X] I believe this is a new bug in dbt-cor… e
- [X] I have searched the existing issues, and I could not find an existing issue for this bug
### Current Behavior
When a customer has “enable partial parsing between deployment runs” enabled in their account settings for their jobs, some jobs will error out with a message that points to a node that had changes made to it recently. However, the job should not be erroring out and the issue is rectified by disabling partial parsing and running the job again (which results in a successful run).
In this specific instance, two jobs in the customer's project errored out, stating that two macros named “dropDataset” were found. Zendesk ticket: https://dbtcloud.zendesk.com/agent/tickets/61977
Similar issues have been reported in another ticket where the removal of a test config from a model caused the job to begin erroring out. Disabling partial parsing resolved the issue: https://dbtcloud.zendesk.com/agent/tickets/60694
### Expected Behavior
Because there was only one macro by this name in their project, these jobs should have both succeeded without the user needing to manually disable partial parsing for jobs.
### Steps To Reproduce
Have not been able to reproduce yet.
### Relevant log output
_No response_
### Environment
```markdown
- OS: NA
- Python: NA
- dbt: 1.6
```
### Which database adapter are you using with dbt?
_No response_
### Additional Context
Example run: https://dbtlabsmt.datadoghq.com/logs?query=service%3Ascheduler-run%20%40extra.run_id%3A254481367%20&cols=host%2Cservice&fromUser=true&index=%2A&messageDisplay=inline&refresh_mode=paused&stream_sort=desc&viz=stream&from_ts=1708498959395&to_ts=1708531778000&live=false
Compilation error for above run: https://dbtlabsmt.datadoghq.com/logs?query=service%3Ascheduler-run%20%40extra.run_id%3A254481367%20&cols=host%2Cservice&event=AgAAAY3L9FnDppjrPwAAAAAAAAAYAAAAAEFZM0w5R2lBQUFBbUc3MGdPOGxGbndBQwAAACQAAAAAMDE4ZGNiZjUtZGI5OC00MzJkLThiOWUtZDU5MmY1MzY5M2U1&fromUser=true&index=%2A&messageDisplay=inline&refresh_mode=paused&stream_sort=desc&viz=stream&from_ts=1708498959395&to_ts=1708531778000&live=false
Other occurrences: https://dbtlabsmt.datadoghq.com/logs?query=service%3Ascheduler-run%20%22dbt%20found%20two%20macros%22%20&cols=host%2Cservice&fromUser=true&index=%2A&messageDisplay=inline&refresh_mode=sliding&stream_sort=desc&viz=stream&from_ts=1707236315915&to_ts=1708532315915&live=true
opened 03:55PM - 29 Mar 24 UTC
bug
partial_parsing
### Is this a new bug in dbt-core?
- [X] I believe this is a new bug in dbt-c… ore
- [X] I have searched the existing issues, and I could not find an existing issue for this bug
### Current Behavior
`dbt docs generate` gives the following error:
```
Compilation Error
dbt found two schema.yml entries for the same resource named model.name. Resources and their associated columns may only be described a single time. To fix this, remove one of the resource entries for model.name in this file:
- models/xxx/properties.yml
```
while it is only defined once
### Expected Behavior
generate the docs
### Steps To Reproduce
- create a model sql file
- create a properties.yml file and add descriptions and column descriptions
- generate the docs -> might work
- generate the docs -> probably gives the error, it seems to happen mostly when you generate the docs twice
- `dbt clean`
- generate the docs -> should work
### Relevant log output
```shell
Example without modifying any files:
$ dbt docs generate
15:45:23 Running with dbt=1.7.11
15:45:23 Registered adapter: snowflake=1.7.3
15:45:23 Encountered an error:
Compilation Error
dbt found two schema.yml entries for the same resource named model.name. Resources and their associated columns may only be described a single time. To fix this, remove one of the resource entries for model.name in this file:
- models/some_models/properties.yml
$ dbt clean
15:45:31 Running with dbt=1.7.11
15:45:31 Checking /home/paul/projects/dbt/dbt_packages/*
15:45:31 Cleaned /home/paul/projects/dbt/dbt_packages/*
15:45:31 Checking /home/paul/projects/dbt/target/*
15:45:31 Cleaned /home/paul/projects/dbt/target/*
15:45:31 Finished cleaning all paths.
$ dbt docs generate INT ✘ ▼ 16:45:34
15:45:37 Running with dbt=1.7.11
15:45:37 Registered adapter: snowflake=1.7.3
15:45:37 Unable to do partial parsing because saved manifest not found. Starting full parse.
15:45:38 Found 115 models, 32 seeds, 3 operations, 2 tests, 0 sources, 0 exposures, 0 metrics, 449 macros, 0 groups, 0 semantic models
15:45:38
15:45:45 Concurrency: 2 threads (target='dev')
15:45:45
15:45:49 Building catalog
15:45:54 Catalog written to /home/paul/projects/dbt/target/catalog.json
```
### Environment
```markdown
- OS: Arch Linux
- Python: 3.11.6
- dbt: 1.7.11
```
### Which database adapter are you using with dbt?
snowflake
### Additional Context
_No response_
opened 08:51AM - 05 Oct 23 UTC
closed 07:40PM - 05 Oct 23 UTC
bug
### Is this a new bug in dbt-core?
- [X] I believe this is a new bug in dbt-cor… e
- [X] I have searched the existing issues, and I could not find an existing issue for this bug
### Current Behavior
While `run`, `compile` or other command for partial parsing received `Compilation Error`:
```
dbt found two macros named "<user_macro>" in the project "<user_project>"
To fix this error, rename or remove one of the following macros:
- /macros/foo/bar.sql
- /macros/foo/bar.sql
```
### Expected Behavior
No error
### Steps To Reproduce
1. Create macros.sql file with 2 macros inside
2. Create macros.yml with documenation for this 2 macros
3. Run `dbt compile`
4. Change something in macros.yml
5. Run `dbt compile` again
### Relevant log output
```shell
08:46:39 Encountered an error:
Compilation Error
dbt found two macros named "<macros_name>" in the project "<project_name>".
To fix this error, rename or remove one of the following macros:
- macros/foo/bar.sql
- macros/foo/bar.sql
08:46:39 Command `dbt compile` failed at 11:46:39.886922 after 1.85 seconds
08:46:39 Flushing usage events
```
### Environment
```markdown
- Python: 3.10.4
- dbt: 1.6.2
```
### Which database adapter are you using with dbt?
other (mention it in "Additional Context")
### Additional Context
dbt-adapter: dbt-athena: 1.6.1
You can see if some of these discussions help, otherwise you can reach out to @doug.beatty in the slack community
1 Like