Hello,
I am using dbt core on snowflake. version 1.6 or 1.7.
DB = database
SC = Schema
I import data from one database to another. Extremely simple, 1 model to import with just a select from a Src= DB1/SC_X to DB2/SC_Y, one other model using this model to create a view on it on DB2/SC_Z. Cannot be more simple.
everything work fine except if SC_Y = SC_Z. When I want to use 2 schema and not one it fails.
We use the override standard macro to not have the standard concatenation of schema .
At the moment I am using SC_Y in the profile.yml and some other model with SC_Z (in the project.yml or model itself) then I have this error using dbt run.
ERRROR
Database error while listing schemas in database “FR”
Database Error
cannot import name ‘SnowflakeOCSPAsn1Crypto’ from partially initialized module ‘snowflake.connector.ocsp_asn1crypto’ (most likely due to a circular import) (C:\miniconda3\envs\dbt2\lib\site-packages\snowflake\connector\ocsp_asn1crypto.py) \miniconda3\envs\dbt2\lib\site-packages\snowflake\connector\ocsp_asn1crypto.py)
TESTS PERFORMED
I did 100 tests including architects without finding a workaround. On the log side, the system tries two times to enumerate schemas on the Snowflake database and it looks like there is the issue.
LOG
e[0m16:09:32.897524 [debug] [ThreadPool]: Opening a new connection, currently in state init
e[0m16:09:32.912523 [debug] [ThreadPool]: Acquiring new snowflake connection ‘list_FR’
e[0m16:09:32.922521 [debug] [ThreadPool]: Using snowflake connection “list_FR”
e[0m16:09:32.926522 [debug] [ThreadPool]: On list_FR: /* {“app”: “dbt”, “dbt_version”: “1.7.2”, “profile_name”: “_local”, “target_name”: “Prod”, “connection_name”: “list_FR”} /
show terse schemas in database FR
limit 10000
e[0m16:09:32.927522 [debug] [ThreadPool]: Opening a new connection, currently in state init
e[0m16:09:34.363516 [debug] [ThreadPool]: Snowflake adapter: Error running SQL: / {“app”: “dbt”, “dbt_version”: “1.7.2”, “profile_name”: “_local”, “target_name”: “Prod”, “connection_name”: “list_FR”} */
show terse schemas in database FR
limit 10000
Conclusion
trying 1.7 instead of 1.6 does not resolve. when you manipulate several time profiles.yml and project.yml, the dbt run works one time (!). the second time it fails. to revert, I need to update 2 times profiles.yml in order for the system to do a full and not partial parsing then it works again with same schema, 2 databases.
Clearly looks a bug in one of the compoenents, but very hard to know where, what and why
any help will be greatly appecrated
Bruno