The problem I’m having
I have been trying to define an external table to be used on dbt with the package dbt_external_tables. Also, based on the official docs, I got that we need to have an Stage table already created and working, it means integration and stage.
What I’ve already tried
Here is my config yaml file that defines connetion to the externl stage:
version: 2
sources:
- name: gcs_bucket
database: PRODUCT_AO_DEV
schema: EXTERNAL_STAGES
loader: gcloud storage
tables:
- name: PIP_ML_PREDICTIONS
ext_full_refresh: true
description: "Table with ML predictions"
external:
location: "@GCS_STAGE"
file_format: FILE_FORMAT = (TYPE=CSV COMPRESSION=AUTO NULL_IF=() field_optionally_enclosed_by='"' quote_identifiers=True)
columns:
- name: ANGEBOT_ID_NAN_1,
data_type: varchar(255)
After I perform dbt run-operation stage_external_sources
I have this error:
5:22:29 Running with dbt=1.8.8
15:22:29 Registered adapter: snowflake=1.8.4
15:22:30 Found 88 models, 13 operations, 441 data tests, 34 sources, 785 macros
15:22:30 1 of 1 START external source EXTERNAL_STAGES.PIP_ML_PREDICTIONS
15:22:30 1 of 1 (1) select 'Schema EXTERNAL_STAGES exists' from dual;
15:22:30 1 of 1 (1) SUCCESS 1
15:22:30 1 of 1 (2) create or replace external table PRODUCT_AO_DEV.EXTERNAL_STAGES.PIP_ML_PREDICTIO...
15:22:30 Encountered an error while running operation: Database Error
001003 (42000): SQL compilation error:
syntax error line 2 at position 41 unexpected '('.
What could be the reason for that? In what direction should I go further? But this is not the error associated with this yaml file. Everything parsed.