The problem I’m having
After i create snowflake stage and create pipe and table using external-tables package like this
- name:
external:
location: “@STAGE”
file_format: “(type = csv record_delimiter = ‘\n’ field_delimiter = ‘,’ skip_header = 1)”
pattern: “.*[.]csv”
snowpipe:
copy_options: “on_error = skip_file”
columns:
- name: COL1
data_type: varchar
- name: COL2
data_type: varchar
- name: COL3
data_type: varchar
- name: COL4
data_type: varchar
I want to use copy option, MATCH_BY_COLUMN_NAME with PARSE_HEADER = TRUE, its not allowing me to do that because external-tables package create pipes using select statement instead of copy into <> without select statement.
Match by column name is not allowed using copy into <> select from @stage.
The context of why I’m trying to do this
Users often submit data in wrong col orders.
Any suggestions would be appreciated!