Hi,
I am trying to setup dbt with spark adapter to load the iceberg table. I have done the cluster level configuration and i am able to connect to EMR cluster but when i am trying to load table i am facing Multiple issue. One issue is highlighted below
My profile.yml :
data_ingestion_framework:
target: dev
outputs:
dev:
type: spark
method: thrift
schema: convera_raw_iceberg_cct_dev
host: XXX
cluster: j-24ZKTN9Y36F25
port: 10001
location: ‘s3a://trinity-dbt-poc/test’
Below is my module:
{{
config(
materialized=‘table’,
file_format=‘iceberg’,
location_root=‘s3://trinity-dbt-poc/test’
)
}}
(select * from convera_raw_iceberg_cct_dev.arinterface_dbo_ackmetadata);
Please provide input on how to access iceberg data from dbt-spark