The problem I’m having is to dynamically pass the test names to dbt.ref and dbt .source()
"
The context of why I’m trying to do this
im trying to build a data frame with test name and test results wrapped in json format .dbt tests creates transient table for each test and i want to wrap the test name and test result for each test into a single data frame .
What I’ve already tried
Some example code or error messages
import pandas as pd
import logging
import os
def model(dbt, session):
dbt.config(packages=[
"pandas",
"numpy",
],
use_anonmyous_sproc=True)
result = dbt.ref('table1').to_pandas()
test_names=list(result['TEST_NAME'])
wrapped_dict={}
for test_name in test_names:
try:
test_result=dbt.source('schema',str(test_name)).to_pandas()
json_result=test_result.to_json(orient="records")
wrapped_dict[test_name]=json_result
except snowflake.connector.errors.ProgrammingError:
print(f"Table {test_name} not found in schema. Skipping this table.")
continue
df=pd.DataFrame(wrapped_dict)
this is the error Parsing Error
Error when trying to literal_eval an arg to dbt.ref(), dbt.source(), dbt.config() or dbt.config.get()
malformed node or string: <_ast.Name object at 0x7f561c4be7c0>
ast — Abstract Syntax Trees — Python 3.12.2 documentation
In dbt python model, dbt.ref
, dbt.source
, dbt.config
, dbt.config.get
function args only support Python literal structures