How can I get the source configuration via jinja?

[armsepehr]

I have several source.yml configuration files which indicates the key-value for each table. We have one source.yml file for each customer and the tables set for each customer would be different; then, the models would be different for each customers e.g. I have the following source.yml file for a customer:

- name: channels
  tables:
     - name: table1
       identifier: identifier1

I need a jinja function call to verify that the table1 is existed in the source.yml file or not. I tried and searched a lot: First, I tried source(“channels”,“table1”) but it gives me the error when the table is not existed e.g. source(“channels”,“table1”). Second, I verified the config() function call but it doesn’t have any information around the source.yml.

In a word, I need a function such as config.has_key('sources.channels.table3') to verify that table1 is existed and table2 is not existed. Would you please give me any hints to check the configurations is existed in the yml files or not?

[Surya]

can u send us the error ur getting?
i think ur source config syntax might be wrong
it should be like below

- name: channels
  tables:
    - name: table1
      identifier: identifier1

[armsepehr]

I did the same if you mean the dash behind the tables tag. I had a simple typo and now correct the question. In a simple word, I would like to find a function that give me true for table1 e.g. config.has_key('channels','table1') and give me false for table2 which is not existed in the configuration e.g. config.has_key('channels','table2').

[brunoszdl]

You can try a macro like this:

{% macro check_source(source_name, table_name) %}
    {% if execute %}

        {% for source in graph.sources.values() %}
    
            {% if source.source_name == source_name and source.name == table_name %}
    
                {{ return(true) }}
    
            {% endif %}

        {% endfor %}
    
        {{ return(false) }}
    
    {% endif %}
{% endmacro %}

Then call this macro like

{% if check_source('channels', 'table_1') %}
# your code here
{% endif %}

Just keep in mind this macro only works at run-time, not parse-time. So I am not sure it will do what you want.

[armsepehr]

Thanks @brunoszdl. It works with slight modifications.

{% macro check_source(source_name, table_name) %}

{%- if execute -%}
    {% for source in graph.sources.values() %}
        {%- if source.source_name == source_name and source.name == table_name -%}

            {{ return(true) }}

        {%- endif -%}
    {%- endfor -%}

    {{ return(false) }}

{%- endif -%}

{% endmacro %}