The problem I’m having
Trying to export doc blocks to a materialized table in BigQuery
The context of why I’m trying to do this
I already have queries extracting column names from tables, and a macro exporting column names from yaml-files. My plan was to bring doc blocks too, so I could
- Identify missing/orphaned column names from the three sources
- Automatically generate documentation of column names in my project by query, using these three sources in addition to further information in specialized tables
What I’ve already tried
I’ve discussed the matter with Copilot (Claude Opus 4.6) who provided the below macro, which resulted in error:
Compilation Error
‘dict object’ has no attribute ‘docs’
Opus then concluded with this:
The graph object in dbt's run-operation context doesn't expose docs directly. The available attributes depend on your dbt version. Let me try a different approach — docs are stored in the manifest, but that's not easily accessible during run-operation.
The most reliable alternative is to parse the docs from the YAML columns we already have.
But this defies the main purpose, to have separate sources and identify discrepancies.
Therefore I’m asking you smart people, whether there is some way for me to export doc blocks in an “integrated manner” or if I have to rely on external ways to parse this.
If there is no integrated manner, this is a feature request to dbt, because I assume more people than me might have interest in being able to query the structures we’re making in dbt
Some example code or error messages
{% macro get_doc_blocks() %}
{#
Extracts all doc block definitions from the dbt project
and stores them in a materialized table for documentation auditing.
Usage: dbt run-operation get_doc_blocks
Target: materialized_tables.z_documentation__doc_blocks
Related files:
- docs/field_descriptions.md
- docs/object_descriptions.md
- macros/get_yaml_columns.sql
#}
{% set query %}
CREATE OR REPLACE TABLE `materialized_tables.z_documentation__doc_blocks` AS
{% set union_parts = [] %}
{% for doc_name, doc_node in graph.docs.items() %}
{% set clean_description = doc_node.block_contents | replace("'", "\\'") | replace("\n", " ") | trim %}
{% do union_parts.append(
"SELECT '" ~ doc_node.name ~ "' AS doc_name, '"
~ doc_node.package_name ~ "' AS package_name, '"
~ doc_node.path ~ "' AS file_path, '"
~ clean_description[:500] ~ "' AS description"
) %}
{% endfor %}
{{ union_parts | join('\nUNION ALL\n') }}
{% endset %}
{% do run_query(query) %}
{{ log("Done: z_documentation__doc_blocks created with " ~ union_parts | length ~ " rows", info=True) }}
{% endmacro %}