# Execute DBT pipeline via plain SQL

**URL:** https://discourse.getdbt.com/t/execute-dbt-pipeline-via-plain-sql/1431
**Category:** Archive
**Created:** [July 18, 2020, 1:59pm UTC](https://discourse.getdbt.com/t/execute-dbt-pipeline-via-plain-sql/1431 "2020-07-18T13:59:54Z")
**Posts on this page:** 1
**Page:** 1

<div class="post-metadata">

### Author: ![evolex](https://sea2.discourse-cdn.com/flex020/user_avatar/discourse.getdbt.com/evolex/32/455_2.png) [@evolex](https://discourse.getdbt.com/u/evolex)
#### Post date: [July 18, 2020, 1:59pm UTC](https://discourse.getdbt.com/t/execute-dbt-pipeline-via-plain-sql/1431/1 "2020-07-18T13:59:54Z")

</div>

Hello everybody.

For the moment, for the production pipeline execution, we can use only plain SQL (T-SQL).

So we need to get SQL Script, which does the same job as DBT RUN. A solution is to collect SQL statements DBT RUN issues. The additional requirement is that we want to have the Script packed into UDF (T-SQL Stored Procedure).

The current solution to get the DBT pipeline SQL Script in UDF (T-SQL Stored Procedure):

1. Launch SQL Server Profiler with SQL Batch Starting Event. [https://docs.microsoft.com/en-us/sql/relational-databases/event-classes/sql-batchstarting-event-class](https://docs.microsoft.com/en-us/sql/relational-databases/event-classes/sql-batchstarting-event-class)
2. Execute DBT RUN command (1 thread, SQL Server development environment with administrative rights).
3. Export SQL Statements from the Profiler to a text file.
4. Add dynamic invocation\_id generation statement to the beginning of the Script (for [GitHub - dbt-labs/dbt-event-logging: a dbt package to make auditing dbt runs easy.](https://github.com/fishtown-analytics/dbt-event-logging)):  
`declare @invocation_id as uniqueidentifier = NEWID()`
5. Replace some statements in the Script:  
5.1. “GO” with semicolon.  
5.2. “COMMIT” with semicolon.  
5.3. User name generated by [GitHub - dbt-labs/dbt-event-logging: a dbt package to make auditing dbt runs easy.](https://github.com/fishtown-analytics/dbt-event-logging) with `SUSER_SNAME()`.  
5.4 Fixed “invocation\_id” with `@invocation_id` from (4).
6. Remove UDFs creation statements. [Using dbt to manage user defined functions](https://discourse.getdbt.com/t/using-dbt-to-manage-user-defined-functions/18)
7. Create UDF (T-SQL Stored Procedure) with the statements from the Script.

There is a number of downsides in the approach, like the loss of the multithreaded execution [run | dbt Docs](https://docs.getdbt.com/reference/commands/run/), the need for separate exceptions handling, etc.

Nevertheless, the solution still works pretty well, and to be more confident before going further in detailed tuning/automation, we are thinking about if there are more recommended approaches.

For example, it seems feasible to work with JSON log

> **[Global CLI flags | dbt Docs](https://docs.getdbt.com/reference/global-cli-flags)**
>
> dbt commands, such as run or test, support their own command-specific CLI flags. In addition, the dbt command itself supports "global" flags applicable to all subcommands.

`–log-format JSON`  
to get the same DBT pipeline SQL Script via python.

Thanks in advance for the recommendations/thoughts on how it may be better to execute DBT pipeline via plain SQL.

Cheers  
Sergey