Hi, I am trying to move data between two SQLite Databases as my first project.
I have three tables in my first SQLite DB, which contains few records (1-2 records)
I am trying to copy these records into my target SQLite database.
I am able to run my models, which run successfully and they also create new identical tables in my second Database. But no data gets copied. I have tried so many ways but it still doesn’t work for me.
my profiles.yaml file,
local_sqlite_project:
target: dev
outputs:
dev:
type: sqlite
threads: 1
database: 'database'
schema: 'main'
schemas_and_paths:
main: 'C:/Programming/Projects/dbt/target.db'
schema_directory: 'file_path'
#optional fields
# extensions:
# - "/path/to/sqlean/crypto.so"
# Add a new connection for your source SQLite file
sources:
dev:
type: sqlite
database: 'C:/Programming/Projects/dbt/dbt-tutorial/tutorial_project/source/source.db'
schema: 'source'
My source.yaml file,
version: 2
sources:
- name: sample_table_database
database: 'source'
schema: 'main'
tables:
- name: first_table
- name: second_table
- name: third_table
My first_model.sql file (other two models are also identical with table name changed)
{{ config(materialized='table') }}
with source_data as (
select ID, Name from {{ source('sample_table_database', 'first_table') }}
),
final as (
select * from source_data
)
select * from final
Not sure if this is necessary but my dbt_project.yaml file is also shared here,
# Name your project! Project names should contain only lowercase characters
# and underscores. A good package name should reflect your organization's
# name or the intended use of these models
name: 'tutorial_project'
version: '1.0.0'
config-version: 2
# This setting configures which "profile" dbt uses for this project.
profile: 'local_sqlite_project'
# These configurations specify where dbt should look for different types of files.
# The `model-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]
target-path: "target" # directory which will store compiled SQL files
clean-targets: # directories to be removed by `dbt clean`
- "target"
- "dbt_packages"
# Configuring models
# Full documentation: https://docs.getdbt.com/docs/configuring-models
# In this example config, we tell dbt to build all models in the example/
# directory as views. These settings can be overridden in the individual model
# files using the `{{ config(...) }}` macro.
models:
tutorial_project:
# Config indicated by + and applies to all files under models/example/
example:
+materialized: view
If you would like to see the whole Repo, you may do so here: GitHub - hassanashas/dbt-tutorial
I have also uploaded the SQLite files, if anyone is interested in checking them out,
https://drive.google.com/drive/folders/1PKdpffsPxAGg1f7yxNG3gHmKl6SsOFwr?usp=sharing