Comment on page

dbt Core (open source)

Follow these steps to connect your dbt Core project to Select Star.

1. Create dbt build artifacts
For dbt Core, we'll need  
manifest.json file  
catalog.json file (if you want the Create a separate Data Source ingestion setting) 
run_results.json for dbt Tests ingestion. run_results.json is optional. 
To generate these, please use these commands:
for manifest.json, run the following command and copy the manifest.json in your target directory to another location.
dbt compile --full-refresh
for catalog.json, run the following command
dbt docs generate
for dbt Tests, run the following command: 
dbt test
after generating these, you should see them in your target directory.
dbt docs generate also generates a manifest.json file, but this doesn't contain enough information about lineage, do not provide this manifest.json to Select Star if you are using Create a separate data source.
This is why we recommend copying the manifest.json to another location
2. Add dbt Core to Select Star
Select dbt Core (open source) in the dbt Type drop down
without Create a Separate Data Source selected
with Create a Separate Data Source selected
Provide the manifest.json and catalog.json and run_results.json files from into the file drop zones depending on what you've selected from the options above. Please note that run_results.json is optional and only required if you want to see dbt tests in Select Star.
Data Source Options
Display Name: This value is dbt by default, but you can override it if desired.
Ingestion Settings: See our previous docs to determine which of the two checkboxes you'll need: 1. Determine what kind of dbt connection you need​
Data Source: Select one of your existing warehouse data sources to connect to (i.e. Snowflake, BigQuery, Redshift).
Note: if you don't have any other data source, you'll need to select the SQL dialect according to which data warehouse you use (i.e. Snowflake, BigQuery, Redshift). This will be labelled as Dialect instead of Data Source.
manifest.json: Drag and drop a manifest.json file from your dbt project's target directory.
catalog.json: Drag and drop a catalog.json file from your dbt project's target directory.
​💡
 Note: This option is only necessary for the Create New Data Source ingestion setting.
run_results.json: Drag and drop a run_results.json file from your dbt project's target directory.
Click the Connect button at the bottom of the Add Data Source modal.
Your metadata should start loading automatically. When the sync is complete, we'll email you and you'll be able to explore dbt in Select Star!
Important: If you are using dbt Core, in order to see changes to dbt reflected in Select Star, you will need to update the build artifacts after making changes.
You can update the build artifacts automatically (see step 3) or manually (see step 4).
A. Update build artifacts automatically (recommended)
if you are using dbt Cloud then you don't need to do this, we'll automatically pull updates and the job created in dbt-cloud-custom-jobwill keep the metadata up to date!
In order to keep your dbt docs to be synched to Select Star, you can automatically update the manifest.json and catalog.json files using our dedicated API endpoint.
Learn how to integrate with the Select Star API by clicking the link below.
💻
Select Star API
B. Update build artifacts manually
Navigate to the data source admin page, and next to the dbt data source click on the Configure button.
Here you can select the dbt Core (open source) dbt Type then update the manifest.json and catalog.json files in by dragging and dropping them from your dbt project's target directory.
See the steps in Data Source Options to see how to create your manifest.json, catalog.json  and run_results.json files.
Once you've updated your manifest.json and/or catalog.json and run_results.json files, click Connect to confirm your changes, and you're good to go!

dbt Cloud

dbt Tags (beta)

Last modified 3mo ago