dbt Core (open source)

Follow these steps to connect your dbt Core project to Select Star.

1. Create dbt build artifacts

For dbt Core, we'll need

• manifest.json file

• catalog.json file

• run_results.json for dbt Tests ingestion. run_results.json is optional.

To generate these, please use these commands:

for manifest.json, run the following command and copy the manifest.json in your target directory to another location.

dbt compile --full-refresh

for catalog.json, run the following command

dbt docs generate

for dbt Tests, run the following command:

dbt test

after generating these, you should see them in your target directory.

dbt docs generate also generates a manifest.json file, but this doesn't contain enough information about lineage, do not provide this manifest.json to Select Star if you are using Create a separate data source.

This is why we recommend copying the manifest.json to another location

2. Add dbt Core to Select Star

Select dbt Core (open source) in the dbt Type drop down

Provide the manifest.json and catalog.json and run_results.json files into the file drop zones. Please note that run_results.json is optional and only required if you want to see dbt tests in Select Star.

Data Source Options

• Display Name: This value is dbt by default, but you can override it if desired.

• Data Source: Select one of your existing warehouse data sources to connect to (i.e. Snowflake, BigQuery, Redshift).

  • Note: if you don't have any other data source, you'll need to select the SQL dialect according to which data warehouse you use (i.e. Snowflake, BigQuery, Redshift). This will be labelled as Dialect instead of Data Source.

• manifest.json: Drag and drop a manifest.json file from your dbt project's target directory.

• catalog.json: Drag and drop a catalog.json file from your dbt project's target directory.

• run_results.json: Drag and drop a run_results.json file from your dbt project's target directory.

Click the Connect button at the bottom of the Add Data Source modal.

Your metadata should start loading automatically. When the sync is complete, we'll email you and you'll be able to explore dbt in Select Star!

Important: If you are using dbt Core, in order to see changes to dbt reflected in Select Star, you will need to update the build artifacts after making changes.

You can update the build artifacts automatically (see step 3) or manually (see step 4).

3. Update build artifacts automatically (recommended)

if you are using dbt Cloud then you don't need to do this, we'll automatically pull updates and the job created in #dbt-cloud-custom-jobwill keep the metadata up to date!

In order to keep your dbt docs to be synched to Select Star, you can automatically update the manifest.json and catalog.json files using our dedicated API endpoint.

Learn how to integrate with the Select Star API by clicking the link below.

💻Select Star API

4. Update build artifacts manually

Navigate to the data source admin page, and next to the dbt data source click on the Configure button.

Here you can select the dbt Core (open source) dbt Type then update the manifest.json and catalog.json files in by dragging and dropping them from your dbt project's target directory.

See the steps in Data Source Options to see how to create your manifest.json, catalog.json and run_results.json files.

Once you've updated your manifest.json and/or catalog.json and run_results.json files, click Connect to confirm your changes, and you're good to go!