dbt Cloud

Follow these steps to connect your dbt Cloud project to Select Star.

1. Setup a dbt Cloud Custom Job

Create a new DBT Job:
Set it up with the following settings:
  • Select the environment
  • Select Generate Docs
  • add a command dbt compile --full-refresh
Schedule, select every day at exactly hour 4. This will run the job before we run our ingestion, so your data source is always up to date.
Finally, don't forget to save!
You should now see a page like this:
To get your job id, check the URL, you should see something like this:
the job id is the last number in the URL.
This job needs to run at least once successfully in order for Select Star to be able to process your dbt project.

a. Running your new job

Click the Green Run now button in the top right corner
Click on the new run generated in the list below:
You should see the following screen:
Wait until that grey Running status in the top left corner changes to a green Success status:
Once the run finishes successfully, you can go to Select Star to add your dbt data source.

2. Add dbt Cloud to Select Star

From your +Add Data Source, select dbt Cloud in the dbt Type drop down
Select dbt Cloud in the dbt Type drop down
Provide your service token and job id (see 1. Setup a dbt Cloud Custom Job for setting up your custom job and getting it's Job Id).

Data Source Options

  • Display Name: This value is dbt by default, but you can override it if desired.
  • Ingestion Settings: See our previous docs to determine which of the two checkboxes you'll need: 1. Determine what kind of dbt connection you need
  • Data Source: Select one of your existing warehouse data sources to connect to (i.e. Snowflake, BigQuery, Redshift).
    • Note: if you don't have any other data source, you'll need to select the SQL dialect according to which data warehouse you use (i.e. Snowflake, BigQuery, Redshift). This will be labelled as Dialect instead of Data Source.
  • API Key: You may either use a ...
    • dbt service token (preferred)
      • Permissions: we only need the Metadata Only permission for the projects that you are connecting
    • dbt user token
  • Job Id: The id for the custom job which will generate the dbt metadata needed for ingestion
Click the Connect button at the bottom of the Add Data Source modal.
Due to the Cloud API limitations, dbt project hierarchy will show as database/schema/table hierarchy. In order to see dbt project hierarchy, please add your manifest.json in Settings > Data > Configure.
Your metadata should start loading automatically. When the sync is complete, we'll email you and you'll be able to explore dbt in Select Star!