November 13, 2024 - New Navigation, Airflow and More!

What's New ✨

New Navigation Look and Feel

We're excited to introduce our new navigation interface - designed with a cleaner, more intuitive look that streamlines your experience. The refreshed layout offers effortless navigation and makes accessing data sources faster and easier than ever.

New navigation

This new update also makes room for exciting features down the line, so stay tuned 📻.

Airflow Integration

With the new integration with Apache Airflow, Airflow pipeline metadata is available in the Select Star catalog where all users have visibility. The new DAG metadata, including schedule, last run status, and run history in addition to DAG lineage gives transparency into Airflow pipelines and a more comprehensive picture of where data is coming from, and what downstream impact delays and breakages have.

Add your Airflow data source to get started!

Airflow Lineage

What's Improved ➕

Relevant Lineage

Lineage can appear to be simple on the surface, at the table and dashboard level, but when you dig into what’s happening at the column and query level, it can reveal a different story. Relevant lineage helps you get that nuanced story without needing to look at each column. Let’s take a look at an example:

Table Lineage

In this example, we have the USERS and the ACCOUNTS tables feeding into a USER_SUMMARY table, which is used in a chart showing User Signups. However, let’s peel back a layer and see what is happening at the column level.

Column Level Lineage

If we look at just the columns, we’ll notice that User Signups doesn’t actually use any data from ACCOUNTS. While ACCOUNTS is technically upstream of User Signups, it does not impact any of the data for it, and is therefore, not relevant to User Signups.

Relevant Lineage lets you filter out any lineage that does not impact the assets you are looking at, and only shows the relevant lineage. It is available in the lineage explorer, as well as lineage list views: upstream, downstream, downstream dashboards, and source tables.

Relevant lineage toggle on downstream dashboards

Less Frequent dbt PRs

If you’re using our dbt docs sync capabilities to sync documentation changes made in Select Star back to dbt, this is an update for you. Previously, Select Star PRs would be closed at the end of each day and re-opened for the next day, generating a lot of noise. With this improvement, PRs will remain open and will be updated if additional modifications are made in Select Star, reducing noise and helping you focus on the details that matter.

dbt Docs sync

Other Incremental Updates 🔧

• Lineage list view search improvements - When you search in the lineage list view, we will keep the path from the source data asset to any matches, so you have full context on how the data is getting to that asset.

  

  Lineage search
• New owner metadata notifications

  • We are rounding out owner notifications on metadata changes, including adding notifications for: custom attribute changes, owner updates, and more!

• Integration Improvements

  • Redshift stored procedure support for lineage - Lineage from stored procedures are now captured and available in Select Star, providing a more complete map of your data.

  • Tableau Ingestion performance has been greatly increased, reducing ingestion time for large Tableau instances.

  • Looker LookML integration to better identify Looker views when there is a github repository connected

  • Missing lineage edges in lineage explorer for cycle relationships (A→B→A)