# OpenLineage (beta)

## Before you start

To connect your Data Sources to Select Star, via OpenLineage, you will need to...

* prepare your OpenLineage job events according to the [specifications](https://openlineage.io/docs/spec/facets/)

{% hint style="info" %}
Select Star does not need any permissions for your underlying data sources or ETL tools, rather it relies on the events prepared and pushed by you.
{% endhint %}

Complete the following steps to connect OpenLineage to Select Star.

1. [Create a new Data Source in Select Star](#id-1.-create-a-new-data-source-in-select-star)
2. [Configure OpenLineage Producer](#id-2.-configure-openlineage-producer)
3. [Sync Metadata in Select Star](#id-3.-sync-metadata-in-select-star)

## 1. Create a new Data Source in Select Star

Go to the Select Star Settings. Click Data in the sidebar, then + Add to create a new Data Source.

<figure><img src="/files/UOcyzCP9jK7RLZfrVl09" alt=""><figcaption></figcaption></figure>

Fill in the form with the required information:

* **Source Type** - Choose `OpenLineage` from the dropdown.
* **Display Name** - This value is `OpenLineage` by default, but you can override it.
* **Base URL** - The URL of your ETL instance. For example, `http://airflow.example.com`.

Click **Save** to proceed.

On the next screen, you will see the **API Token**, **Events Endpoint**, and the **Events URL**. You will need these in the next steps to configure your OpenLineage producer environment.

<figure><img src="/files/bUFO0ydlMr3w0L4Bkmth" alt=""><figcaption></figcaption></figure>

* **API Token** - This is a secret key that Select Star will use to authenticate the traffic coming from your OpenLineage producer instance.
* **Events Endpoint** - This is the Select Star endpoint where your producer will send OpenLineage events, containing the metadata about your Jobs, Job Runs, and Datasets.
* **Events URL** - This is the Select Star Base URL where your producer will send OpenLineage events.

## 2. Configure OpenLineage Producer

You must use the values provided above in your producer to start sending your OpenLineage events to Select Star.

**That's it!** OpenLineage events will be sent to Select Star when your producer starts creating events.

{% hint style="info" %}
Select Star's OpenLineage integration can be used to generate lineage from your Spark, Airflow, and Custom OpenLineage events.
{% endhint %}

*For more details on configuring and producing OpenLineage events, please read the* [*official openlineage documentation*](https://openlineage.io/getting-started)*.*

## 3. Sync Metadata in Select Star

After you have configured your OpenLineage environment, make sure to trigger your health check jobs. This will send OpenLineage events to Select Star, and help you verify that the integration is working correctly.

Afterwards, you can go to the Select Star Settings and click on the Data in the sidebar. Click on the Sync metadata button on your OpenLineage Data Source.

{% hint style="warning" %}
Note that Select Star does not connect to your sources directly. That means the lineage and your job metadata will be available in Select Star only after you run your jobs and OpenLineage events are sent to Select Star.
{% endhint %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.selectstar.com/integrations/openlineage.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.