Redshift

Follow these steps to connect your AWS Redshift instance to Select Star.

Before you start

To connect AWS Redshift to Select Star, you will need...

• access to CloudFormation with permissions to modify IAM, AWS Lambda, Redshift cluster and VPC

• access to AWS Redshift admin

Select Star requires only minimal metadata access to AWS Redshift. The granted permissions are defined in CloudFormation template:

• IAM permission defined by resource "CrossAccountRolePolicy" in file SelectStarRedshift.json

• AWS Redshift user permission defined in file provision.py

1. Create a new data source

2. Create CloudFormation stack

3. Confirm authorization

4. Choose databases and schemas

1. Create a new data source

1. Go to the Select Star Settings. Click Data in the sidebar, then + Add to create a new Data Source.

Animations shows where to click to create a new data source.

2. Fill form in the required information:

• Source Type: Select "Redshift"

• Display Name: This value is Redshift by default, but you can override it if desired.

• Cluster Name: The name of your AWS Redshift cluster in the AWS management console. Also known as "Cluster identifier" by AWS.

• Database: The name of the database in AWS Redshift you've given us access to.

• AWS Region: ID of the AWS region where the cluster was created. For example us-east-2,us-west-1, eu-central-1

Screenshot shows a new data source form.

3. Click Connect.

2. Create CloudFormation stack

Select Star recommends use of AWS CloudFormation to setup integration, which allows you to make necessary changes to the Redshift cluster environment in a automatic, transparent, safe and auditable manner.

AWS CloudFormation create AWS resources, modify and validate the Redshift configuration for safe integration with the Select Star services:

• validate Redshift cluster compatibility

• create an AWS IAM Role to enable access for Select Star and add it to Redshift cluster

• create S3 bucket for query logs

• setup logging to S3 bucket of all Redshift queries

• create a custom parameter group or modify an existing one to set a parameter enable_user_activity_logging

• create Redshift user selectstar and grants permissions in Redshift cluster

• configure the security group to allow Select Star access to the Redshift cluster

The source code of the CloudFormation template along with build scripts and real-time logs of the continuous deployment system is available on public repository on GitHub "selectstar/cloudformation-templates" to be freely audited.

You can pass the link to CloudFormation to the infrastructure team to enable the integration to be created.

1. A simple form will be displayed in Select Star.

Screenshot shows the access authorization form.

2. Select the "Open CloudFormation" button. A new window will open to proceed to the creation of a CloudFormation stack by AWS Management Console. Make sure you are logged into the AWS account in which the Redshift cluster exist.

3. The Create Stack form will be displayed. Fill form in the required information:

• Under Parameters, enter the Amazon Redshift cluster name, list of comma separated database names, and your database user name. The indicated user will be used only by CloudFormation to create a dedicated user "select_star". Select Star will not have any access to the indicated user.

• Select "true" in the "Configure S3 logging" and "Restart Cluster (if necessary to apply changes)" fields for fully automatic cluster configuration.

Select Star will only have access to the databases that exist at the time of provisioning. If you are planning to add new databases at a later stage, we suggest you do that before provisioning all the permissions.

The user you provide (DbUser) needs to have admin access. This is the user through which the CloudFormation template will create the user that Select Star needs with minimal access.

Redshift configuration

4. Review the information and under Capabilities choose "I acknowledge that AWS CloudFormation might create IAM resources".

Screenshot shows the "Capabilities" section in "Quick create stack" form.

5. Choose Create stack.

6. Wait until the stack changes it status to "CREATE_COMPLETE" from "CREATE_IN_PROGRESS" in tab "Stack info". The operation should take up to 5 minutes. You need to refresh tab to see the progress.

Screenshot shows the "State" value in "Stack info" tab.

7. After completing stack creation, the Role ARN is available from the "Outputs". Copy and save the RoleArn for later use.

Screenshot shows where to obtain an Role ARN in the Outputs tab

3. Confirm authorization

1. Return to Select Star. You should see a form that allows you to provide "Role ARN". Fill form in the required information:

• Role ARN: Identifier of AWS IAM Role to use by Select Star. You'll see this after completing step 2.7 of the instructions.

Screenshot shows authroization in Select Star

2. Click Connect.

4. Choose databases and schemas

After you fill in the information, you'll be asked to select the databases you'd like to load into Select Star.

Select Star will not read queries or metadata or generate lineage for databases, schemas, or tables that are not loaded. Please load all data for which you expect to see lineage.

You can change the databases and schemas you have loaded if needed.

Select the database and click Next.

Database selection

For each database you selected, you'll be able to select the schemas.

Schema selection

Your metadata should start loading automatically. Please allow 24-48 hours to completely generate popularity and lineage.

When the sync is complete, you'll be able to explore Redshift in Select Star.