Redshift
Follow these steps to connect your AWS Redshift instance to Select Star.
Before you start
To connect AWS Redshift to Select Star, you will need...
- access to CloudFormation with permissions to modify IAM, AWS Lambda, Redshift cluster and VPC
- access to AWS Redshift admin
Select Star requires only minimal metadata access to AWS Redshift. The granted permissions are defined in CloudFormation template:
1. Create a new data source
1. Go to the Select Star Settings. Click Data in the sidebar, then + Add to create a new Data Source.
Animations shows where to click to create a new data source.
2. Fill form in the required information:
- Source Type: Select "Redshift"
- Display Name: This value is
Redshiftby default, but you can override it if desired. - Cluster Name: The name of your AWS Redshift cluster in the AWS management console. Also known as "Cluster identifier" by AWS.
- Database: The name of the database in AWS Redshift you've given us access to.
- AWS Region: ID of the AWS region where the cluster was created. For example
us-east-2,us-west-1,eu-central-1
Screenshot shows a new data source form.
3. Click Connect.
2. Create CloudFormation stack
Select Star recommends use of AWS CloudFormation to setup integration, which allows you to make neccessary changes to the Redshift cluster environment in a automatic, transparent, safe and auditable manner.
AWS CloudFormation create AWS resources, modify and validate the Redshift configuration for safe integration with the Select Star services:
- validate Redshift cluster compatibility
- create an AWS IAM Role to enable access for Select Star and add it to Redshift cluster
- create S3 bucket for query logs
- setup logging to S3 bucket of all Redshift queries
- create a custom parameter group or modify an existing one to set a parameter
enable_user_activity_logging - create Redshift user
selectstarand grants permissions in Redshift cluster - configure the security group to allow Select Star access to the Redshift cluster
The source code of the CloudFormation template along with build scripts and real-time logs of the continous deployment system is available on public repository on GitHub "selectstar/cloudformation-templates" to be freely audited.
You can pass the link to CloudFormation to the infrastructure team to enable the integration to be created.
1. A simple form will be displayed in Select Star.
Screenshot shows the access authorization form.
2. Select the "Open CloudFormation" button. A new window will open to proceed to the creation of a CloudFormation stack by AWS Management Console. Make sure you are logged into the AWS account in which the Redshift cluster exist.
3. The Create Stack form will be displayed. Fill form in the required information:
- Under Parameters, enter the Amazon Redshift cluster name, list of comma separated database names, and your database user name. The indicated user will be used only by CloudFormation to create a dedicated user "select_star". Select Star will not have any access to the indicated user.
- Select "true" in the "Configure S3 logging" and "Restart Cluster (if necessary to apply changes)" fields for fully automatic cluster configuration.
Screenshot shows the "Quick create stack" form.
4. Review the information and under Capabilities choose "I acknowledge that AWS CloudFormation might create IAM resources".
Screenshot shows the "Capabilities" section in "Quick create stack" form.
5. Choose Create stack.
6. Wait until the stack changes it status to "CREATE_COMPLETE" from "CREATE_IN_PROGRESS" in tab "Stack info". The operation should take up to 5 minutes. You need to refresh tab to see the progress.
Screenshot shows the "State" value in "Stack info" tab.
7. A completing stack creation, the
Role ARN is available from the "Outputs". Copy and save the RoleArn for later use.
Screenshot shows where to obtain an Role ARN in the Outputs tab
3. Confirm authorization
1. Return to Select Star. You should see a form that allows you to provide "Role ARN". Fill form in the required information:
- Role ARN: Identifier of AWS IAM Role to use by Select Star. You'll see this after completing step 2.7 of the instructions.
Screenshot shows authroization in Select Star
2. Click Connect.
4. Choose databases and schemas
After you fill in the information, you'll be asked to select the databases you'd like to load into Select Star.
Select Star will not read queries or metadata or generate lineage for databases, schemas, or tables that are not loaded. Please load all data for which you expect to see lineage.
Select the database and click Next.
For each database you selected, you'll be able to select the schemas.
Your metadata should start loading automatically. Please allow 24-48 hours to completely generate popularity and lineage.
When the sync is complete, you'll be able to explore Redshift in Select Star.
Last modified 5d ago