AWS Redshift
Follow these steps to connect your AWS Redshift instance to Select Star.
Last updated
Follow these steps to connect your AWS Redshift instance to Select Star.
Last updated
To connect AWS Redshift to Select Star, you will need...
access to CloudFormation with permissions to modify IAM, AWS Lambda, Redshift cluster and VPC
access to AWS Redshift admin
Select Star requires only minimal metadata access to AWS Redshift. The granted permissions are defined in CloudFormation template:
IAM permission defined by resource "CrossAccountRolePolicy" in file SelectStarRedshift.json
AWS Redshift user permission defined in file provision.py
For instances operating within a non-publicly accessible environment, such as an AWS VPC, please refer to our guide on Integrating Private Network Data Sources for detailed instructions and best practices.
1. Go to the Select Star Settings. Click Data in the sidebar, then + Add to create a new Data Source.
2. Fill form in the required information:
Source Type: Select "Redshift"
Display Name: This value is Redshift by default, but you can override it if desired.
Cluster Name: The name of your AWS Redshift cluster in the AWS management console. Also known as "Cluster identifier" by AWS.
Database: The name of the database in AWS Redshift you've given us access to.
AWS Region: ID of the AWS region where the cluster was created. For example us-east-2,us-west-1, eu-central-1
3. Click Connect.
Select Star recommends use of AWS CloudFormation to setup integration, which allows you to make necessary changes to the Redshift cluster environment in a automatic, transparent, safe and auditable manner.
AWS CloudFormation template create AWS resources, modify and validate the Redshift configuration for safe integration with the Select Star services:
validate Redshift cluster compatibility
create an AWS IAM Role to enable access for Select Star and add it to Redshift cluster
create S3 bucket for query logs
activates log export to S3 bucket of all Redshift logs, if log export is not enabled so far
create a custom parameter group or modify an existing one to set a parameter enable_user_activity_logging
create Redshift user selectstar and grants permissions in Redshift cluster
configure the security group to allow Select Star access to the Redshift cluster
By default, AWS CloudFormation template activates log export to AWS S3. To use AWS CW, activate exporting to AWS CW beforehead.
The source code of the CloudFormation template along with build scripts and real-time logs of the continuous deployment system is available on public repository on GitHub "selectstar/cloudformation-templates" to be freely audited.
You can pass the link to CloudFormation to the infrastructure team to enable the integration to be created.
1. A simple form will be displayed in Select Star.
2. Select the "Open CloudFormation" button. A new window will open to proceed to the creation of a CloudFormation stack by AWS Management Console. Make sure you are logged into the AWS account in which the Redshift cluster exist.
3. The Create Stack form will be displayed. Fill form in the required information:
Under Parameters, enter the Amazon Redshift cluster name, list of comma separated database names, and your database user name. The indicated user will be used only by CloudFormation to create a dedicated user "select_star". Select Star will not have any access to the indicated user.
Select "true" in the "Configure S3 logging" and "Restart Cluster (if necessary to apply changes)" fields for fully automatic cluster configuration.
Select Star will only have access to the databases that exist at the time of provisioning. If you are planning to add new databases at a later stage, we suggest you do that before provisioning all the permissions.
The user you provide (DbUser) needs to have admin access. This is the user through which the CloudFormation template will create the user that Select Star needs with minimal access.
4. Review the information and under Capabilities choose "I acknowledge that AWS CloudFormation might create IAM resources".
5. Choose Create stack.
6. Wait until the stack changes it status to "CREATE_COMPLETE" from "CREATE_IN_PROGRESS" in tab "Stack info". The operation should take up to 5 minutes. You need to refresh tab to see the progress.
7. After completing stack creation, the Role ARN is available from the "Outputs". Copy and save the RoleArn for later use.
1. Return to Select Star. You should see a form that allows you to provide "Role ARN". Fill form in the required information:
Role ARN: Identifier of AWS IAM Role to use by Select Star. You'll see this after completing step 2.7 of the instructions.
2. Click Connect.
After you fill in the information, you'll be asked to select the databases you'd like to load into Select Star.
Select Star will not read queries or metadata or generate lineage for databases, schemas, or tables that are not loaded. Please load all data for which you expect to see lineage.
You can change the databases and schemas you have loaded if needed.
Select the database and click Next.
For each database you selected, you'll be able to select the schemas.
Your metadata should start loading automatically. Please allow 24-48 hours to completely generate popularity and lineage.
When the sync is complete, you'll be able to explore Redshift in Select Star.
