About Us Customers Resources

MySQL (beta)

Follow these steps to connect your MySQL instance to Select Star.

  1. Create a Service Account

  2. Create a new datasource in Select Star

  3. Enable query logging (optional)

  4. Choose databases and schemas

For instances operating within a non-publicly accessible environment, such as a private network, please refer to our guide on Integrating Private Network Data Sources for detailed instructions and best practices.

1. Create a Service Account

We recommend creating a dedicated service account for Select Star with metadata-only roles and permissions.

  1. Execute the following command:

    CREATE USER '<username>' IDENTIFIED BY '<password>';
GRANT SELECT ON *.* TO '<username>'
GRANT SHOW VIEW ON *.* TO '<username>'

-- Apply the changes
FLUSH PRIVILEGES;

2. Create a new data source in Select Star

  • Display Name - The Name you want to give to your new datasource.

  • Hostname - Your hostname defines the location where your database is hosted.

  • Port - The communication endpoint used to connect clients to the MySQL instance.

  • Username - Specify the User to connect to. It should have enough privileges to read all the metadata.

  • Password - Password to connect the MySQL instance.

Select Star currently does not support Popularity and Lineage for this integration. If needed, please feel free to contact us so we can explore alternative solutions.

3. Enable query logging (optional)

For instances hosted on AWS RDS we have the ability to ingest query log which allows us to get detailed lineage and popularity. Here's a step-by-step guide to enable the General Query Log for a MySQL database hosted on AWS RDS and export the logs to AWS CloudWatch.

If you are using different hosting provider and you want ingest query log please contact our sales team.

Step 1: Sign in to the AWS Management Console

  1. Go to the AWS Management Console.

  2. Sign in using your AWS account credentials.

Step 2: Navigate to the RDS Dashboard

  1. In the AWS Management Console, search for "RDS" in the search bar at the top and select RDS from the search results.

  2. This will take you to the RDS dashboard.

Step 3: Select Your RDS Instance

  1. In the RDS dashboard, click on Databases in the left-hand menu.

  2. Find and select the MySQL RDS instance for which you want to enable the General Query Log.

Step 4: Modify the DB Parameter Group

  1. On your selected DB instance page, scroll down to find the Configuration section.

  2. Under Parameter group, note the name of the currently associated parameter group. If it's a default parameter group, you need to create a new custom parameter group because default ones cannot be modified.

  3. In the left-hand menu, click on Parameter groups.

  4. Click Create parameter group if you need to create a new one:

    • Choose the Parameter group family corresponding to your MySQL version.

    • Enter a name for the new parameter group.

    • Click Create.

  5. Once created, find your new parameter group in the list, and click on it to modify the parameters.

Step 5: Enable General Query Log in the Parameter Group

  1. In the parameter group settings, use the search bar to find the general_log parameter.

  2. Set general_log to 1 (enabled).

  3. Find the log_output parameter and set it to FILE as required for logging to CloudWatch.

  4. Save the changes.

Step 6: Apply the Parameter Group to Your RDS Instance

  1. Go back to the Databases section.

  2. Select your MySQL RDS instance again.

  3. Click on the Modify button in the upper right corner.

  4. In the Database options section, change the DB Parameter Group to the new custom parameter group you just modified.

  5. Scroll down and choose whether to apply the changes immediately or during the next maintenance window.

  6. Click Continue and then Modify DB Instance.

Step 7: Enable Logging to CloudWatch

  1. With the RDS instance selected, go to the Logs & events tab.

  2. Under Manage export logs to CloudWatch, ensure that General Log is checked.

  3. Click on Configure.

Step 8: Confirm Logs in CloudWatch

  1. Go to the AWS Management Console and search for CloudWatch.

  2. In the CloudWatch dashboard, click on Logs in the left-hand menu.

  3. You should see a log group corresponding to your RDS instance. The General Query Log entries should now appear here.

Step 9: Test and Verify

  1. Run a few queries on your MySQL database.

  2. Check CloudWatch Logs to ensure that the queries are being logged and that the General Query Log is working correctly.

Step 10. Create CloudFormation stack

Select Star recommends setting up integration using AWS CloudFormation, which allows you to make necessary changes to the RDS instance environment automatically, transparently, safely, and auditably.

AWS CloudFormation will create AWS resources to enable us to access AWS CloudWatch logs.

The source code of the CloudFormation template, build scripts, and real-time logs of the continuous deployment system are available on our public repository selectstar/cloudformation-templates, hosted on Github, to be freely audited.

You can pass the link to CloudFormation to the infrastructure team to enable the integration to be created.

1. A simple form will be displayed in Select Star. In option Source Type select "CloudWatch Logs".

2. Click Open CloudFormation. A new window will open to proceed to the creation of a CloudFormation stack by AWS Management Console. Make sure you are logged into the AWS account in which the RDS instance is hosted.

3. The Create Stack form will be displayed. Some of the values will be filled in by default, under Parameters, enter:

  • Log Group Name: The name of the log group to be consumed by Select Star. Example: /aws/rds/instance/dev-mysql8/general

4. Review the information and under Capabilities choose "I acknowledge that AWS CloudFormation might create IAM resources".

5. Choose Create stack.

  1. Wait until the stack changes its status to "CREATE_COMPLETE" from "CREATE_IN_PROGRESS " in the tab "Stack info." The operation should take up to 5 minutes. You need to refresh the tab to see the progress.

7. After completing stack creation, the Role ARN is available from the "Outputs". Copy and save the RoleArn for later use.

Step 11. Confirm authorization

1. Return to Select Star. You should see a form that allows you to provide "Role ARN" . Fill form in the required information:

  • AWS region: Identifier of AWS Region where your MySQL instance is hosted eg. us-east-2

  • Role ARN: Identifier of AWS IAM Role to use by Select Star.

  • Log group name: The name of the log group to be consumed by Select Star. Example: /aws/rds/instance/dev-mysql8/general

2. Click Connect.

4. Choose databases and schemas

After you fill in the information, you'll be asked to select the databases you'd like to load into Select Star.

Select Star will not read queries or metadata for databases, schemas, or tables that are not loaded. Please load all data for which you expect to see metadata.

You can change the databases and schemas you have loaded if needed.

Your metadata should start loading automatically, once the sync is complete, you'll be able to see it in Select Star.

See the link below for more information on how to navigate through data warehouses.

Getting Started: Data Warehouse
PreviousMicrosoft SQL Server (beta)NextOracle (beta)

Last updated