Setting Up Data Warehouse Connections
Last updated
Last updated
In this guide, you'll learn how to set up and manage multiple data warehouse connections within your Paradime workspace. This process is crucial for configuring connections for both development in the Paradime Code IDE and running dbt™ in production with the Bolt Scheduler.
Estimated completion time: 10 minutes
An admin role In Paradime to create workspaces
In this guide, you'll learn how to:
The following video provides step-by-step instructions for adding a scheduler environment (bolt scheduler) and a Turbo CI connection.
This connection will be used as the default target for your Bolt schedules to run dbt™ in production.
Navigate to your account settings
Locate the Connections
tab.
Click Add New
button in the Scheduler section.
Select your data warehouse provider (e.g., Snowflake)
Configure the connection details.
Click 'Test Connection' to verify the setup.
💡 For visual guidance, follow video tutorial.
This connection will be used specifically for running dbt™ with TurboCI to build and test your dbt™ models when opening a Pull Request.
Follow steps 1-3 from the above section, Add a Bolt Scheduler Connection
Configure the connection details:
Provide a dbt™ profile name (This should match with the profile name set in your dbt_project.yml
).
Set the target as CI
.
Enter your data warehouse credentials (similar to the Scheduler prod connection).
Change the schema name to dbt_ci
(or your preferred CI schema).
Configure thread settings as needed.
Click Test Connection
to verify the setup.
💡 For visual guidance, follow video tutorial.
After setting up your connections, you'll have:
A prod
target connection for orchestrating production jobs.
A ci
target connection for running dbt™ with TurboCI.
These connections enable you to separate your development, testing, and scheduler environments effectively within Paradime.
After setting up multiple connections, it's important to understand how to direct your dbt™ commands to use a specific target. This allows you to switch between multiple environments easily.
By default, when you run a dbt™ command without specifying a target:
In the Code IDE: It will use the default connection set for the Code IDE.
In the Scheduler: It will use the default connection set for the Scheduler.
Example:
It will use the default target for your current environment.
To use a specific target, append the --target
argument to your dbt command:
For example, to run dbt using the 'CI' target we set up earlier:
This command will execute the dbt run using the connection details specified in the ci
target, regardless of the default setting.
You've now set up multiple data warehouse connections, learned how to manage them, and understand how to use different targets with dbt™ commands. These skills will allow you to efficiently work across development, testing, and production environments in Paradime.
Next, we'll dive into Managing Workspace Configurations to further customize your Paradime environment.