Last updated
Was this helpful?
Last updated
Was this helpful?
Dremio is a lakehouse platform that enables high-performance SQL queries directly on data lake storage. Paradime supports connections to both Dremio Cloud and Dremio Software. As a scheduler environment in Paradime, Dremio enables deploying dbt™ in production with .
IP RESTRICTIONS
Make sure to allow traffic from one of the Paradime IPs in your firewall depending on the data location selected.
👉 See also: .
Dremio offers different authentication methods for Dremio Cloud and Dremio Software deployments. Choose the appropriate method and ensure complete their respective prerequisites.
Access to a Dremio Cloud workspace
Access to the ID of the Sonar project you want to use. See .
A valid personal access token (PAT) for authenticating to Dremio Cloud. See .
Python 3.9 or later installed on the system that you are running dbt on.
Dremio Software version 22.0 or later
Python 3.9 or later installed
Required support keys enabled in your Dremio cluster (See docs for details)
dremio.iceberg.enabled
dremio.iceberg.ctas.enabled
dremio.execution.support_unlimited_splits
Click Settings in the top menu bar of the Paradime interface to access Account Settings
In the left sidebar, click "Connections"
Click "Add New" next to the Bolt Schedules section
Select "Dremio"
Provide a dbt™ Profile Name (This should match with the profile name set in your dbt_project.yml
).
The "Threads" value is pre-configured to 1, but you can adjust this number to specify how many parallel operations your dbt project can run.
In the Profile Configuration field, add the following parameters based on your authentication method. Reference the section below for additional context and configurations:
After adding the configuration, ensure it's properly formatted using a YAML formatter like .
In the Target field, enter "prod
". This is the default target your dbt™ project will use for your
object_storage_source
Source name for storing tables and other objects
$scratch
❌
object_storage_path
Path within storage for object creation
no_schema
❌
dremio_space
Space designation for view creation
@username
❌
dremio_space_folder
Folder structure within Dremio space
no_schema
❌
cloud_host
Dremio Cloud API endpoint
https://api.dremio.cloud
✅
cloud_project_id
Project identifier in Dremio Cloud
None
✅
user
Email address for authentication
None
✅
pat
Authentication token
None
✅
use_ssl
SSL connection enablement
true
✅
software_host
Host address of Dremio instance
None
✅
port
Connection port number
9047
✅
user
Authentication username
None
✅
password
User password (if not using PAT)
None
✅*
pat
Authentication token (if not using password)
None
✅*
use_ssl
SSL connection enablement
false
✅