Dremio - Paradime

Dremio is a lakehouse platform that enables high-performance SQL queries directly on data lake storage. Paradime supports connections to both Dremio Cloud and Dremio Software. As a scheduler environment in Paradime, Dremio enables deploying dbt™ in production with Bolt.

IP RESTRICTIONSMake sure to allow traffic from one of the Paradime IPs in your firewall depending on the data location selected.👉 See also: Paradime IP addresses.

Prerequisites

Dremio offers different authentication methods for Dremio Cloud and Dremio Software deployments. Choose the appropriate method and ensure complete their respective prerequisites.

Dremio Cloud

Access to a Dremio Cloud workspace
Access to the ID of the Sonar project you want to use. See Obtaining the ID of a Project.
A valid personal access token (PAT) for authenticating to Dremio Cloud. See Creating a Token.
Python 3.9 or later installed on the system that you are running dbt on.

Dremio Software

Dremio Software version 22.0 or later
Python 3.9 or later installed
Required support keys enabled in your Dremio cluster (See Support Keys docs for details)
- dremio.iceberg.enabled
- dremio.iceberg.ctas.enabled
- dremio.execution.support_unlimited_splits

Setup Instructions

Click Settings in the top menu bar of the Paradime interface to access Account Settings
In the left sidebar, click “Connections”
Click “Add New” next to the Bolt Schedules section
Select “Dremio”
In the Profile Configuration field, add the following parameters based on your authentication method. Reference the Profile Fields Reference section below for additional context and configurations:

Dremio Cloud
Dremio Software (Username/Password)
Dremio Software (Personal Access Token)

cloud_host: https://api.dremio.cloud      # US Control Plane
cloud_project_id: 12345678                # Sonar project ID
object_storage_source: my_storage         # Filesystem name for objects
object_storage_path: data/analytics       # Filesystem path for objects
dremio_space: analytics_space             # Space for view creation
dremio_space_folder: reports.daily        # Space folder path
pat: drm_cl_1a2b3c4d5e6f                 # Personal access token
use_ssl: true                             # Required for Cloud
user: analyst@company.com                 # Your email

software_host: dremio.company.com         # Hostname or IP address
port: 9047                                # Default: 9047
user: dremio_analyst                      # Username
password: your_secure_password            # Password
object_storage_source: company_storage    # Filesystem name for objects
object_storage_path: data/transforms      # Filesystem path for objects
dremio_space: dev_space                   # Space for view creation
dremio_space_folder: testing.models       # Space folder path
threads: 4                                # Number of threads
use_ssl: true                             # Enable SSL

pat: drm_sw_1a2b3c4d5e6f                 # Token for authentication
port: 9047                                # Port for Dremio Software cluster API endpoints
software_host: dremio.internal.net        # Hostname or IP address of the Dremio coordinator node
object_storage_source: analytics_source    # Name of filesystem for creating tables and views
object_storage_path: production/models     # Path in filesystem for objects
dremio_space: prod_space                  # Dremio space for creating views
dremio_space_folder: models.validated     # Folder path in Dremio space
threads: 4                                # Number of threads for dbt project execution
use_ssl: true                             # Enable/disable SSL connection
user: dremio_prod                         # Username for authentication

After adding the configuration, ensure it’s properly formatted using a YAML formatter like YAML Formatter.

Provide a dbt™ Profile Name (This should match with the profile name set in your dbt_project.yml).
In the Target field, enter “prod”. This is the default target your dbt™ project will use for your Bolt Schedules.
The “Threads” value is pre-configured to 1, but you can adjust this number to specify how many parallel operations your dbt project can run.

Example Profile Configuration

Profile Fields Reference

Common Fields
Dremio Cloud
Dremio Software

Field	Description	Default Value	Required
object_storage_source	Source name for storing tables and other objects	`$scratch`	❌
object_storage_path	Path within storage for object creation	`no_schema`	❌
dremio_space	Space designation for view creation	`@username`	❌
dremio_space_folder	Folder structure within Dremio space	`no_schema`	❌

Field	Description	Default Value	Required
cloud_host	Dremio Cloud API endpoint	`https://api.dremio.cloud`	✅
cloud_project_id	Project identifier in Dremio Cloud	None	✅
user	Email address for authentication	None	✅
pat	Authentication token	None	✅
use_ssl	SSL connection enablement	`true`	✅

Field	Description	Default Value	Required
software_host	Host address of Dremio instance	None	✅
port	Connection port number	`9047`	✅
user	Authentication username	None	✅
password	User password (if not using PAT)	None	✅*
pat	Authentication token (if not using password)	None	✅*
use_ssl	SSL connection enablement	`false`	✅

*: Either password or port must be provided, but not both

​Prerequisites

​Dremio Cloud

​Dremio Software

​Setup Instructions

​Example Profile Configuration

​Profile Fields Reference​

Prerequisites

Dremio Cloud

Dremio Software

Setup Instructions

Example Profile Configuration

Profile Fields Reference