Dremio
Dremio is a lakehouse platform that enables high-performance SQL queries directly on data lake storage. Paradime supports connections to both Dremio Cloud and Dremio Software. As a development environment in Paradime, Dremio enables dbt™️ development from Paradime's Code IDE.
IP RESTRICTIONS
Make sure to allow traffic from one of the Paradime IPs in your firewall depending on the data location selected.
👉 See also: Paradime IP addresses.
Prerequisites
Dremio offers different authentication methods for Dremio Cloud and Dremio Software deployments. Choose the appropriate method and ensure complete their respective prerequisites.
Dremio Cloud
Access to a Dremio Cloud workspace
Access to the ID of the Sonar project you want to use. See Obtaining the ID of a Project.
A valid personal access token (PAT) for authenticating to Dremio Cloud. See Creating a Token.
Python 3.9 or later installed on the system that you are running dbt on.
Dremio Software
Dremio Software version 22.0 or later
Python 3.9 or later installed
Required support keys enabled in your Dremio cluster (See Support Keys docs for details)
dremio.iceberg.enabled
dremio.iceberg.ctas.enabled
dremio.execution.support_unlimited_splits
Setup Instructions
Click the gear Icon (⚙️) on the top right of the Paradime interface to access Account Settings
In the left sidebar, click "Connections"
Click "Add New" next to the Code IDE section
Select "Dremio"
In the Profile Configuration field, add the following parameters based on your authentication method. Reference the Profile Fields Reference section below for additional context and configurations:
After adding the configuration, ensure it's properly formatted using a YAML formatter like YAML Formatter.
Provide a dbt™ Profile Name (This should match with the profile name set in your
dbt_project.yml
).In the Target field, enter "dev". This is the default target your dbt project will use for development work.
The "Threads" value is pre-configured to 1, but you can adjust this number to specify how many parallel operations your dbt project can run.
Example Profile Configuration
Profile Fields Reference
object_storage_source
Source name for storing tables and other objects
$scratch
❌
object_storage_path
Path within storage for object creation
no_schema
❌
dremio_space
Space designation for view creation
@username
❌
dremio_space_folder
Folder structure within Dremio space
no_schema
❌
Last updated