Elementary Data
With Elementary, an open-source data observability solution, data and analytics engineers can achieve instant visibility into their dbt™️ projects. It streamlines the process of detecting data anomalies, issuing targeted alerts, and unraveling both impact and root causes.
Getting started
Paradime seamlessly integrates with Elementary, offering a ready-to-use installation of the Elementary CLI.
Install Elementary dbt™️ package
To start using Elementary, you need to add our the Elementary dbt™️ package to your dbt™️ project.
Add elementary to packages.yml
packages.yml
Add the following to your packages.yml
file (if missing, create it where dbt_project.yml
is):
Configure your dbt™️ project
Paradime already provides an Elementary profile configured to work with Elementary based on your default connections (for both IDE and Scheduler connnections.
The schema name is by default set as <target_schema>_elementary
. You can override this by configured a system variable called PARADIME_EDR_SCHEMA.
Check how to configure environment variables here.
Update your dbt™️ project with the below code:
This means Elementary models will have their own schema. Depending on your project custom schema macro, the schema will be named elementary
or <target_schema>_elementary
.
IMPORTANT - Starting from dbt 1.8, users must explicitly allow packages to override materializations. Elementary requires this ability to support collection of samples and failed row count for dbt tests.
Please add the following flag to
dbt_project.yml
to allow it:
Notes:
This is a temporary measure that will result in a deprecation warning, please ignore it for now. Elementary is working with the dbt-core team on a more permanent solution.
This message can be muted by setting the
mute_ensure_materialization_override
var to true.
Build Elementary models
This will mostly create empty tables, that will be updated with artifacts, metrics and test results in your future dbt executions. (This is required both environments, IDE and Scheduler.
Finally, lets run dbt™️ deps to install the package and build the Elementary models in your databse.
Last updated