Elementary Data

Elementary is an open-source data observability solution for data & analytics engineers.

Monitor your dbt™️ project and data in minutes, gain immediate visibility, detect data issues, send actionable alerts, and understand the impact and root cause.

Paradime integrates natively with Elementary Data CLI to enable you to develop v

Check the Elementary Data guide on how you can configure your dbt™️ to computing metrics & anomalies for your dbt™️ models & sources.

See also Elementary docs reference here:

Make sure to first setup the Elementary Data dbt™️ package in your dbt™️ project.

Elementary Data in development

Paradime supports Elementary Data CLI so that you can generate the Elementary data observability report. Check Elementary Data docs here.

After running edr report in the Paradime terminal, this will generate an HTML and JSON files and saves this in a folder called edr_target in your dbt™️ project.

We suggest overriding the default location of the Elementary CLI artifacts by using the the environment variable EDR_REPORT_TARGET_PATHand setting target/edr_targetas the env variable value. Make sure to add this environment variable for each Paradime user

To preview the Elementary data observability report directly from Paradime, using the terminal, navigate to the folder containing elementary output, if followed the above guide it should be target/edr_target and run the below command:

$ paradime serve

You can then click on the generated URL in your terminal to open the Elementary Data observability report in a new browser tab and explore your report.

When done serving the Elementary Data observability report, use Ctrl+C to interrupt the process in the terminal.

Elementary Data in production

You can run Elementary Data CLI in production using Paradime Bolt scheduler. Simply add Elementary CLI command to the command configuration in your production run.

We suggest overriding the default location of the Elementary CLI artifacts by using the the environment variable EDR_REPORT_TARGET_PATHand setting target/edr_targetas the env variable value. Make sure to add this environment variable at the workspace level.

paradime_schedules.yml
schedules:
  - name: run_with_re_data
    schedule: "*/10 * * * *"
    environment: production
    commands:
      - dbt seed
      - dbt run
      - dbt test
      - edr report --env prod #Elementary Data command to generate the observability report
    owner_email: "john@acme.io"
    slack_on:
      - passed
      - failed
    slack_notify:
      - "#data-alerts"
      - "@john"
    email_notify:
      - "john@acme.com"
      - "data_team@acme.com"
    email_on:
      - passed
      - failed

Last updated