# Elementary Data

{% hint style="info" %}
For new Elementary data users, we recommend starting out [Elementary Data's Tutorial Guide](https://docs.elementary-data.com/tutorial/tutorial-start) to understand the core concepts and implementation. 
{% endhint %}

With Elementary, an open-source data observability solution, data and analytics engineers can achieve instant visibility into their dbt projects. It streamlines the process of detecting data anomalies, issuing targeted alerts, and unraveling both impact and root causes.

## Getting started

Paradime seamlessly integrates with Elementary, offering a ready-to-use installation of the Elementary CLI.

{% @arcade/embed flowId="20HwrU191OgxtgNqUsQe" url="<https://app.arcade.software/share/20HwrU191OgxtgNqUsQe>" %}

### 1. Install Elementary dbt™️ package

To start using Elementary, you need to add the Elementary dbt package to your dbt project.

### 2. Add elementary to `packages.yml`

Add the following to your `packages.yml`file (if missing, create it on the same path s your`dbt_project.yml)`:

{% code title="dbt\_project.yml" %}

```yaml
packages:
  - package: elementary-data/elementary
    version: 0.15.0
    # check latest package version on https://hub.getdbt.com/elementary-data/elementary/latest/
```

{% endcode %}

{% embed url="<https://hub.getdbt.com/elementary-data/elementary/latest/>" %}

### 3. Configure Elementary Profile in your `dbt_project.yml`

Paradime simplifies the setup process by providing a pre-configured Elementary profile that works with your default connections for both IDE and Scheduler.

By default, the Elementary schema name is set as `<target_schema>_elementary`. You can override this by setting a system variable called `PARADIME_EDR_SCHEMA`. For instructions on configuring environment variables, click [here](https://docs.paradime.io/app-help/documentation/settings/environment-variables).&#x20;

#### 3.1 - Add the following code to you `dbt_project.yml`:

{% code title="dbt\_project.yml" %}

```yaml
models:
  elementary:
    +schema: "elementary"
    ## To disable elementary for dev, uncomment this:
    # enabled: "{{ target.name in ['prod','analytics'] }}"
```

{% endcode %}

This configuration ensures Elementary models have their own schema.&#x20;

#### 3.2 - Add this flag to your `dbt_project.yml`:

{% code title="dbt\_project.yml" %}

```yaml
flags:
  require_explicit_package_overrides_for_builtin_materializations: False
  source_freshness_run_project_hooks: True
```

{% endcode %}

Starting from **dbt 1.8**, you must explicitly allow the Elementary packages to override default dbt™ materializations. Elementary needs this permission to collect samples and failed row counts for dbt tests.

{% hint style="warning" %}
These flags may trigger a deprecation warning. Elementary is collaborating with the dbt core team on a permanent solution. To mute the warning, to mute this warning, add the following flag:
{% endhint %}

{% code title="dbt\_project.yml" %}

```yaml
flags:
  mute_ensure_materialization_override: TRUE
```

{% endcode %}

<details>

<summary>Important: Materialization config</summary>

Elementary requires some models to be created as incremental tables. Ensure there are no global materialization configurations that might interfere with Elementary. For example, avoid settings like:

{% code title="dbt\_project.yml" %}

```yml
materialized: "{{ 'table' if target.name == 'prod-cloud' else 'view' }}"
```

{% endcode %}

Instead, structure your `dbt_project.yml` like this:

{% code title="dbt\_project.yml" %}

```yaml
models:
  my_project:
    materialized: "{{ 'table' if target.name == 'prod-cloud' else 'view' }}"
  elementary:
    +schema: "elementary"
```

{% endcode %}

</details>

### 4. Build Initial Elementary Models

Once you've configured your Elementary in your `packages.yml` and `dbt_project.yml`, Run the following dbt commands to install Elementary and materialize the initial elementary models in your data warehouse:

{% code title="Terminal" %}

```shell
dbt deps
dbt run --select elementary
```

{% endcode %}

This will mostly create empty tables, that will be updated with artifacts, metrics and test results in your future dbt™ executions. (This is required for both environments, IDE and Scheduler.)

### 5. Generate Elementary Tests

After installing and configuring Elementary, you're ready generate a variety on test, including: [Anomaly Detection Tests](https://docs.paradime.io/app-help/documentation/integrations/observability/elementary-data/anomaly-detection-tests), [Schema Tests](https://docs.paradime.io/app-help/documentation/integrations/observability/elementary-data/schema-tests), etc.&#x20;

Additionally, you can leverage you can leverage DinoIA's one-click command, [**Generate Elementary Tests**](https://docs.paradime.io/app-help/guides/paradime-101/getting-started-with-the-paradime-ide/dinoai-accelerating-your-analytics-engineering-workflow/accelerating-data-governance#id-1.-generating-dbt-tm-tests) to automatically create comprehensive tests tailored to your dbt™ model's structure and content. This ensures a seamless setup of data observability across your project.

When complete, execute execute `dbt test`to verify setup.&#x20;

### 6.  (Optional) Configure Slack / Teams notifications

Paradime Alerts allow you to set up and manage notifications for your dbt™ projects, keeping you informed of important events and anomalies. With support for both Slack and Microsoft Teams, you can customize your alerts to fit your team's communication preferences. See [docs](https://docs.paradime.io/app-help/documentation/integrations/observability/elementary-data/sending-alerts) for details.&#x20;

<figure><img src="https://2337193041-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHET0AD04uHMgdeLAjptq%2Fuploads%2FVuBQQFpMv5noeRNTJ3AD%2Fimage.png?alt=media&#x26;token=5f71a5dd-914e-4bd9-85e5-1aa69038cf68" alt="" width="375"><figcaption><p>Example Slack Notifications</p></figcaption></figure>

### 7.  (Optional)  Run Elementary in Production

While this documentation demonstrates running Elementary in a development environment, you can execute Elementary command in your Production Schedules ([Bolt](https://docs.paradime.io/app-help/documentation/bolt))