Schema changes from baseline

The elementary.schema_changes_from_baseline test checks for schema changes against baseline columns defined in a source's or model's configuration. For this test to work, the configuration should contain columns and data types.

Sources configuration

Sources

version: 2

sources:
  - name: < source name >
    database: < database name >
    schema: < schema name >
    tables:
      - name: < table name >
        columns:
          - name: < column 1 >
            data_type: < data type 1 >
          - name: < column 2 >
            data_type: < data type 2 >
        tests:
          - elementary.schema_changes_from_baseline

Sources example

version: 2

sources:
  - name: < source name >
    database: < database name >
    schema: < schema name >
    tables:
      - name: < table name >
        columns:
          - name: < column 1 >
            data_type: < data type 1 >
          - name: < column 2 >
            data_type: < data type 2 >
        tests:
          - elementary.schema_changes_from_baseline

Models configuration

Models

version: 2

models:
  - name: < model name >
    columns:
      - name: < column 1 >
        data_type: < data type 1 >
      - name: < column 2 >
        data_type: < data type 1 >
    tests:
      - elementary.schema_changes_from_baseline

Models example

version: 2

models:
  - name: login_events
    columns:
      - name: event_name
        data_type: text
      - name: event_id
        data_type: integer
    tests:
      - elementary.schema_changes_from_baseline:
          tags: ["elementary"]

Supported Parameters

• fail_on_added: If set, the test will fail if there are columns in the table that do not exist in the baseline (default: False).

• enforce_types: If set, the test will raise an error if there are columns defined without a data type (default: False).

Auto-generate Baseline Schema

To make it easier to configure schema tests, Elementary provides dbt operations to auto-generate tests configuration based on the existing schemas.

Usage

# Generate a schema changes from baseline test for all sources
dbt run-operation elementary.generate_schema_baseline_test

# Generate a test for a specific model/source named "orders"
dbt run-operation elementary.generate_schema_baseline_test --args '{"name": "orders"}'

# Generate tests for all sources and all models
dbt run-operation elementary.generate_schema_baseline_test --args '{"include_models": true}'

# Generate tests with "fail_on_added" and "enforce_types" set to true
dbt run-operation elementary.generate_schema_baseline_test --args '{"fail_on_added": true, "enforce_types": true}'

This command will output the generated configuration for the schema changes from baseline test, which can be copied and pasted into the relevant yml file.