Audit Variables

Paradime provides a set of system environment variables that can be added to your dbt™ models to inject metadata at run time to add more information to your materialized tables in your data warehouse. These can be used to help identify which schedules run relates to a given row of data in your tables.

Variable name	Example
`PARADIME_WORKSPACE_NAME`	jaffle_shop
`PARADIME_SCHEDULE_NAME`	Hourly run (the schedule’s display name)
`PARADIME_SCHEDULE_SLUG`	hourly-run-a1b2c3 (the schedule’s unique identifier)
`PARADIME_SCHEDULE_RUN_ID`	39828
`PARADIME_SCHEDULE_RUN_START_DTTM`	2022-07-15 21:50:21.268372
`PARADIME_SCHEDULE_TRIGGER`	one of: • scheduler• manual from user@email.com• API• TURBO CI
`PARADIME_SCHEDULE_GIT_SHA`	4a9fcff9277dfabd447157dea86dd7fdbd17d6f8
`GITHUB_PR_NUMBER` (available only in TurboCI or on Merge runs when the GitHub app integration is configured).	1253

Include env variables in your dbt™ models to audit your runs

Include any of the above variable in your dbt™ models. At run time, for each row of data in your data warehouse tables, with metadata to help audit each row.

{% set payment_methods = ['credit_card', 'coupon', 'bank_transfer', 'gift_card'] %}

with orders as (

    select * from {{ ref('stg_orders') }}

),

order_payments as (

    select * from {{ ref('order_payments') }}

),

final as (

    select
        orders.order_id,
        orders.customer_id,
        orders.order_date,
        orders.status,
        

        {% for payment_method in payment_methods -%}

        order_payments.{{payment_method}}_amount,

        {% endfor -%}

        order_payments.total_amount as amount

    from orders

    left join order_payments using (order_id)

)

select 

    *,
    -- Inject the audit metadata if available, otherwise use "manual"
    '{{ env_var("PARADIME_WORKSPACE_NAME", "manual") }}' as _audit_workspace_name,
    '{{ env_var("PARADIME_SCHEDULE_NAME", "manual") }}' as _audit_schedule_name,
    '{{ env_var("PARADIME_SCHEDULE_SLUG", "manual") }}' as _audit_schedule_slug,
    '{{ env_var("PARADIME_SCHEDULE_RUN_ID", "manual") }}' as _audit_schedule_run_id,
    '{{ env_var("PARADIME_SCHEDULE_RUN_START_DTTM", "manual") }}' as _audit_schedule_run_start_dttm,
    '{{ env_var("PARADIME_SCHEDULE_TRIGGER", "manual") }}' as _audit_schedule_trigger,
    '{{ env_var("PARADIME_SCHEDULE_GIT_SHA", "manual") }}' as _audit_schedule_git_sha

from final

Trace a run in your warehouse logs

To find which schedule and run sent a query, add the audit variables to the dbt™ query comment in dbt_project.yml. dbt™ then adds them to every query it sends, so they show in your warehouse query history.

query-comment:
  comment: "schedule={{ env_var('PARADIME_SCHEDULE_SLUG', 'manual') }} run_id={{ env_var('PARADIME_SCHEDULE_RUN_ID', 'manual') }}"

Use PARADIME_SCHEDULE_SLUG to identify the schedule. It does not change when you rename the schedule. Use PARADIME_SCHEDULE_RUN_ID to identify one run.

FAQ

What is the hash in job names like schedule-user-manual-<id>?

Bolt gives each run that is not started by the cron schedule a new random ID. Manual, API, Turbo CI and on-merge runs all get one. The ID is not the schedule ID or the run ID, so you cannot use it to find the schedule. Use PARADIME_SCHEDULE_SLUG and PARADIME_SCHEDULE_RUN_ID in your logs instead, as shown above.

Can I set the job name of a schedule?

No. The job name is generated by Bolt. To give a schedule an identifier that you choose, set a slug in your schedule YAML. Bolt uses the slug in the API, webhooks and the PARADIME_SCHEDULE_SLUG variable. See Schedule slugs for the format and how to mint one.

Schedule Anatomy

Deploy Schedules

Managing Schedules

CI/CD

Include env variables in your dbt™ models to audit your runs

Trace a run in your warehouse logs

FAQ

​Include env variables in your dbt™ models to audit your runs

​Trace a run in your warehouse logs

​FAQ

Include env variables in your dbt™ models to audit your runs

Trace a run in your warehouse logs

FAQ