dbt™️ --defer to production

With Paradime you develop continuously using production data and production schema to read when building a model and write in your development. Paradime allows to add to your dbt™️ command which production schedule you want to use to get the manifest.json to get the state.

Using the --defer flag dbt™️ will resolve ref depending on two criteria:

1. Is the referenced node included in the model selection criteria of the current run? I.e does your command looks like dbt run --select +dim_customer.

2. Does the reference node exist as a database object in the current environment? I.e. is the model used in the ref() available in your development database/schema.

If for both criteria the answer is No, then --defer will will resolve the ref() using the namespace provided by the state manifest of the selected schedule name using --schedule-name=hourly in your dbt™️ command.

Deferred Run

$ dbt run --select dim_customers.sql --defer --schedule-name=hourly

dim_customers.sql

with orders as (

    select * from `dbt-demo-project`.`dbt_prod`.`stg_orders`

),

final as (

    select
        customer_id,
        min(order_date) as first_order,
        max(order_date) as most_recent_order,
        count(order_id) as number_of_orders
    from orders
    group by 1
)

select * from final

Standard Run

$ dbt run --select dim_customers

dim_customers.sql

with orders as (

    select * from `dbt-demo-project`.`dbt_fabio`.`stg_orders`

),

final as (

    select
        customer_id,
        min(order_date) as first_order,
        max(order_date) as most_recent_order,
        count(order_id) as number_of_orders
    from orders
    group by 1
)

select * from final

Using favor state

You can optionally skip the second criterion by passing the --favor-state flag. If passed, dbt™️ will favor using the node defined in your --state namespace, even if the node exists in the current target.

Example dbt™️ command using --defer and --favor-state

$ dbt run --select customer_orders --defer --favor-state --schedule-name=daily_run

View the schedule used for --defer

In the Integrated Terminal, after running a dbt™️ command using the --defer to production feature you will be able also to see details of the runID used for to get manifest.json.

The full URL of the schedule runID will be provided in the terminal output so you can ⌘+Click / Ctrl+Click to go to the Bolt UI and see all the details of the production run.