Using --defer in Paradime

The --defer feature in Paradime allows you to leverage production data and schemas during development, significantly speeding up your dbt™ workflow. This guide will walk you through using --defer, from basic usage to advanced features.

Basic Usage of --defer

With Paradime, you can continuously develop using production data and schema. By enabling defer to prod in a dbt™ run command, Paradime automatically fetches the latest manifest.json, ensuring you always work with the most current data and schema.

Prerequisites

1. A connected Scheduler Connection to your data warehouse

2. An existing Bolt Schedule

3. An Enabled "Defer to Production" Schedule

How it works

When using --defer, dbt™ resolves ref calls based on two criteria:

1. Is the referenced node included in the current run's model selection?

2. Does the reference node exist as a database object in your development environment?

If both answers are No, --defer resolves the ref() using the namespace from the state manifest of the specified schedule.

Using --defer in the Code IDE

When defer to prod if you run a dbt™️ command Paradime will automatically run the dbt command in "defer mode" in the Code IDE's integrated terminal (you wont need to use --defer) but simply :

dbt run --select <model_name>

Example: Deferred vs Standard Run

To illustrate the effect of using defer to prod, here's a comparison of compiled SQL:

Deferred Run

dim_customers.sql

with orders as (

    select * from `dbt-demo-project`.`dbt_prod`.`stg_orders`

),

final as (

    select
        customer_id,
        min(order_date) as first_order,
        max(order_date) as most_recent_order,
        count(order_id) as number_of_orders
    from orders
    group by 1
)

select * from final

Standard Run

dim_customers.sql

with orders as (

    select * from `dbt-demo-project`.`dbt_fabio`.`stg_orders`

),

final as (

    select
        customer_id,
        min(order_date) as first_order,
        max(order_date) as most_recent_order,
        count(order_id) as number_of_orders
    from orders
    group by 1
)

select * from final

Notice how the deferred run uses the production schema (dbt_prod), while the standard run uses the development schema (dbt_fabio).

Viewing the Deferred Schedule

After running a dbt™ command with "defer to prod", you can view details of the production run used for deferral in the Integrated Terminal. The output includes a clickable URL to the Bolt UI for more information.

Advanced Use of --defer

Using --favor-state

The --favor-state flag provides additional control over how dbt™ resolves node references:

dbt run --select customer_orders --favor-state

This command tells dbt™ to prefer the state from the 'daily_run' schedule, even if the models exist in your current environment.

Read about state comparison caveats and limitations.