Using --defer in Paradime
The --defer
feature in Paradime allows you to leverage production data and schemas during development, significantly speeding up your dbt™ workflow. This guide will walk you through using --defer
, from basic usage to advanced features.
Basic Usage of --defer
With Paradime, you can continuously develop using production data and schema. By enabling defer to prod
in a dbt™ run command, Paradime automatically fetches the latest manifest.json
, ensuring you always work with the most current data and schema.
Prerequisites
A connected Production Data Warehouse
An existing Bolt Schedule
An Enabled "Defer to Production" Schedule
How it works
When using --defer
, dbt™ resolves ref
calls based on two criteria:
Is the referenced node included in the current run's model selection?
Does the reference node exist as a database object in your development environment?
If both answers are No, --defer
resolves the ref()
using the namespace from the state manifest of the specified schedule.
Using --defer in the Terminal
Similarly, You can use --defer
via the Code IDE's integrated terminal:
Example: Deferred vs Standard Run
To illustrate the effect of using --defer
, here's a comparison of compiled SQL:
Notice how the deferred run uses the production schema (dbt_prod
), while the standard run uses the development schema (dbt_fabio
).
Viewing the Deferred Schedule
After running a dbt™ command with --defer
, you can view details of the production run used for deferral in the Integrated Terminal. The output includes a clickable URL to the Bolt UI for more information.
Advanced Use of --defer
Using --favor-state
The --favor-state
flag provides additional control over how dbt™ resolves node references:
This command tells dbt™ to prefer the state from the 'daily_run' schedule, even if the models exist in your current environment.
Read about state comparison caveats and limitations.
The "source_status" Method
The source_status
method allows you to run only models with dependencies on fresher source data. Here's an example configuration:
This configuration runs dbt source freshness
and then builds only models affected by fresher source data.
Read more here about state
method or source_status
method.
Additional schedule parameters available for deferred_schedule
deferred_schedule
When configuring a deferred schedule, you can use the following parameters in the deferred_schedule
section:
Parameters | Description | Example |
---|---|---|
| [Required] Set this to TRUE to enable deferral for the schedule |
|
| [Required] The name of the Bolt schedule used to look for the most recent successful run artifacts (manifest.json, sources.json, run_results.json) for state comparison. It can be another schedule or the same schedule name (self-deferring). |
|
| [Optional] By default, Paradime will look for the last successful run. Set this to |
|
Last updated