ref, without installing the producer as a package. Paradime resolves these references with dbt-loom, which downloads the latest manifest.json from a Bolt schedule in the producer workspace each time dbt™ runs in the consumer project.
Prerequisites
- Both workspaces run dbt™ 1.7 to 1.12. dbt-loom isn’t available in workspaces on dbt™ 2.0.
- The producer project sets
access: publicon the models you want to share. - A Bolt schedule in the producer workspace has run successfully since those models became public, with a command that writes
target/manifest.json, such asdbt build,dbt run, ordbt compile. You don’t need a new schedule: any producer schedule that runs regularly works. - Admin access in the producer workspace, to create an API key, and in each consumer workspace, to set environment variables.
Steps
1
Copy the producer schedule's slug
dbt-loom finds the producer schedule by its slug, not its display name. In the producer workspace, open the schedule in Bolt and click the copy icon next to its name (Copy schedule slug). You can also hover over a schedule’s name in the Bolt schedules list to see its slug.New schedules get a generated slug, such as
running-raptor-a3f1b2. A schedule created before slugs were introduced uses its original name as its slug. Either way, use the value you copied.2
Create an API key in the producer workspace
In the producer workspace, go to Settings > API Keys, open the Workspace tab, and generate an API key with only the Bolt schedules metadata viewer capability. Copy the API Key, API Secret, and API Endpoint: you can’t see them again after you leave the screen.A workspace API key works on every dbt™ version that has dbt-loom. On dbt™ 1.11 and 1.12, you can use an account API key instead. See Use an account API key.
3
Add dbt_loom.config.yml to the consumer project
In the consumer project’s repository, create a file named
dbt_loom.config.yml in the same directory as dbt_project.yml, and describe the producer:- dbt™ 1.11 and 1.12
- dbt™ 1.7 to 1.10
dbt_loom.config.yml
namemust match thenamein the producer’sdbt_project.yml.- The
${...}values are the names of environment variables that you set in the next step. Use names that are specific to the producer, so they don’t clash with anyPARADIME_API_KEY,PARADIME_API_SECRET, orPARADIME_API_ENDPOINTset for your own workspace. - On dbt™ 1.11 and 1.12,
schedule_namestill works, but dbt-loom logs a deprecation note. Useschedule_slug. - If you also run dbt™ with dbt-loom outside Paradime, for example in VS Code or Cursor, use the dbt™ 1.7 to 1.10 form with a workspace API key, and set the same variables in your local environment. The public dbt-loom package doesn’t support
schedule_slug,workspace_uid, or account API keys.
4
Set the variables once in each consumer workspace
In the consumer workspace, go to Settings > Environment Variables and click Add variable. Enter a variable name from your config as the Key, for example
PRODUCER_API_KEY, enter its value in the Default box, leave the other boxes empty, and click Save. Repeat for each variable.Default values are inherited by every environment in the workspace: Bolt schedules, every developer’s Code IDE, catalog builds, and DinoAI agent sessions. Developers don’t set anything themselves, and they don’t need access to the producer workspace, because dbt-loom authenticates with the API key, not with their account. See Environment Variables.A developer’s Code IDE picks up a new Default value the next time their IDE profile is written, for example when they save any of their own Code IDE variables. A value in a more specific box, such as a developer’s own Code IDE value, overrides the default for that environment. If only some people see dbt-loom errors, check whether they set the same names for themselves.
5
Reference producer models
In the consumer project, reference a producer model with the two-argument
ref, with the producer’s project name first:Only models with
access: public can be referenced from the producer project.In the consumer workspace, run
dbt parse in the Code IDE terminal. If it finishes without a dbt-loom error, the producer’s public models are available to ref. If it fails, see Troubleshooting.Use an account API key
On dbt™ 1.11 and 1.12, you can authenticate with an account API key instead of a workspace API key. An account API key isn’t tied to one workspace, so you also setworkspace_uid to the producer workspace’s UID, and you leave out api_key:
dbt_loom.config.yml
- Go to Settings > API Keys, open the Company tab, and generate a key with only the Bolt schedules metadata viewer capability and access to only the producer workspace. You need to be an admin of the producer workspace.
- Copy the API Token and API Endpoint. The success screen lists each workspace you selected with its UID: copy the producer workspace’s UID.
- Set the three variables in the Default box of each consumer workspace, as in step 4.
Lineage across workspaces
Catalog lineage crosses one workspace boundary in each direction. In a consumer workspace, the producer models you reference appear as nodes tagged with the producer workspace, and lineage stops there. In the producer workspace, a model’s lineage shows the models in other workspaces that reference it. See dbt™ Mesh projects. To see a pull request’s impact on consumer projects, set up Lineage Diff for dbt™ Mesh. It needs aparadime_dbt_mesh schedule in each consumer workspace.
Next steps
Lineage Diff for dbt™ Mesh
Show a pull request’s impact on consumer projects.
Environment Variables
How the Default, Bolt, Code IDE, and Agent scopes combine.
Troubleshooting
Schedule for uuid ... could not be found in our system(dbt™ 1.10 to 1.12) orNo schedule name '...'(dbt™ 1.7 to 1.9): there is no schedule with that slug in the workspace the key belongs to. Check that the config holds the schedule’s slug, not its display name, and that the key comes from the producer workspace. For an account API key, checkworkspace_uid.the following environment variable(s) referenced in the config are not set(dbt™ 1.11 and 1.12): add the named variables in the Default box of the consumer workspace.A Paradime API key, secret, and endpoint must be provided to dbt-loom(dbt™ 1.7 to 1.10): the variables your config names are missing in this environment. Set those names. You don’t needPARADIME_API_KEY,PARADIME_API_SECRET, orPARADIME_API_ENDPOINT.No valid api key was provided: the key or secret is wrong, expired, or revoked. Create a new key and update the variables.api key does not have capability: give the key the Bolt schedules metadata viewer capability.workspace_uid is required when authenticating with a company-level API token: addworkspace_uidto the config.company API key is not authorised for workspace: give the account API key access to the producer workspace, or correctworkspace_uid.No runs found for schedule,No latest run ID found for schedule, orNo artifact found for schedule: the producer schedule has no recent run with a successful command that wrotetarget/manifest.json. Run the schedule, or point to one that runsdbt build,dbt run, ordbt compile. dbt™ 1.10 to 1.12 search the schedule’s 50 most recent runs. dbt™ 1.7 to 1.9 check only its latest run.- No lineage for a consumer project in the Code IDE: the lineage view can show no lineage instead of the dbt-loom error. Check the build status in the Catalog tab, or run
dbt parsein the terminal.