Column-Level Lineage Diff

​

Key features

• Field-Level Lineage: Identify changes to columns in your dbt™ models and generate a detailed report of all impacted downstream objects.
• Automated Comments: Receive automated comments in your PRs listing all downstream dbt™ models and BI nodes affected by the changes.
• Impact Assessment: Understand what nodes and other elements might be impacted by the changes introduced in the PR.

​

Use cases

• Assess all downstream nodes impacted by changes both within a dbt project and in downstream application (example: BI)
• For Data Mesh architectures, see how your current project’s changes impact other project changes.

​

Tutorial

​

Prerequisites

1. Git Integration: Install the Paradime GitHub app and authorize access to the dbt™ repository used in Paradime or use alternative methods based on your Git Provider. See setup instructions.
2. Production Connection: Add a production connection with access to your sources and models generated when running production jobs. This allows Paradime to run information schema queries and build field-level lineage. See connection guide for instructions based on your data warehouse provider.
3. Have at least one Bolt TurboCI configured. This is required to generate field-level lineage for your dbt™ project. See Bolt Scheduler for configuration.

​

Set up your Git provider

​

Lineage Diff Feature - Supported Use Cases

• Column renaming: When a column is renamed in a SELECT statement
• Column removal: When a column is removed from a SELECT statement
• Column addition: When a new column is added to a SELECT statement

-- Before
SELECT
    customer_id,
    customer_name,
    email
FROM customers

-- After (column renamed)
SELECT
    customer_id,
    full_name,  -- renamed from customer_name
    email
FROM customers

• WHERE clause modifications: Changes to filtering conditions
• JOIN modifications: Adding, removing, or changing JOIN conditions
• GROUP BY changes: Modifications to grouping logic
• ORDER BY changes: Changes to sorting logic

• WHERE clauses: Column references in filtering conditions
• JOIN conditions: Column references in table joins
• GROUP BY clauses: Column references in grouping logic
• ORDER BY clauses: Column references in sorting logic

• Column calculation changes: Modifications to how a column value is computed
• NULL handling changes: Changes in NULL value treatment
• Data type transformations: Changes that affect data representation but not structure

-- Before
SELECT
    customer_id,
    customer_name,
    revenue * 1.1 as adjusted_revenue
FROM customers
WHERE status = 'active'
ORDER BY customer_name

-- After (non-supported changes)
SELECT
    customer_id,
    customer_name,
    revenue * 1.2 as adjusted_revenue  -- calculation change (not detected)
FROM customers
WHERE status IN ('active', 'pending')  -- WHERE clause change (not detected)
ORDER BY full_name  -- ORDER BY with renamed column (not detected)

​

Summary

​

Next steps

1. Connect your Git provider so Paradime can post PR comments.
2. Add a production connection and confirm at least one Bolt schedule runs.
3. Connect BI tools like Looker, Tableau, or ThoughtSpot for wider impact coverage.
4. Open a test PR with a column change and review the generated lineage report.