Lineage Diff

Prerequisites:

This feature is available with the Paradime Bolt plan.
Your API keys must have either Bolt Schedules Admin or Bolt Schedules Metadata Viewer capabilities.

These examples authenticate with an account API key (api_secret="prdm_cmp_..." plus workspace_uid), which requires paradime-io 6.0.0 or later. Legacy workspace API keys (api_key + api_secret) are still supported. See Getting Started.

The Lineage Diff module allows you to easily interact with Paradime Lineage Diff feature to trigger, generate and return column level lineage reports.

Trigger Lineage Diff report and wait until report is returned

Triggers a lineage diff report for the specified parameters and waits until generation is completed.

int

required

The ID of the completed Turbo CI bolt run.

int

required

The number of the pull request.

str

required

The email of the user triggering the report (pull request author).

List[str]

required

A list of file paths that have changed in the pull request.

int

default:"3600"

Maximum time in seconds to wait for the report to be available.

Report

The lineage diff report.

# First party modules
from paradime import Paradime

# Create a Paradime client with your API credentials
paradime = Paradime(api_endpoint="API_ENDPOINT", api_secret="prdm_cmp_...", workspace_uid="WORKSPACE_UID")

# Replace with all the arguments to generate the lineage diff report
BOLT_RUN_ID = 12345
PULL_REQUEST_NUMBER = 24
USER_EMAIL = "john@acme.io"
CHANGED_FILES = ["dbt/models/marts/core/order_items.sql", "dbt/models/marts/core/new_model.sql"]

# Trigger lineage diff report and wait until completed
report = paradime.lineage_diff.trigger_report_and_wait(
    bolt_run_id=BOLT_RUN_ID,
    pull_request_number=PULL_REQUEST_NUMBER,
    user_email=USER_EMAIL,
    changed_file_paths=CHANGED_FILES,
)

Trigger Lineage Diff report

Triggers a lineage diff report for the specified parameters and returns the report UUID immediately, without waiting for generation to complete.

int

required

The ID of the completed Turbo CI bolt run.

int

required

The number of the pull request.

str

required

The email of the user triggering the report (pull request author).

List[str]

required

A list of file paths that have changed in the pull request.

str

The UUID of the triggered lineage diff report.

# First party modules
from paradime import Paradime

# Create a Paradime client with your API credentials
paradime = Paradime(api_endpoint="API_ENDPOINT", api_secret="prdm_cmp_...", workspace_uid="WORKSPACE_UID")

# Replace with all the arguments to generate the lineage diff report
BOLT_RUN_ID = 12345
PULL_REQUEST_NUMBER = 24
USER_EMAIL = "john@acme.io"
CHANGED_FILES = ["dbt/models/marts/core/order_items.sql", "dbt/models/marts/core/new_model.sql"]

# Trigger lineage diff report and get the UUID
report_uuid = paradime.lineage_diff.trigger_report(
    bolt_run_id=BOLT_RUN_ID,
    pull_request_number=PULL_REQUEST_NUMBER,
    user_email=USER_EMAIL,
    changed_file_paths=CHANGED_FILES,
)

Fetch Lineage Diff report

Fetches a lineage diff report by its UUID.

str

required

The UUID of the lineage diff report.

Report

The lineage diff report.

# First party modules
from paradime import Paradime

# Create a Paradime client with your API credentials
paradime = Paradime(api_endpoint="API_ENDPOINT", api_secret="prdm_cmp_...", workspace_uid="WORKSPACE_UID")

# Replace with the UUID of a previously triggered lineage diff report
REPORT_UUID = "d1e2f3a4-5678-90ab-cdef-1234567890ab"

# Fetch the lineage diff report
report = paradime.lineage_diff.fetch_report(uuid=REPORT_UUID)

Report object details

The report object provides several options that can be selected with the following attributes:

message: A string containing details of the report status.
status: Indicates the status of the operation or request.
url: A string representing a URL to the Bolt schedule compiling the two git branches.
uuid: A unique identifier for the report.
result_json: Contains the result in JSON format.
result_markdown: Provides the result in Markdown format.

API Keys & Access

GraphQL API

Python SDK

Paradime CLI

Webhooks

Virtual Environments

Trigger Lineage Diff report and wait until report is returned

Trigger Lineage Diff report

Fetch Lineage Diff report

Report object details

​Trigger Lineage Diff report and wait until report is returned

​Trigger Lineage Diff report

​Fetch Lineage Diff report

​Report object details

Trigger Lineage Diff report and wait until report is returned

Trigger Lineage Diff report

Fetch Lineage Diff report

Report object details