> ## Documentation Index
> Fetch the complete documentation index at: https://docs.paradime.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Column-Level Lineage Diff on GitLab

> Set up Paradime Column-Level Lineage Diff on GitLab merge requests to post upstream and downstream column-impact reports as MR comments for dbt™ changes.

export const Arcade = ({src, title}) => <div style={{
  position: 'relative',
  paddingBottom: 'calc(56.2225% + 41px)',
  height: 0,
  width: '100%'
}}>
    <iframe src={src} title={title} frameBorder="0" loading="lazy" allow="clipboard-write" allowFullScreen style={{
  position: 'absolute',
  top: 0,
  left: 0,
  width: '100%',
  height: '100%',
  colorScheme: 'light'
}} />
  </div>;

Set up Column-Level Lineage Diff on GitLab. First complete the shared [Lineage Diff prerequisites](/products/bolt/ci-cd/lineage-diff/index), then configure your GitLab pipeline.

<Frame>
  <img src="https://2337193041-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FHET0AD04uHMgdeLAjptq%2Fuploads%2Fl3HYjPJu8jk2PXSiOuv9%2Fimage.png?alt=media&token=1eef3ecf-7218-4125-8205-925f22098123" alt="" />
</Frame>

***

## Generate API keys

Generate a Paradime [API key](/developers/api-keys) with the **Bolt schedules admin** capability. Use an **Account API key** (a `prdm_cmp_...` Bearer token), and note your **workspace token** and **API endpoint**; you will add them as CI/CD variables in a later step.

<Card title="Generate API keys" href="/developers/api-keys" horizontal icon="arrow-right" />

## Create a Pipeline

Now you will need to create a new `.gitlab-ci.yml` file in your dbt™️ repository. Copy the code below.

This pipeline has two stages:

1. **`paradime_turbo_ci`** — Runs the Turbo CI bolt schedule and exports the `BOLT_RUN_ID` for the next stage.
2. **`lineage_diff_report`** — Uses the `BOLT_RUN_ID` to trigger the lineage diff report and posts the result as a comment on the Merge Request.

```yaml theme={"system"}
stages:
  - paradime_turbo_ci
  - lineage_diff_report

paradime_turbo_ci:
  stage: paradime_turbo_ci
  image: python:3.11
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      when: always
  variables:
    PARADIME_SCHEDULE_SLUG: "flowing-pachy-wyy4fs"
  before_script:
    - pip install paradime-io
  script:
    - echo "Merge Request Number: $CI_MERGE_REQUEST_IID"
    - echo "Commit SHA: $CI_COMMIT_SHA"
    - |
      python3 -u << 'EOF'
      import os
      import time
      from paradime import Paradime

      paradime = Paradime(
        workspace_uid=os.environ['PARADIME_WORKSPACE_UID'],
        api_secret=os.environ['PARADIME_API_SECRET'],
        api_endpoint=os.environ['PARADIME_API_ENDPOINT']
      )

      run_id = paradime.bolt.trigger_run(
        slug=os.environ['PARADIME_SCHEDULE_SLUG'],
        branch=os.environ['CI_COMMIT_SHA'],
        pr_number=int(os.environ['CI_MERGE_REQUEST_IID']),
      )
      print(f"Bolt Run ID: {run_id}")

      while True:
        run_status = paradime.bolt.get_run_status(run_id)
        print(f"Status: {run_status.state}")
        if run_status.state in ("success", "error", "failed", "canceled"):
          break
        time.sleep(30)

      with open("bolt.env", "w") as f:
        f.write(f"BOLT_RUN_ID={run_id}\n")

      if run_status.state != "success":
        raise SystemExit(f"Bolt run {run_id} failed with state: {run_status.state}")
      EOF
  artifacts:
    reports:
      dotenv: bolt.env
  timeout: 60 minutes

lineage_diff_report:
  stage: lineage_diff_report
  image: python:3.11
  needs:
    - job: paradime_turbo_ci
      artifacts: true
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      when: always
  variables:
    GIT_DEPTH: 0
  before_script:
    - pip install paradime-io
  script:
    - echo "Bolt Run ID: $BOLT_RUN_ID"
    - echo "Pull Request Number: $CI_MERGE_REQUEST_IID"
    - echo "Pull Request Author Email: $GITLAB_USER_EMAIL"
    - export CHANGED_FILES=$(git diff --name-only $CI_MERGE_REQUEST_DIFF_BASE_SHA $CI_COMMIT_SHA)
    - echo "Files changed in this MR:"; echo "$CHANGED_FILES"
    - export REPORT_FILE="report_$(openssl rand -hex 12).json"
    - python3 -u <<EOF
      import os
      import json
      from pathlib import Path
      from paradime import Paradime
      paradime = Paradime(
        workspace_uid=os.environ['PARADIME_WORKSPACE_UID'],
        api_secret=os.environ['PARADIME_API_SECRET'],
        api_endpoint=os.environ['PARADIME_API_ENDPOINT']
      )
      report = paradime.lineage_diff.trigger_report_and_wait(
        bolt_run_id=int(os.environ['BOLT_RUN_ID']),
        pull_request_number=int(os.environ['CI_MERGE_REQUEST_IID']),
        user_email=os.environ['GITLAB_USER_EMAIL'],
        changed_file_paths=os.environ['CHANGED_FILES'].split('\n'),
      )
      print("Lineage Diff Report Message:", report.message)
      if report.result_markdown:
        data = json.dumps({'body': report.result_markdown})
        Path(os.environ['REPORT_FILE']).write_text(data)
      EOF
    - |
      if [ -f "$REPORT_FILE" ]; then
        COMMENT_BODY=$(cat "$REPORT_FILE")
        curl --request POST --header "PRIVATE-TOKEN: ${GITLAB_TOKEN}" \
          --header "Content-Type: application/json" \
          --data "$COMMENT_BODY" \
          "https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/merge_requests/${CI_MERGE_REQUEST_IID}/notes"
      else
        echo "Report file not found, skipping comment."
      fi
  timeout: 60 minutes
```

## Create a GitLab Access Token

You will need to create GitLab Access Token with API to enable Comments to be generated when opening a Merge Request.

**Create a group access token (Only available on GitLab Premium Tier or Higher)**

Navigate to your GitLab project

1. Go to **Settings > Access Tokens** in the left sidebar
2. Enter a name for your token (e.g., "API Comment Access")
3. Select the appropriate scopes:
   * `api` (for general API access)
   * Specifically for comments, you'll need at least `api` scope, which includes the ability to write comments

**Create a GitLab personal access token**

<Warning>
  If using a GitLab personal access token, we suggest creating a new user, called "Paradime" and create the access token attached to this user.
</Warning>

Log in to your GitLab account

1. Go to your user profile > Preferences > Access Tokens (or navigate to `https://gitlab.com/-/profile/personal_access_tokens`)
2. Enter a name for your token (e.g., "API Comment Access")
3. Select the appropriate scopes:
   * `api` (for general API access)
   * Specifically for comments, you'll need at least `api` scope, which includes the ability to write comments

## Add the API key and Credential in the GitLab variables

Finally you need to add the API key and Credentials generated in the previous step in GitLab CI/CD settings (not in the `.gitlab-ci.yml` file directly). Set the corresponding values for the following variable names:

* `PARADIME_WORKSPACE_UID`
* `PARADIME_API_SECRET`
* `PARADIME_API_ENDPOINT`
* `GITLAB_TOKEN`

<Arcade src="https://demo.arcade.software/UcTn34OhEg5OwXDVOKVu?embed&embed_mobile=tab&embed_desktop=inline&show_copy_link=true" title="Column-Level Lineage Diff" />


## Related topics

- [Column-Level Lineage Diff](/products/bolt/ci-cd/lineage-diff/index.md)
- [Column-Level Lineage Diff on GitHub](/products/bolt/ci-cd/lineage-diff/github.md)
- [Column-Level Lineage Diff on BitBucket](/products/bolt/ci-cd/lineage-diff/bitbucket.md)
- [Column-Level Lineage Diff on Azure DevOps](/products/bolt/ci-cd/lineage-diff/azure-devops.md)
- [Data Mesh Setup](/guides/data-mesh-setup/index.md)