Using Azure Pipelines

You can build your custom Continuous Deployment jobs using Azure Pipelines and Bolt APIs.

To use this feature it is required to have a production environment configured in Paradime.

ℹ️ Check our setup guide here based on your data warehouse provider.

Create a CD Bolt schedule

To start create a new Bolt schedule and make sure to add the deferred_schedule configuration. Depending on your intended behavior you can choose to defer to another production job or defer back to the last CD run.

The deferred_schedule_name set in the configuration should have at least one successful run available, so that Paradime can pick the manifest.json for state comparison.

Example schedule

...
  - name: continuous_deployment_run # the name of your CD job
    deferred_schedule:
      enabled: true # true to enabled this Turbo CI job to run on pull request
      deferred_schedule_name: continuous_deployment_run #the name of the bolt schedule where the CD job will look for the most recent successful run manifest.json for state comparison
    schedule: "OFF" # set the schedule configuration to not run on a schedule (to be used for PR only)
    environment: production #the environment used to run the schedule -> this is always production
    commands:
      - dbt run --select state:modified+ #the dbt™️ command you want to run after the pull request is merged
    owner_email: "[e-mail]" #the email of the CD job owner_email
...

Generate API keys and find you workspace token

API keys are generate at a workspace level.

To be able to trigger Bolt using the API, you will first need to generate API keys for your workspace. Got to account settings and generate your API keys, make sure to save in your password manager:

• API key

• API secret

• API Endpoint

You will need this later when setting up the secrete in Azure pipelines.

API Keys

Create an Azure Pipeline

Now you will need to create a new azure-pipeline.yml file in your dbt™️ repository. Copy the code block below and enter the values required.

Example Azure pipelines configuration file

azure-pipelines.yml

# Define when the pipeline should trigger
trigger:
 - main    # Pipeline will run when changes are pushed to main branch

# Define pipeline-level variables
variables:
 # Name of the Paradime schedule/job to run
 - name: PARADIME_SCHEDULE_NAME
   value: "continuous_deployment_run"

# Define the steps to be executed in this pipeline
steps:
 # Step 1: Set up Python environment
 - task: UsePythonVersion@0    # Azure DevOps task to configure Python
   inputs:
     versionSpec: "3.11"       # Specify Python version
     addToPath: true          # Add Python to system PATH
   displayName: Use Python 3.11 # Name shown in Azure DevOps UI

 # Step 2: Install dependencies and run Paradime job
 # Check for latest version of the Paradime Python SDK on https://github.com/paradime-io/paradime-python-sdk/releases
 - script: |  
     # Install Paradime Python SDK
     pip install paradime-io==4.7.1
     
     # Export required environment variables for Paradime
     # $(VARIABLE) syntax is used to reference Azure Pipeline variables
     export PARADIME_API_ENDPOINT=$(PARADIME_API_ENDPOINT)
     export PARADIME_API_KEY=$(PARADIME_API_KEY)
     export PARADIME_API_SECRET=$(PARADIME_API_SECRET)
     
     # Run the Paradime bolt schedule/job
     # --wait flag makes the pipeline wait for completion
     paradime bolt run "$(PARADIME_SCHEDULE_NAME)" --wait

   displayName: "Run Paradime CD"

Add the API keys and Credential in the Azure Pipeline variables

Finally you need to add the API key and credentials generated in the previous step in Azure Pipelines.

Set the corresponding values using your credentials for the variable names:

• PARADIME_API_KEY

• PARADIME_API_SECRET

• PARADIME_API_ENDPOINT