> ## Documentation Index
> Fetch the complete documentation index at: https://docs.paradime.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom Integration

> Extend Paradime lineage and catalog with your own sources by creating custom integrations and uploading nodes from Python using the paradime-io SDK.

### Overview

<Info>
  **Prerequisites:**

  * This feature is available with the [**Paradime Enterprise pack**](https://www.paradime.io/enterprise).
  * Your API keys ***must*** have the Custom Integrations Admin capability.
</Info>

<Info>
  These examples authenticate with an **account API key** (`api_secret="prdm_cmp_..."` plus `workspace_uid`), which requires `paradime-io` 6.0.0 or later. Legacy **workspace API keys** (`api_key` + `api_secret`) are still supported. See Getting Started.
</Info>

The Custom Integration API empowers users to seamlessly extend Paradime's lineage and catalog capabilities with external applications. This powerful interface enables you to:

* Create custom integrations tailored to your specific needs
* Ingest and manage nodes from various data sources and applications
* Upload node information for Paradime to incorporate into its lineage and catalog
* Enhance data visibility by connecting Paradime with your entire data ecosystem

By leveraging this module, you can create a comprehensive, interconnected view of your data landscape, improving data governance, traceability, and insights across your organization.

You can find examples and a template on how to get started [here](https://github.com/paradime-io/paradime-custom-integration-api-examples)

### Create a custom integration and upload nodes using JSON

This example uses as inputs the `node_types.json` and the `nodes.json`.

<Tabs>
  <Tab title="node_types.json">
    <Info>
      The below example is where we define our node types and the related attributes.
    </Info>

    **Defining color for a node:**

    The `color` can be one defined from one of the provided color palettes:

    * VIOLET `#827be6`
    * ORANGE `#fb982e`
    * MANDY `#ef6292`
    * TEAL `#33a9a9`
    * GREEN `#27ae60`
    * CORAL `#FF8559`
    * LEAF
    * CYAN

    **Defining icon for a node:**

    * `icon_name` can be chosen from [Blueprint.js icons library](https://blueprintjs.com/docs/#icons/icons-list)

    **Example node\_type.json**

    ```json theme={"system"}
    [
        {
            "node_type": "Datasource",
            "icon_name": "database",
            "color": "ORANGE"
        },
        {
            "node_type": "Chart",
            "icon_name": "pie-chart",
            "color": "TEAL"
        },
        {
            "node_type": "Dashboard",
            "icon_name": "dashboard",
            "color": "CORAL"
        }
    ]
    ```
  </Tab>

  <Tab title="nodes.json">
    <Info>
      The below example is where we define the attributes for each node and the related upstream/downstream dependencies. For all types and attributes refer to types [here](https://github.com/paradime-io/paradime-python-sdk/blob/main/paradime/apis/custom_integration/types.py).
    </Info>

    **Example nodes.json**

    ```json theme={"system"}
    [
        {
            "name": "Finance Mart",
            "node_type": "Datasource",
            "attributes": {
                "description": "This is my first datasource"
            },
            "lineage": {
                "upstream_dependencies": [
                    {
                        "table_name": "order_items"
                    }
                ]
            }
        },
        {
            "name": "Daily Revenue",
            "node_type": "Chart",
            "attributes": {
                "description": "This is my first chart"
            },
            "lineage": {
                "upstream_dependencies": [
                    {
                        "integration_name": "MyParadimeIntegration",
                        "node_type": "Datasource",
                        "node_name": "Finance Mart"
                    }
                ],
                "downstream_dependencies": [
                    {
                        "integration_name": "MyParadimeIntegration",
                        "node_type": "Dashboard",
                        "node_name": "Finance Daily Report"
                    }
                ]
            }
        },
        {
            "name": "Finance Daily Report",
            "node_type": "Dashboard",
            "attributes": {
                "description": "This is my first dashboard"
            },
            "lineage": {
                "upstream_dependencies": [],
                "downstream_dependencies": []
            }
        }
    ]
    ```
  </Tab>
</Tabs>

```python theme={"system"}
# First party modules
import json
from pathlib import Path
from typing import List

from paradime import Paradime
from paradime.apis.custom_integration.types import Node, NodeType
from paradime.tools.pydantic import parse_obj_as

# Create a Paradime client with your API credentials
paradime = Paradime(api_endpoint="API_ENDPOINT", api_secret="prdm_cmp_...", workspace_uid="WORKSPACE_UID")

# Load node types and nodes from JSON files
node_types = parse_obj_as(List[NodeType], json.loads(Path("node_types.json").read_text()))
nodes = parse_obj_as(List[Node], json.loads(Path("nodes.json").read_text()))

# Create a custom integration or update it if it already exists
my_integration = paradime.custom_integration.upsert(
    name="MyParadimeIntegration",
    logo_url="https://example.com/logo.png", # Optional, replace with the logo URL of the integration, or remove this line.
    node_types=node_types,
)

# Add nodes to the custom integration.
paradime.custom_integration.add_nodes(
    integration_uid=my_integration.uid,
    nodes=nodes,
)
```

### Methods

All methods are called on `paradime.custom_integration`.

#### create

Creates a custom integration with the specified name, logo URL, and node types.

<ParamField path="name" type="str" required>
  The name of the custom integration.
</ParamField>

<ParamField path="logo_url" type="Optional[str]" required>
  The URL of the logo for the custom integration. If not provided (`None`), a default logo is used.
</ParamField>

<ParamField path="node_types" type="List[NodeType]" required>
  A list of `NodeType` objects representing the node types for the custom integration.
</ParamField>

<ResponseField name="returns" type="str">
  The integration UID of the created custom integration.
</ResponseField>

All arguments are keyword-only.

#### update

Updates a custom integration with the specified parameters. Only the parameters that are not `None` are updated.

<ParamField path="integration_uid" type="str" required>
  The unique identifier of the integration.
</ParamField>

<ParamField path="name" type="Optional[str]" default="None">
  The new name of the integration.
</ParamField>

<ParamField path="logo_url" type="Optional[str]" default="None">
  The new logo URL of the integration.
</ParamField>

<ParamField path="node_types" type="Optional[List[NodeType]]" default="None">
  The new list of node types for the integration. Overrides the existing node types.
</ParamField>

<ParamField path="active" type="Optional[bool]" default="None">
  Whether the integration should be active.
</ParamField>

<ResponseField name="returns" type="None">
  Returns nothing.
</ResponseField>

All arguments are keyword-only.

#### upsert

Upserts an integration by either updating an existing integration with the given name, or creating a new integration if it doesn't exist.

<ParamField path="name" type="str" required>
  The name of the integration.
</ParamField>

<ParamField path="logo_url" type="Optional[str]" default="None">
  The URL of the integration's logo.
</ParamField>

<ParamField path="node_types" type="List[NodeType]" required>
  A list of node types associated with the integration.
</ParamField>

<ResponseField name="returns" type="Integration">
  The upserted integration.
</ResponseField>

All arguments are keyword-only.

#### list\_all

Retrieves a list of all custom integrations, including both active and inactive integrations. Takes no arguments.

<ResponseField name="returns" type="List[Integration]">
  A list of `Integration` objects representing the custom integrations.
</ResponseField>

#### list\_active

Retrieves a list of all active custom integrations. Takes no arguments.

<ResponseField name="returns" type="List[Integration]">
  A list of `Integration` objects representing the active custom integrations.
</ResponseField>

#### get

Retrieves an active integration with the specified UID.

<ParamField path="uid" type="str" required>
  The UID of the integration to retrieve. Positional argument.
</ParamField>

<ResponseField name="returns" type="Optional[Integration]">
  The `Integration` object if found, otherwise `None`.
</ResponseField>

#### get\_by\_name

Retrieves an active custom integration with the specified name.

<ParamField path="name" type="str" required>
  The name of the integration to retrieve. Positional argument.
</ParamField>

<ResponseField name="returns" type="Optional[Integration]">
  The `Integration` object if found, otherwise `None`.
</ResponseField>

#### add\_nodes

Adds all nodes to a new snapshot in the custom integration. Internally batches the nodes across multiple requests. To add nodes in a streaming fashion, use `add_nodes_to_snapshot`.

<ParamField path="integration_uid" type="str" required>
  The unique identifier of the integration.
</ParamField>

<ParamField path="nodes" type="List[Node]" required>
  The list of all nodes to be added to the integration.
</ParamField>

<ResponseField name="returns" type="None">
  Returns nothing.
</ResponseField>

All arguments are keyword-only.

#### add\_nodes\_to\_snapshot

Adds nodes to a snapshot in the custom integration. A snapshot is a collection of nodes added to the custom integration. Nodes are added in batches, and the snapshot ID keeps track of the nodes added to the snapshot. This method is useful when adding a large number of nodes in batches.

<ParamField path="integration_uid" type="str" required>
  The UID of the custom integration.
</ParamField>

<ParamField path="nodes" type="List[Node]" required>
  The list of nodes to be added to the snapshot.
</ParamField>

<ParamField path="snapshot_has_more_nodes" type="bool" required>
  Indicates whether the snapshot has more nodes. Set to `False` when adding the last batch of nodes.
</ParamField>

<ParamField path="snapshot_id" type="Optional[int]" default="None">
  The ID of the snapshot. If not provided, a new snapshot is created. If provided, the nodes are added to the existing snapshot.
</ParamField>

<ResponseField name="returns" type="int">
  The ID of the snapshot after adding the nodes.
</ResponseField>

All arguments are keyword-only.


## Related topics

- [Custom Integration API](/developers/graphql-api/api-reference/custom-integration-api.md)
- [Custom Tests](/guides/dbt-fundamentals/configuring-your-dbt-project/custom-tests.md)
- [Deploying from a Custom Branch](/products/bolt/creating-schedules/schedules-as-code/deploying-yaml-schedules-from-a-custom-branch.md)
- [Alerts Configuration and Customization](/integrations/elementary-data/sending-alerts/alerts-configuration-and-customization.md)
- [Customize the Slack Agent](/products/dino-ai/slack-agent/customize-agent.md)