> ## Documentation Index
> Fetch the complete documentation index at: https://docs.paradime.io/llms.txt
> Use this file to discover all available pages before exploring further.

# dlt Source and Destination Credentials via Environment Variables

> Learn how dlt (data load tool) to maps environment variables to source and destination credentials.

## Setting Environment Variables in dlt

In this guide, you'll learn how to configure credentials and secrets for your dlt pipelines using environment variables. By the end of this 5-minute tutorial, you'll understand the naming convention, how to translate your `secrets.toml` keys into environment variable names, and how to apply this across common use cases.

***

### How dlt Reads Configuration

dlt retrieves configuration and secrets from multiple locations in the following order of priority:

1. **Environment Variables** — highest priority; if a value is found here, dlt stops searching
2. **`secrets.toml`** — for secrets like API keys and passwords
3. **`config.toml`** — for non-sensitive configuration values
4. **Default values** — defined in your pipeline code

***

### Naming Convention

Environment variables follow a specific naming convention that maps directly to the structure of your `secrets.toml` or `config.toml` files.

The rules are simple:

* All letters are **capitalized**
* Nested sections (dots `.` in TOML) are replaced with **double underscores `__`**

For example, the following `secrets.toml` entry:

```toml theme={"system"}
[destination.bigquery.credentials]
project_id = "my_project"
private_key = "my_key"
client_email = "my_email@project.iam.gserviceaccount.com"
```

Translates to these environment variables:

```bash theme={"system"}
DESTINATION__BIGQUERY__CREDENTIALS__PROJECT_ID="my_project"
DESTINATION__BIGQUERY__CREDENTIALS__PRIVATE_KEY="my_key"
DESTINATION__BIGQUERY__CREDENTIALS__CLIENT_EMAIL="my_email@project.iam.gserviceaccount.com"
```

<Info>
  **Quick rule:** Replace every `.` with `__` and capitalize everything. `sources.pipedrive.pipedrive_api_key` becomes `SOURCES__PIPEDRIVE__PIPEDRIVE_API_KEY`.&#x20;
</Info>

***

## Setting Environment Variables

#### On Paradime

In Paradime, navigate to your [environment variable settings](/products/settings/environment-variables/index) in your workspace and set the key-value pair for your **Code IDE** and **Bolt** sections.

#### On Linux / macOS

Export variables directly in your terminal session:

```bash theme={"system"}
export DESTINATION__BIGQUERY__CREDENTIALS__PROJECT_ID="my_project"
export DESTINATION__BIGQUERY__CREDENTIALS__PRIVATE_KEY="my_key"
export DESTINATION__BIGQUERY__CREDENTIALS__CLIENT_EMAIL="my_email@project.iam.gserviceaccount.com"
```

#### On Windows (Command Prompt)

```cmd theme={"system"}
set DESTINATION__BIGQUERY__CREDENTIALS__PROJECT_ID=my_project
```

#### In a `.env` File (Local Development)

For local development, create a `.env` file in your project root and use `python-dotenv` to load it automatically:

```bash theme={"system"}
# .env
DESTINATION__BIGQUERY__CREDENTIALS__PROJECT_ID="my_project"
DESTINATION__BIGQUERY__CREDENTIALS__PRIVATE_KEY="my_key"
DESTINATION__BIGQUERY__CREDENTIALS__CLIENT_EMAIL="my_email@project.iam.gserviceaccount.com"
```

Then load it in your pipeline script:

```python theme={"system"}
from dotenv import load_dotenv
load_dotenv()

import dlt
# Your pipeline code here
```

<Warning>
  **Never commit your `.env` file to version control.** Add it to your `.gitignore` to keep secrets out of your repository.
</Warning>

***

### Examples

#### Example 1: Source Credentials (Pipedrive)

The equivalent `secrets.toml` entry:

```toml theme={"system"}
[sources.pipedrive]
pipedrive_api_key = "abc123"
```

As an environment variable:

```bash theme={"system"}
export SOURCES__PIPEDRIVE__PIPEDRIVE_API_KEY="abc123"
```

***

#### Example 2: Destination Credentials (BigQuery)

The equivalent `secrets.toml` entry:

```toml theme={"system"}
[destination.bigquery.credentials]
project_id = "my_project"
private_key = "-----BEGIN RSA PRIVATE KEY-----\n..."
client_email = "service@project.iam.gserviceaccount.com"
```

As environment variables:

```bash theme={"system"}
export DESTINATION__BIGQUERY__CREDENTIALS__PROJECT_ID="my_project"
export DESTINATION__BIGQUERY__CREDENTIALS__PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----\n..."
export DESTINATION__BIGQUERY__CREDENTIALS__CLIENT_EMAIL="service@project.iam.gserviceaccount.com"
```

***

#### Example 3: Mixing Environment Variables with Existing Variables

If you already have credentials stored under different environment variable names (for example, from another tool), you can map them to the dlt naming convention in your pipeline script:

```python theme={"system"}
import dlt
import os

# Map existing environment variables to dlt's expected naming convention
os.environ["DESTINATION__CREDENTIALS__CLIENT_EMAIL"] = os.environ.get("BIGQUERY_CLIENT_EMAIL")
os.environ["DESTINATION__CREDENTIALS__PRIVATE_KEY"] = os.environ.get("BIGQUERY_PRIVATE_KEY")
os.environ["DESTINATION__CREDENTIALS__PROJECT_ID"] = os.environ.get("BIGQUERY_PROJECT_ID")

# For source credentials
dlt.secrets["sources.credentials.client_email"] = os.environ.get("SHEETS_CLIENT_EMAIL")
dlt.secrets["sources.credentials.private_key"] = os.environ.get("SHEETS_PRIVATE_KEY")
dlt.secrets["sources.credentials.project_id"] = os.environ.get("SHEETS_PROJECT_ID")
```

***

#### Example 4: Filesystem Source (AWS S3)

The equivalent `secrets.toml` entry:

```toml theme={"system"}
[sources.filesystem.credentials]
aws_access_key_id = "AKIAIOSFODNN7EXAMPLE"
aws_secret_access_key = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
```

As environment variables:

```bash theme={"system"}
export SOURCES__FILESYSTEM__CREDENTIALS__AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
export SOURCES__FILESYSTEM__CREDENTIALS__AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
```

***

### How dlt Searches for a Value

When a configuration value is missing, dlt logs the exact lookup path it tried. This makes it easy to diagnose issues. For example, if a `password` field is missing for a Postgres destination, dlt will report something like:

```
In Environment Variables key DESTINATION__POSTGRES__CREDENTIALS__PASSWORD was not found.
In Environment Variables key DESTINATION__CREDENTIALS__PASSWORD was not found.
In Environment Variables key CREDENTIALS__PASSWORD was not found.
In secrets.toml key destination.postgres.credentials.password was not found.
```

Use these messages to confirm the exact environment variable name your pipeline expects.

***

### Best Practices

* **Use environment variables in CI/CD and production** to avoid storing secrets in files that could be accidentally committed to version control.
* **Rely on dlt's error messages** when debugging missing credentials — they show the exact keys and providers that were checked.
* **Don't mix providers unnecessarily** — define each credential in one place to avoid confusion over which value dlt is picking up.


## Related topics

- [Using dlt in Paradime](/guides/using-dlt-in-paradime/index.md)
- [dlt Data Pipeline - Google Sheets to Snowflake](/guides/using-dlt-in-paradime/python-data-pipeline-using-dlthub-google-sheets-to-snowflake.md)
- [Reuse dbt™ Connection Credentials in dlt Pipelines - BigQuery](/guides/using-dlt-in-paradime/reuse-dbt-tm-connection-credentials-in-dlt-pipelines-bigquery.md)
- [Fivetran Connector Sync](/products/bolt/creating-schedules/command-settings/fivetran-connector-sync.md)
- [Environment Variables](/products/settings/environment-variables/index.md)