# dbt™️ Source checks

## `check-source-columns-have-desc`

{% hint style="info" %}
**What it does**

Ensures that the source has columns with descriptions in the properties file (usually `schema.yml`).

**When to use it**

You want to make sure that all specified columns in the properties files (usually `schema.yml`) have some description. **This hook does not validate if all database columns are also present in a properties file.**
{% endhint %}

**Example**

```yaml
repos:
  - repo: https://github.com/dbt-checkpoint/dbt-checkpoint
    rev: v1.0.0
    hooks:
      - id: check-source-columns-have-desc
```

**Requirements**

| Model exists in `manifest.json` *1* | Model exists in `catalog.json` *2* |
| :---------------------------------: | :--------------------------------: |
|                  ❌                  |            ❌ Not needed            |

1 It means that you need to run `dbt parse` before run this hook (dbt >= 1.5).\
2 It means that you need to run `dbt docs generate` before run this hook.

**How it works**

* Hook takes all changed `yml`.
* All sources from yml file are parsed.
* If any column in the source does not contain a description, the hook fails.

***

## `check-source-has-all-columns`

{% hint style="info" %}
**What it does**

Ensures that all columns in the database are also specified in the properties file. (usually `schema.yml`).

**When to use it**

You want to make sure that you have all the database columns listed in the properties file, or that your properties file no longer contains deleted columns.
{% endhint %}

**Arguments**

`--manifest`: location of `manifest.json` file. Usually `target/manifest.json`. This file contains a full representation of dbt project. **Default: `target/manifest.json`**\
`--catalog`: location of `catalog.json` file. Usually `target/catalog.json`. dbt uses this file to render information like column types and table statistics into the docs site. In dbt-checkpoint is used for column operations. **Default: `target/catalog.json`**

**Example**

```yaml
repos:
  - repo: https://github.com/dbt-checkpoint/dbt-checkpoint
    rev: v1.0.0
    hooks:
      - id: check-source-has-all-columns
```

**Requirements**

| Model exists in `manifest.json` *1* | Model exists in `catalog.json` *2* |
| :---------------------------------: | :--------------------------------: |
|                 ❌ No                |                ✅ Yes               |

1 It means that you need to run `dbt parse` before run this hook (dbt >= 1.5).\
2 It means that you need to run `dbt docs generate` before run this hook.

**How it works**

* Hook takes all changed `yml`.
* All sources from yml file are parsed.
* The catalog is scanned for a model.
* If there is any discrepancy between found yml sources and catalog sources, the hook fails.

**Known limitations**

If you did not update the catalog and manifest results can be wrong.

***

## `check-source-table-has-description`

{% hint style="info" %}
**What it does**

Ensures that the source table has a description in the properties file (usually `schema.yml`).

**When to use it**

You want to make sure that all sources have a description.
{% endhint %}

**Example**

```yaml
repos:
  - repo: https://github.com/dbt-checkpoint/dbt-checkpoint
    rev: v1.0.0
    hooks:
      - id: check-source-table-has-description
```

**Requirements**

| Model exists in `manifest.json` *1* | Model exists in `catalog.json` *2* |
| :---------------------------------: | :--------------------------------: |
|             ❌ Not needed            |            ❌ Not needed            |

1 It means that you need to run `dbt parse` before run this hook (dbt >= 1.5).\
2 It means that you need to run `dbt docs generate` before run this hook.

**How it works**

* Hook takes all changed `yml`.
* All sources from yml file are parsed.
* If the source table does not have a description, the hook fails.

***

## `check-source-has-freshness`

{% hint style="info" %}
**What it does**

Ensures that the source has freshness options in the properties file (usually `schema.yml`).

**When to use it**

You want to make sure that all freshness is correctly set.
{% endhint %}

**Example**

```yaml
repos:
  - repo: https://github.com/dbt-checkpoint/dbt-checkpoint
    rev: v1.0.0
    hooks:
      - id: check-source-has-freshness
        args: ["--freshness", "error_after", "warn_after", "--"]
```

⚠️ do not forget to include `--` as the last argument. Otherwise `pre-commit` would not be able to separate a list of files with args.

**Requirements**

| Model exists in `manifest.json` *1* | Model exists in `catalog.json` *2* |
| :---------------------------------: | :--------------------------------: |
|             ❌ Not needed            |            ❌ Not needed            |

1 It means that you need to run `dbt parse` before run this hook (dbt >= 1.5).\
2 It means that you need to run `dbt docs generate` before run this hook.

**How it works**

* Hook takes all changed `yml`.
* All sources from yml file are parsed.
* If the source does not have freshness correctly set, the hook fails.

***

## `check-source-has-loader`

{% hint style="info" %}
**What it does**

Ensures that the source has a loader option in the properties file (usually `schema.yml`).

**When to use it**

You want to make sure that the source has loader specified.
{% endhint %}

**Example**

```yaml
repos:
  - repo: https://github.com/dbt-checkpoint/dbt-checkpoint
    rev: v1.0.0
    hooks:
      - id: check-source-has-loader
```

**Requirements**

| Model exists in `manifest.json` *1* | Model exists in `catalog.json` *2* |
| :---------------------------------: | :--------------------------------: |
|             ❌ Not needed            |            ❌ Not needed            |

1 It means that you need to run `dbt parse` before run this hook (dbt >= 1.5).\
2 It means that you need to run `dbt docs generate` before run this hook.

**How it works**

* Hook takes all changed `yml`.
* All sources from yml file are parsed.
* If the source does not have a loader set, the hook fails.

***

## `check-source-has-meta-keys`

{% hint style="info" %}
**What it does**

Ensures that the source has a list of valid meta keys. (usually `schema.yml`).

**When to use it**

If every source needs to have certain meta keys.
{% endhint %}

**Arguments**

`--meta-keys`: list of the required keys in the meta part of the model.

**Example**

```yaml
repos:
  - repo: https://github.com/dbt-checkpoint/dbt-checkpoint
    rev: v1.0.0
    hooks:
      - id: check-source-has-meta-keys
        args: ['--meta-keys', 'foo', 'bar', "--"]
```

⚠️ do not forget to include `--` as the last argument. Otherwise `pre-commit` would not be able to separate a list of files with args.

**Requirements**

|          Model exists in `manifest.json` *1*          | Model exists in `catalog.json` *2* |
| :---------------------------------------------------: | :--------------------------------: |
| ❌ Not needed since it also validates properties files |            ❌ Not needed            |

1 It means that you need to run `dbt parse` before run this hook (dbt >= 1.5).\
2 It means that you need to run `dbt docs generate` before run this hook.

**How it works**

* Hook takes all changed `yml`.
* All sources from yml file are parsed.
* If the source does not have the required meta keys set, the hook fails.

***

## `check-source-has-labels-keys`

{% hint style="info" %}
**What it does**

Ensures that the source has a list of valid labels keys. (usually `schema.yml`).

**When to use it**

If every source needs to have certain labels keys.
{% endhint %}

**Arguments**

`--labels-keys`: list of the required keys in the labels part of the model.

**Example**

```yaml
repos:
  - repo: https://github.com/dbt-checkpoint/dbt-checkpoint
    rev: v1.0.0
    hooks:
      - id: check-source-has-labels-keys
        args: ['--labels-keys', 'foo', 'bar', "--"]
```

⚠️ do not forget to include `--` as the last argument. Otherwise `pre-commit` would not be able to separate a list of files with args.

**Requirements**

|          Model exists in `manifest.json` *1*          | Model exists in `catalog.json` *2* |
| :---------------------------------------------------: | :--------------------------------: |
| ❌ Not needed since it also validates properties files |            ❌ Not needed            |

1 It means that you need to run `dbt parse` before run this hook (dbt >= 1.5).\
2 It means that you need to run `dbt docs generate` before run this hook.

**How it works**

* Hook takes all changed `yml`.
* All sources from yml file are parsed.
* If the source does not have the required labels keys set, the hook fails.

***

## `check-source-has-tests-by-name`

{% hint style="info" %}
**What it does**

Ensures that the source has a number of tests of a certain name (e.g. data, unique).

**When to use it**

You want to make sure that every source has certain tests.
{% endhint %}

**Arguments**

`--tests`: key-value pairs of test names. Key is the name of test and value is required minimal number of tests eg. --test unique=1 not\_null=2 (do not put spaces before or after the = sign).

**Example**

```yaml
repos:
  - repo: https://github.com/dbt-checkpoint/dbt-checkpoint
    rev: v1.0.0
    hooks:
      - id: check-source-has-tests-by-name
        args: ["--tests", "unique=1", "data=1", "--"]
```

⚠️ do not forget to include `--` as the last argument. Otherwise `pre-commit` would not be able to separate a list of files with args.

**Requirements**

| Model exists in `manifest.json` *1* | Model exists in `catalog.json` *2* |
| :---------------------------------: | :--------------------------------: |
|                ❌ Yes                |            ❌ Not needed            |

1 It means that you need to run `dbt parse` before run this hook (dbt >= 1.5).\
2 It means that you need to run `dbt docs generate` before run this hook.

**How it works**

* Hook takes all changed `yml`.
* All sources from yml file are parsed.
* If the source does not have the required test names, the hook fails.

***

## `check-source-has-tests-by-type`

{% hint style="info" %}
**What it does**

Ensures that the source has a number of tests of a certain type (data, schema).

**When to use it**

You want to make sure that every source has certain tests.
{% endhint %}

**Arguments**

`--tests`: key-value pairs of test types. Key is a type of test (data or schema) and value is required eg. --test data=1 schema=2 (do not put spaces before or after the = sign).

**Example**

```yaml
repos:
  - repo: https://github.com/dbt-checkpoint/dbt-checkpoint
    rev: v1.0.0
    hooks:
      - id: check-source-has-tests-by-type
        args: ["--tests", "schema=1", "data=1", "--"]
```

⚠️ do not forget to include `--` as the last argument. Otherwise `pre-commit` would not be able to separate a list of files with args.

**Requirements**

| Model exists in `manifest.json` *1* | Model exists in `catalog.json` *2* |
| :---------------------------------: | :--------------------------------: |
|                ❌ Yes                |            ❌ Not needed            |

1 It means that you need to run `dbt parse` before run this hook (dbt >= 1.5).\
2 It means that you need to run `dbt docs generate` before run this hook.

**How it works**

* Hook takes all changed `yml`.
* All sources from yml file are parsed.
* If the source does not have the required test types, the hook fails.

***

## `check-source-has-tests`

{% hint style="info" %}
**What it does**

Ensures that the source has a number of tests.

**When to use it**

You want to make sure that every source was tested.
{% endhint %}

**Arguments**

`--test-cnt`: Minimum number of tests required.

**Example**

```yaml
repos:
  - repo: https://github.com/dbt-checkpoint/dbt-checkpoint
    rev: v1.0.0
    hooks:
      - id: check-source-has-tests
        args: ["--test-cnt", "2", "--"]
```

⚠️ do not forget to include `--` as the last argument. Otherwise `pre-commit` would not be able to separate a list of files with args.

**Requirements**

| Model exists in `manifest.json` *1* | Model exists in `catalog.json` *2* |
| :---------------------------------: | :--------------------------------: |
|                ❌ Yes                |            ❌ Not needed            |

1 It means that you need to run `dbt parse` before run this hook (dbt >= 1.5).\
2 It means that you need to run `dbt docs generate` before run this hook.

**How it works**

* Hook takes all changed `yml`.
* All sources from yml file are parsed.
* If the source does not have the required test count, the hook fails.

***

## `check-source-has-tests-by-group`

{% hint style="info" %}
**What it does**

Ensures that the source has a number of tests from a group of tests.

**When to use it**

You want to make sure that every source has one (or more) of a group of eligible tests (e.g. a set of unique tests).
{% endhint %}

**Arguments**

`--tests`: list of test names. `--test_cnt`: number of tests required across test group.

**Example**

```yaml
repos:
  - repo: https://github.com/dbt-checkpoint/dbt-checkpoint
    rev: v1.0.0
    hooks:
      - id: check-source-has-tests-by-group
        args: ["--tests", "unique", "unique_where", "--test-cnt", "1", "--"]
```

⚠️ do not forget to include `--` as the last argument. Otherwise `pre-commit` would not be able to separate a list of files with args.

**Requirements**

| Model exists in `manifest.json` *1* | Model exists in `catalog.json` *2* |
| :---------------------------------: | :--------------------------------: |
|                ✅ Yes                |            ❌ Not needed            |

1 It means that you need to run `dbt parse` before run this hook (dbt >= 1.5).\
2 It means that you need to run `dbt docs generate` before run this hook.

**How it works**

* Hook takes all changed `SQL` files.
* The source name is obtained from the `SQL` file name.
* If any source does not have the number of required tests, the hook fails.

***

## `check-source-tags`

{% hint style="info" %}
**What it does**

Ensures that the source has only valid tags from the provided list.

**When to use it**

Make sure you did not typo in tags.
{% endhint %}

**Arguments**

`--tags`: A list of tags that sources can have.

**Example**

```yaml
repos:
  - repo: https://github.com/dbt-checkpoint/dbt-checkpoint
    rev: v1.0.0
    hooks:
      - id: check-source-tags
        args: ["--tags", "foo", "bar", "--"]
```

⚠️ do not forget to include `--` as the last argument. Otherwise `pre-commit` would not be able to separate a list of files with args.

**Requirements**

| Model exists in `manifest.json` *1* | Model exists in `catalog.json` *2* |
| :---------------------------------: | :--------------------------------: |
|             ❌ Not needed            |            ❌ Not needed            |

1 It means that you need to run `dbt parse` before run this hook (dbt >= 1.5).\
2 It means that you need to run `dbt docs generate` before run this hook.

**How it works**

* Hook takes all changed `yml`.
* All sources from yml file are parsed.
* If the source has different tags than specified, the hook fails.

***

## `check-source-childs`

{% hint style="info" %}
**What it does**

Ensures the source has a specific number (max/min) of childs.

**When to use it**

You want to find orphaned sources without any dependencies. Or you want to make sure that every source is used somewhere.
{% endhint %}

**Arguments**

`--manifest`: location of `manifest.json` file. Usually `target/manifest.json`. This file contains a full representation of dbt project. **Default: `target/manifest.json`**\
`--min-child-cnt`: Minimal number of child models. `--max-child-cnt`: Maximal number of child models.

**Example**

```yaml
repos:
  - repo: https://github.com/dbt-checkpoint/dbt-checkpoint
    rev: v1.0.0
    hooks:
      - id: check-source-childs
        args: ["--min-child-cnt", "2", "--"]
```

⚠️ do not forget to include `--` as the last argument. Otherwise `pre-commit` would not be able to separate a list of files with args.

**Requirements**

| Model exists in `manifest.json` *1* | Model exists in `catalog.json` *2* |
| :---------------------------------: | :--------------------------------: |
|                ✅ Yes                |            ❌ Not needed            |

*1* It means that you need to run `dbt parse` before run this hook (dbt >= 1.5).\
2 It means that you need to run `dbt docs generate` before run this hook.

**How it works**

* Hook takes all changed `yml`.
* All sources from yml file are parsed.
* The manifest is scanned for child models.
* If any source does not have a number of required childs, the hook fails.<br>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.paradime.io/app-help/documentation/integrations/code-ide/pre-commit/dbt-tm-checkpoint-hooks/dbt-tm-source-checks.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.