Defining Your Sources in sources.yml

Working with Tags in Your dbt™ Project

Tags in dbt™ provide a flexible way to categorize and manage models, allowing teams to organize resources, optimize execution workflows, and selectively run models. By applying tags, users can efficiently filter and execute transformations within their dbt projects.

---

Why Use Tags?

Using tags helps teams streamline model selection and execution while improving project organization. Tags are particularly useful for:

• Grouping related models – Helps organize models into logical categories.

• Controlling execution workflows – Selectively run specific models or exclude others.

• Improving maintainability – Easily adjust execution without modifying code.

• Supporting data classification – Differentiate datasets (e.g., PII, financial data).

• Simplifying CI/CD automation – Automate dbt runs by tagging critical models.

---

Defining Tags in dbt™

Tags can be applied in multiple ways within a dbt project.

Applying Tags in Model Configurations

Tags can be set directly within a model's SQL file using the config() function.

{{ config(
    tags=["finance", "daily_refresh"]
) }}

SELECT ...

Applying Tags in dbt_project.yml

Tags can be set at the directory level in dbt_project.yml, applying them to all models within that directory.

models:
  my_project:
    +tags: "base_tag"  # Applies to all models in the project

    staging:
      +tags: ["staging", "raw_data"]  # Applies to all models in "staging/"
    
    marts:
      +tags: ["mart", "business_logic"]  # Applies to all models in "marts/"

---

Tag Inheritance and Resource Application

Tags inherit from parent folders unless explicitly overridden, simplifying large-scale tag management.

How Tag Inheritance Works

Tags applied at the folder level automatically apply to all models inside that folder.

models:
  staging:  
    customers.sql:
      tags: ["staging", "raw_data"]  # customers.sql inherits ["staging", "raw_data"] from the staging/ folder.
    orders.sql:
      tags: ["staging", "raw_data"]  # orders.sql inherits ["staging", "raw_data"] from the staging/ folder.

  marts:  
    dim_customer.sql:
      tags: ["mart", "business_logic"]  # dim_customers.sql inherits ["mart", "business_logic"] from the mmarts/ folder.

If a model has its own tag defined, it adds to the inherited tags rather than replacing them.

---

Applying Tags to Other Resource Types

Tags can also be applied to snapshots, seeds, and sources to improve organization and execution flexibility.

Tagging Snapshots

Snapshots track historical data. Apply tags to snapshots in dbt_project.yml for better resource selection.

snapshots:
  my_project:
    +tags: ["historical_data"]

Tagging Seeds

For preloaded datasets, assign tags to seeds to categorize static data.

seeds:
  my_project:
    +tags: ["seed_data"]

Tagging Sources

Apply tags to external data sources within sources.yml for better organization.

sources:
  - name: external_source
    tags: ['external']

This allows you to filter and execute specific sources using dbt commands like:

dbt run --select tag:external

---

Using Tags in dbt Commands

Once tags are defined, you can use them to execute specific resources.

Basic Selection

Run only models, seeds, or snapshots with a specific tag.

dbt run --select tag:daily_refresh
dbt seed --select tag:marketing
dbt snapshot --select tag:weekly

Advanced Selection

Combine multiple tags, exclude tags, or mix selection criteria.

# Run models with either "daily" or "critical" tags
dbt run --select tag:daily tag:critical

# Exclude models with the "deprecated" tag
dbt run --select tag:daily --exclude tag:deprecated

# Run all models tagged "finance" AND in the "staging" folder
dbt run --select tag:finance, staging

---

Best Practices for Using Tags

To ensure an effective tagging strategy, follow these best practices:

1. Use Consistent Naming Conventions

Standardized naming improves clarity and prevents confusion.

# Good
+tags: ["daily_refresh", "finance_data"]

# Avoid inconsistent casing or spacing
+tags: ["Daily", "Finance", "financial-data"]

2. Document Your Tagging Strategy

Clearly define tag meanings in your project's documentation.

# Tag Definitions
- `daily_refresh`: Models refreshed daily.
- `finance_data`: Contains financial-related tables.
- `pii`: Includes personally identifiable information.

3. Use Granular Tags

Avoid broad, generic tags. Instead, use precise labels for better control.

# Good
+tags: ["customer_metrics", "daily_refresh"]

# Too broad
+tags: ["metrics", "regular"]

4. Tag Models by Layer

Use tags to represent data modeling layers in your project.

models:
  staging:
    +tags: ["bronze_layer"]
  intermediate:
    +tags: ["silver_layer"]
  marts:
    +tags: ["gold_layer"]

---

Common Use Cases for Tags

Refresh Scheduling

Define tags based on refresh frequency for better execution control.

models:
  my_project:
    hourly:
      +tags: ["hourly_refresh"]
    daily:
      +tags: ["daily_refresh"]

Data Classification

Differentiate datasets based on sensitivity or access level.

models:
  my_project:
    +tags: ["contains_pii"]
    public:
      +tags: ["public_data"]

Testing Strategy

Prioritize critical models in testing workflows.

models:
  my_project:
    critical:
      +tags: ["critical_path", "requires_alert"]

---

Troubleshooting Tag Issues

If dbt isn't selecting resources correctly based on tags, consider these troubleshooting steps:

1. Tag Inheritance Issues

   1. Verify parent folder configurations in dbt_project.yml.

   2. Use dbt ls to confirm applied tags.

2. Selection Syntax Errors

   1. Ensure tag names match exactly (case-sensitive