Snapshot Source Data Freshness

This template creates a schedule to monitor source data freshness using dbt source freshness, ensuring your data meets defined SLAs by regularly checking when source tables were last updated and alerting you of any staleness issues. Use this template when you need to monitor critical data sources - for example, ensuring your daily sales data is loading properly or verifying that customer data is being updated within expected timeframes.

Key Benefits

Ensures critical data sources meet defined freshness SLAs
Quickly identifies issues with upstream data loading jobs
Prevents analytics teams from using stale data
Tracks and reports on source data freshness for downstream processes

Prerequisites

Scheduler Environment is connected to your data warehouse provider.
Source freshness configurations in your sources.yml file

Default Configuration

Schedule Settings

Setting	Value	Explanation
Schedule Type	`Standard`	Ensures consistent monitoring of source freshness in production environment
Schedule Name	`source freshness check`	Descriptive name that indicates purpose
Git Branch	`main`	Uses your default production branch to ensure you’re checking against the latest source configurations

Command Settings

The template uses a single command to monitor your data sources:

dbt source freshness: Validates the freshness of all configured source tables by:
- Checking the timestamp in each table’s loaded_at_field
- Comparing this timestamp against your defined warning and error thresholds
- Reporting success or failure based on these comparisons

For additional source freshness configurations, see Source Freshness documentation.

Trigger Type

Type: Scheduled Run (Cron)
Cron Schedule: 0 */2 * * * (Every 2 hours, starting at minute 0 to balance frequent data updates and reasonable resource usage)

For custom Trigger configurations, see Trigger Types documentation.

Notification Settings

Email Alerts:
- Success: Confirms all sources are within freshness thresholds
- Failure: Alerts when sources exceed error thresholds
- SLA Breach: Alerts when the freshness check itself exceeds 120 minutes

For custom notification configurations, see Notification Settings documentation.

When to Customize

Tailor this template to your specific needs:

Adjust trigger type based on SLA requirements:
- Hourly checks for critical sources (0 * * * *)
- Daily checks for stable sources (0 0 * * *)
- Weekly checks for historical analysis (0 0 * * 0)
Add notification destinations (Slack, MS Teams) for team collaboration

​Default Configuration

​Schedule Settings

​Command Settings

​Trigger Type

​Notification Settings

​When to Customize