Event freshness anomalies
The elementary.event_freshness_anomalies
test monitors the freshness of event data over time, measuring the expected time it takes each event to load. This is calculated as the time between when the event actually occurs (the event timestamp
) and when it is loaded to the database (the update timestamp
).
This test complements the freshness_anomalies
test and is primarily intended for data that is updated in a continuous or streaming fashion.
How it works
The test can operate in two modes:
If only an
event_timestamp_column
is supplied:The test measures the difference between the current timestamp ("now") and the most recent event timestamp.
If both an
event_timestamp_column
and anupdate_timestamp_column
are provided:The test measures the difference between these two columns.
models:
- name: < model name >
tests:
- elementary.event_freshness_anomalies:
event_timestamp_column: < timestamp column > # Mandatory
update_timestamp_column: < timestamp column > # Optional
where_expression: < sql expression >
time_bucket: # Daily by default
period: < time period >
count: < number of periods >
Test configuration
tests:
— elementary.event_freshness_anomalies:
event_timestamp_column: column name
update_timestamp_column: column name
where_expression: sql expression
anomaly_sensitivity: int
detection_period:
period: [hour | day | week | month]
count: int
training_period:
period: [hour | day | week | month]
count: int
time_bucket:
period: [hour | day | week | month]
count: int
seasonality: day_of_week
detection_delay:
period: [hour | day | week | month]
count: int
ignore_small_changes:
spike_failure_percent_threshold: int
drop_failure_percent_threshold: int
anomaly_exclude_metrics: [SQL expression]
Last updated
Was this helpful?