Description
Ability to test the incremental behavior of models using an incremental strategy that requires partitioning (insert_overwrite or delete+insert). See #8422.
If I have an incremental model that uses a strategy that uses "partitioning":
{{
config(
materialized='incremental',
unique_key='date_day',
incremental_strategy='insert_overwrite',
...
)
}}
select
date_trunc('day', event_at) as date_day,
count(distinct user_id) as daily_active_users
from {{ ref('events') }}
{% if is_incremental() %}
-- this filter will only be applied on an incremental run
-- (uses >= to include records arriving later on the same day as the last run of this model)
where date_day >= (select max(date_day) from {{ this }})
{% endif %}
group by 1I should be able to add a unit test to this model and have it work as expected.
unit_tests:
- name: my_favorite_unit_test
model: my_incremental_model
given:
- input: ref('events')
format: csv
fixture: my_events_input_fixture
- input: this
format: csv
fixture: my_incremental_model_this_fixture
expect:
format: csv
fixture: my_incremental_model_output_fixture
overrides:
macros:
is_incremental: true
Acceptance Criteria
Unit testing of incremental models with insert_overwrite or delete+insert strategy.
Impact to Other Teams
None
Will backports be required?
No
Context
Description
Ability to test the incremental behavior of models using an incremental strategy that requires partitioning (
insert_overwriteordelete+insert). See #8422.If I have an incremental model that uses a strategy that uses "partitioning":
{{ config( materialized='incremental', unique_key='date_day', incremental_strategy='insert_overwrite', ... ) }} select date_trunc('day', event_at) as date_day, count(distinct user_id) as daily_active_users from {{ ref('events') }} {% if is_incremental() %} -- this filter will only be applied on an incremental run -- (uses >= to include records arriving later on the same day as the last run of this model) where date_day >= (select max(date_day) from {{ this }}) {% endif %} group by 1I should be able to add a unit test to this model and have it work as expected.
Acceptance Criteria
Unit testing of incremental models with
insert_overwriteordelete+insertstrategy.Impact to Other Teams
None
Will backports be required?
No
Context