Repository navigation
fix(dagster-dbt): use the asset's own key for last-update freshness checks - #34432
Open
barlowa124 wants to merge 1 commit into
Open
barlowa124 wants to merge 1 commit into
barlowa124 wants to merge 1 commit into
Conversation
…hecks build_freshness_checks_from_dbt_assets passed the translator left over from the last-processed AssetsDefinition into build_last_update_freshness_checks, so freshness check keys were computed with the wrong DagsterDbtTranslator whenever multiple @dbt_assets used different translators (e.g. custom get_asset_key prefixes). Pass the asset key directly, matching the time-partitioned branch. Fixes dagster-io#34394
Contributor
[Medium impact] This PR appears safe to merge. SummaryFixes last-update freshness checks to use each asset’s own key instead of the last translator encountered.
Reviews (1) · Last reviewed commit: "fix(dagster-dbt): use the asset's own ke..." · Reviewed by Greptile |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary & Motivation
build_freshness_checks_from_dbt_assetscomputed last-update freshness check keys with whicheverDagsterDbtTranslatorwas left over from the loop over the inputdbt_assets. The translator that belongs to theAssetsDefinitionproducing each asset was not used. When multiple@dbt_assetsdefinitions use different translators (for example a customget_asset_keythat adds a prefix), freshness checks were attached to wrongly keyed assets. This passes the asset's ownAssetKeydirectly, matching the time-partitioned branch. Fixes #34394.Test Plan
test_last_update_multiple_assets_defs_custom_translator. The test builds two@dbt_assetsdefinitions that share one manifest and gives the second a prefixing translator. It asserts each freshness check targets the correctly translated key. The test fails before this change and passes after.pytest python_modules/libraries/dagster-dbt/dagster_dbt_tests/core/test_freshness_checks.pyfrompython_modules/libraries/dagster-dbt: 7 passed.ruff check --fixandruff formatclean on the touched files.