Skip to content

test(dagster): cover backlog cap edge cases - #504

Merged
hugocorreia90 merged 1 commit into
rocky-data:mainfrom
snoopuppy582:test/sensor-backlog-cap
May 19, 2026
Merged

hugocorreia90 merged 1 commit into
rocky-data:mainfrom
snoopuppy582:test/sensor-backlog-cap

Conversation

@snoopuppy582

Copy link
Copy Markdown
Contributor

Summary

Adds focused Dagster sensor coverage for the backlog-cap edge cases requested in #479.

Subproject(s) touched

  • engine/ (Rust CLI / crates)
  • integrations/dagster/ (Python package)
  • editors/vscode/ (TypeScript extension)
  • examples/playground/ (sample project / benchmarks)
  • schemas/ (canonical CLI JSON schema)
  • Cross-project (CI, scripts, root docs)

Changes

  • Added a tag-aware mock Dagster instance helper for backlog-cap tests.
  • Covered partial suppression when one source is at cap and another can still emit.
  • Covered a later sync emitting after the in-flight count drops.
  • Covered per-group cursor advancement when one group is suppressed and another emits.

Closes #479

Test Plan

  • cd integrations/dagster && uv run pytest tests/test_sensor.py -q
  • cd integrations/dagster && uv run ruff check .
  • cd integrations/dagster && uv run ruff format --check .
  • git diff --check

Checklist

  • Commit messages follow conventional commits, scoped by subproject when relevant (e.g. feat(engine/rocky-databricks): ..., fix(dagster): ...)
  • No Co-Authored-By trailers
  • If this is a CLI JSON schema or DSL syntax change, all consumers (integrations/dagster/, editors/vscode/) are updated in this same PR

Engine (engine/)

  • cargo test --all-targets passes
  • cargo clippy --all-targets -- -D warnings is clean
  • cargo fmt --check passes

Dagster (integrations/dagster/)

  • uv run pytest passes
  • uv run ruff check is clean
  • uv run ruff format --check passes

VS Code (editors/vscode/)

  • npm run compile succeeds
  • npm run test:unit passes (electron tests run in CI)
  • npm run lint is clean

@hugocorreia90 hugocorreia90 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @snoopuppy582 ! ❤️

@hugocorreia90
hugocorreia90 merged commit fbde759 into rocky-data:main May 19, 2026
hugocorreia90 added a commit that referenced this pull request May 19, 2026
…1.23.0 (#595)

Bumps the engine to 1.39.0, dagster-rocky to 1.37.0, and the VS Code extension to 1.23.0. Includes Cargo.lock + uv.lock + package-lock.json refreshes.

Engine 1.39.0 ships three small but high-leverage changes:
- rocky import-dbt now imports dbt unit tests from manifest.unit_tests as Rocky [[test]] sidecar blocks (#594).
- rocky import-dbt accepts data_tests: as an alias for tests: in dbt YAML, restoring column-test conversion for dbt 1.7+ projects (#593).
- rocky-snowflake classifies statement spans for consistent OTLP attributes across adapters (#540).
- rocky-fivetran fetch_envelope tolerates connectors with no schema config instead of aborting the whole build (#592).

Dagster 1.37.0 and VS Code 1.23.0 are codegen-companion releases picking up the unit_tests_found/converted/skipped counters and the two new ImportWarningCategory variants on ImportDbtOutput. Dagster also gains a test-only backlog-cap edge-case suite (#504).

See engine/CHANGELOG.md, integrations/dagster/CHANGELOG.md, and editors/vscode/CHANGELOG.md for the per-artifact entries.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add pytest coverage for sensor cursor advancement and backlog-cap edge cases

2 participants