Skip to content

feat(ci): report backend test fails#109543

Merged
joshuarli merged 26 commits intomasterfrom
feat-ci-report-backend-test-fails
Feb 28, 2026
Merged

feat(ci): report backend test fails#109543
joshuarli merged 26 commits intomasterfrom
feat-ci-report-backend-test-fails

Conversation

@joshuarli
Copy link
Copy Markdown
Member

@joshuarli joshuarli commented Feb 27, 2026 •
edited
Loading

Adds a PR comment summarising backend test failures across all shards as they happen.

Each shard's job runs a github-script step immediately after its tests finish. The step reads .artifacts/pytest.json directly from the runner filesystem (no artifact upload/download needed) and appends any new failures to the PR comment. The comment body itself is the shared accumulator — extractNodeids() parses existing <code> tags to skip already-reported tests, so re-runs and concurrent shards are idempotent.

Each failure links to its shard's job log via listJobsForWorkflowRun.

The step runs with continue-on-error: true so a GitHub API hiccup never fails CI.

@github-actions github-actions bot added Scope: Frontend Automatically applied to PRs that change frontend components Scope: Backend Automatically applied to PRs that change backend components labels Feb 27, 2026
github-advanced-security[bot]
github-advanced-security bot found potential problems Feb 27, 2026
View reviewed changes
@getsentry getsentry deleted a comment from github-actions bot Feb 27, 2026
@getsentry getsentry deleted a comment from github-actions bot Feb 27, 2026
@joshuarli joshuarli requested review from a team and rbro112 February 27, 2026 17:29
@getsentry getsentry deleted a comment from github-actions bot Feb 27, 2026
@joshuarli @claude
ref(ci): Remove delete-on-clean-rerun from backend failure comment
199f584 
Simplify the report job — if tests pass, the previous failure comment
is left as-is rather than being deleted. Also defers the listComments
API call until we know there are failures to report.

Co-Authored-By: Claude <[email protected]>
@joshuarli joshuarli marked this pull request as ready for review February 27, 2026 17:38
@joshuarli joshuarli requested a review from a team as a code owner February 27, 2026 17:38
sentry[bot]
sentry bot reviewed Feb 27, 2026
View reviewed changes
@joshuarli joshuarli requested review from a team as code owners February 27, 2026 17:42
@joshuarli joshuarli removed request for a team February 27, 2026 22:55
@joshuarli @claude
ci: Create per-commit failure comments with SHA in header
68dbe64 
Each push now gets its own failure comment instead of appending to
a single comment. The comment header shows the short SHA linking to
the commit, making it clear which push the failures belong to.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
sentry[bot]
sentry bot reviewed Feb 27, 2026
View reviewed changes
@joshuarli @claude
ci: Modernize failure reporting script for Node 24
8cb182f 
- flatMap/filter/map chains instead of imperative loops
- Data-driven JOB_MATCHERS table with named capture groups
- Iterator.prototype.map() on matchAll (no intermediate spread)
- Extracted truncateBody helper (was duplicated)
- Destructured context.repo once instead of repeating everywhere

Co-Authored-By: Claude Opus 4.6 <[email protected]>
cursor[bot]
cursor bot reviewed Feb 27, 2026
View reviewed changes
.github/workflows/scripts/report-backend-test-failures.js

const {owner, repo} = context.repo;
const prNumber = context.payload.pull_request.number;
const {sha} = context;
Copy link
Copy Markdown
Contributor

@cursor cursor bot Feb 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uses merge commit SHA instead of PR head SHA

Medium Severity

context.sha in a pull_request event is the temporary merge commit SHA, not the actual PR head commit. The comment will display a SHA developers don't recognize and link to a commit URL that likely 404s. Other scripts in this directory (e.g., getsentry-dispatch.js) correctly use context.payload.pull_request.head.sha for the PR commit. The commitMarker still changes per push so comment-tracking works, but the user-visible SHA and link are wrong.

Additional Locations (1)

Fix in Cursor Fix in Web

sentry[bot]
sentry bot reviewed Feb 27, 2026
View reviewed changes
cursor[bot]
cursor bot reviewed Feb 27, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 28, 2026 •
edited
Loading

Backend Test Failures

Failures on 1a6d74c in this run:

tests/sentry/replays/endpoints/test_organization_replay_index.py::OrganizationReplayIndexTest::test_get_replays_filter_clicks_unsupported_operators — log 
tests/sentry/replays/endpoints/test_organization_replay_index.py:1542: in test_get_replays_filter_clicks_unsupported_operators
    assert response.status_code == 400, query
E   AssertionError: click.selector:"[aria-label~=button]"
E   assert 500 == 400
E    +  where 500 = <Response status_code=500, "application/json">.status_code
tests/sentry/replays/endpoints/test_project_replay_clicks_index.py::OrganizationReplayDetailsTest::test_get_replays_filter_clicks_not_selector — log 
tests/sentry/replays/endpoints/test_project_replay_clicks_index.py:214: in test_get_replays_filter_clicks_not_selector
    assert response.status_code == 200
E   assert 500 == 200
E    +  where 500 = <Response status_code=500, "application/json">.status_code
tests/sentry/replays/endpoints/test_organization_replay_index.py::OrganizationReplayIndexTest::test_get_replays_filter_clicks_nested_selector — log 
tests/sentry/replays/endpoints/test_organization_replay_index.py:1478: in test_get_replays_filter_clicks_nested_selector
    assert response.status_code == 400
E   assert 500 == 400
E    +  where 500 = <Response status_code=500, "application/json">.status_code
tests/sentry/replays/endpoints/test_organization_replay_index.py::OrganizationReplayIndexTest::test_get_replays_filter_clicks_unsupported_selector — log 
tests/sentry/replays/endpoints/test_organization_replay_index.py:1507: in test_get_replays_filter_clicks_unsupported_selector
    assert response.status_code == 400, query
E   AssertionError: click.selector:div:is(2)
E   assert 500 == 400
E    +  where 500 = <Response status_code=500, "application/json">.status_code
tests/sentry/replays/endpoints/test_organization_replay_index.py::OrganizationReplayIndexTest::test_get_replays_filter_clicks — log 
tests/sentry/replays/endpoints/test_organization_replay_index.py:1300: in test_get_replays_filter_clicks
    assert response.status_code == 200, query
E   AssertionError: click.selector:div#myid
E   assert 500 == 200
E    +  where 500 = <Response status_code=500, "application/json">.status_code
tests/sentry/replays/endpoints/test_project_replay_clicks_index.py::OrganizationReplayDetailsTest::test_get_replays_filter_clicks — log 
tests/sentry/replays/endpoints/test_project_replay_clicks_index.py:146: in test_get_replays_filter_clicks
    assert response.status_code == 200, query
E   AssertionError: click.selector:div
E   assert 500 == 200
E    +  where 500 = <Response status_code=500, "application/json">.status_code
tests/sentry/replays/endpoints/test_organization_replay_index.py::OrganizationReplayIndexTest::test_get_replays_filter_clicks_unsupported_attribute_selector — log 
tests/sentry/replays/endpoints/test_organization_replay_index.py:1522: in test_get_replays_filter_clicks_unsupported_attribute_selector
    assert response.status_code == 400, query
E   AssertionError: click.selector:div[xyz=test]
E   assert 500 == 400
E    +  where 500 = <Response status_code=500, "application/json">.status_code

@joshuarli
Revert "test"
2553a5a 
This reverts commit e09cf29.
@joshuarli joshuarli merged commit bf7f90e into master Feb 28, 2026
75 checks passed
@joshuarli joshuarli deleted the feat-ci-report-backend-test-fails branch February 28, 2026 00:57
@joshuarli joshuarli restored the feat-ci-report-backend-test-fails branch February 28, 2026 01:05
@joshuarli joshuarli deleted the feat-ci-report-backend-test-fails branch February 28, 2026 01:08
jan-auer added a commit that referenced this pull request Mar 2, 2026
@jan-auer
Merge remote-tracking branch 'origin/master' into ref/objectstore-acc…
e3f9052 
…ept-encoding

* origin/master: (63 commits)
  fix(api): Add missing cursor query parameter to paginated endpoint OpenAPI schemas (#109642)
  docs(sentry-apps): Add sentryAppId to sentry-app-installations API schema (#109628)
  feat(occurrences on eap): Implement double reads from EAP in organization events trace API endpoint (#109391)
  feat(occurrences on eap): Implement double reads from EAP for reprocessing2 flow (#109345)
  feat(ci): report backend test fails (#109543)
  feat(seer): Add signed viewer context header to Seer API requests (#109626)
  devenv: cleanup devenv-managed uv (#109617)
  feat(seer): Iterate on the instructions at the top of seer settings pages (#109586)
  ref(seer): Add typed wrappers for remaining Seer API callsites (#109607)
  feat(preprod): Make snapshots endpoint org scoped (#109575)
  chore: capture exception (#109620)
  fix(formatting): run ruff format (#109618)
  feat(preprod): Create admin gated recompare snapshots endpoint (#109546)
  feat(cells): expand locality/cell distinction (#109538)
  feat(cells): add db migration for synapse (#109615)
  feat(preprod): Add public install-details endpoint and shared utilities (#109583)
  fix(tests): Fix flaky test_cross_trace_query_with_spans_and_logs (#109572)
  fix(grouping): Resolve mypy possibly-undefined errors in grouphash caching (#109602)
  fix(dashboards): Default axisRange to auto for existing widgets in builder (#109598)
  fix(billing): Fix category display names in pending changes (#109612)
  ...
@github-actions github-actions bot locked and limited conversation to collaborators Mar 15, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@sentry sentry[bot] sentry[bot] left review comments

@cursor cursor[bot] cursor[bot] left review comments

@scttcper scttcper scttcper approved these changes

@rbro112 rbro112 Awaiting requested review from rbro112

Assignees

No one assigned

Labels

Scope: Backend Automatically applied to PRs that change backend components Scope: Frontend Automatically applied to PRs that change frontend components

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@joshuarli @scttcper