Skip to content

feat(code-review): PR Review Dashboard for Seer code reviews#1

Open
vaind wants to merge 69 commits intomasterfrom
feat/pr-review-dashboard
Open

feat(code-review): PR Review Dashboard for Seer code reviews#1
vaind wants to merge 69 commits intomasterfrom
feat/pr-review-dashboard

Conversation

@vaind
Copy link
Copy Markdown
Owner

@vaind vaind commented Feb 17, 2026

Summary

Adds an observability dashboard for monitoring Seer's PR code review pipeline. Today, users have zero visibility into whether their PRs were reviewed, why reviews were skipped, or how effective Seer's reviews are — this dashboard surfaces some of this and opens up room for followup work

Linear project: PR Review Dashboard

What's included

Data model & instrumentation

  • New CodeReviewEvent model that records every webhook event as it flows through the review pipeline (received → preflight → enqueued → sent to Seer → started → completed/failed)
  • Instrumented all existing webhook handlers (pull_request, issue_comment, check_run, task) to create and update CodeReviewEvent records at each stage
  • Seer reports review completion via send_seer_webhook (async webhook — same pattern as autofix), dispatched to a new process_pr_review_completion Celery task
  • Periodic cleanup task to prune old events

API endpoints (gated behind organizations:pr-review-dashboard)

  • GET /api/0/organizations/{org}/code-review-prs/ — paginated PR list with filters (repo, PR state, date range)
  • GET /api/0/organizations/{org}/code-review-prs/{repo_id}/{pr_number}/ — PR detail with event timeline and summary stats
  • GET /api/0/organizations/{org}/code-review-stats/ — aggregated stats (totals, time series, top authors)

Frontend (/explore/pr-review/)

  • Stats overview: total PRs, reviews, comments, skipped count, top authors
  • Grouped bar chart: reviews/comments/skipped over time
  • Filterable PR list with repo selector, PR state filter, and date range
  • PR detail view with summary cards, event timeline with pipeline stages, and commit links

Backfill tooling - development only

  • bin/backfill-code-review-events script to populate historical data from seer_event_metadata JSON blobs

Test plan

  • 187 backend tests pass (pytest tests/sentry/seer/code_review/)
  • Frontend tests pass (CI=true pnpm test static/app/views/explore/prReview/)
  • Pre-commit hooks pass on all modified files
  • Verify dashboard loads in dev with feature flag enabled
  • End-to-end: trigger a PR review and verify the event appears in the dashboard

…eviews

Adds a full-stack observability dashboard for tracking GitHub PR code reviews
processed by Seer. This enables the ML/AI team to monitor the webhook-to-review
pipeline, identify failures, and inspect individual review results.

Backend:
- CodeReviewEvent model with status tracking and pipeline timestamps
- Event recorder for instrumenting the existing webhook pipeline
- Seer RPC callback for receiving review completion results
- Three API endpoints: list, detail (with Seer comments), and stats
- Periodic cleanup task for old events
- 44 backend tests

Frontend:
- Dashboard page with stats cards, filters, and paginated event list
- Detail view with metadata, pipeline timeline, and Seer review comments
- Gated behind `pr-review-dashboard` feature flag
- 8 frontend tests
@github-actions
Copy link
Copy Markdown

🚨 Warning: This pull request contains Frontend and Backend changes!

It's discouraged to make changes to Sentry's Frontend and Backend in a single pull request. The Frontend and Backend are not atomically deployed. If the changes are interdependent of each other, they must be separated into two pull requests and be made forward or backwards compatible, such that the Backend or Frontend can be safely deployed independently.

Have questions? Please ask in the #discuss-dev-infra channel.

vaind added 26 commits February 17, 2026 15:31
…ew_result

Remove the fallback org/repo/pr lookup — Seer always has the delivery ID
since it's passed through the entire webhook pipeline.
- Extract shared statusToTagType/formatStatus into utils.ts
- Extract _fetch_pr_comments helper in detail endpoint
- Replace status if/elif chain with SEER_STATUS_MAP dict
- Extract _parse_timestamp and _iso_or_none helpers
- Remove unused METRICS_PREFIX and self-import in task.py
- Clean up redundant import and docstring in check_run.py
- Remove ExploreBreadcrumb (requires traceItemDataset, not applicable)
- Replace Layout.Main fullWidth with width="full"
- Replace Tag type prop with variant (error→danger, default→muted)
- Replace CompactSelect triggerProps with trigger render prop
Reads CSV exported from Seer events query and creates CodeReviewEvent
records in the Sentry database. Supports --dry-run mode and is
idempotent via synthetic github_delivery_id.
date_added reflects when the DB record was created, which is wrong
for backfilled records. Use Coalesce(trigger_at, date_added) for
ordering, filtering, and time series grouping so events display
their actual trigger time.
- Parse run_state_value JSON to extract real github_delivery_id
- Batch-fetch PR titles via gh GraphQL API (100 PRs per query)
- Support env vars (BACKFILL_CSV, BACKFILL_DRY_RUN) for getsentry exec
Replace event-first endpoints with PR-grouped endpoints that match
how users think about code reviews — by pull request, not individual
pipeline events.

Backend:
- /code-review-events/ → /code-review-prs/ (grouped by repo+pr_number)
- /code-review-events/:id/ → /code-review-prs/:repo_id/:pr_number/
- Remove DetailedCodeReviewEventSerializer (timeline now implicit)

Frontend:
- List view shows PRs with aggregated stats (event count, comments, last activity)
- Detail view shows PR metadata, events table, and comments
- Delete PrReviewTimeline component
- Update routing from :eventId to :repoId/:prNumber
Separate the PR column into distinct PR # and Title columns for
better scannability — number links to GitHub with an external icon,
title links to the internal detail view.
Rename model fields for clarity: github_delivery_id → trigger_id,
github_event_type → trigger_event_type, github_event_action →
trigger_event_action. These fields aren't GitHub-specific concepts at
the model level.

Also add pr_state field (open/closed/merged) to CodeReviewEvent, derived
from the webhook payload, and surface it in the PR list dashboard as a
new "PR Status" column.

Migration pending via getsentry.
Combine the initial creation and field rename migrations into one
migration that creates the table in its final state.
- Adapt to field renames: trigger_id, trigger_event_type/action
- Fetch PR state (open/closed/merged) via GraphQL alongside titles
- Extract github_delivery_id from Seer JSON for trigger_id
…tions

Use ScoreCard components in a responsive grid for stats, matching the
Stats & Usage page pattern. Add Grid wrapper with consistent gap, put
filters above metrics, and add unified prop to HeaderContent to match
Replays/Profiles explore page spacing.
Replace event-level stats with PR-focused metrics: Total PRs (with
skipped count as subtle trend), Reviews (with comments posted as trend),
and a stacked MiniBarChart showing daily reviewed/skipped/comments.
…osed)

The status filter now filters by PR state instead of review pipeline
status, which is more useful at the PR-list level.
The Total PRs card title now reflects the active status filter (e.g.
"Open PRs", "Merged PRs"). Both the PR list and stats endpoints now
filter by pr_state when a status is selected.
…y filter

Add a multi-select, searchable repository filter populated from repos
that have code review events. The stats endpoint now returns a
repositories list and both endpoints support multiple repositoryId
params.
Show relative timestamps (e.g. "3hr ago") with full date tooltip,
matching the Last Seen pattern used in the issues list.
Use PageFilterBar for connected filter bar matching Replays/Issues
conventions. Order: All Repositories | All Statuses | 14D. Add time
range filter (24H/7D/14D/30D/90D) that converts to a start param for
the API. Show "All Repositories" when no repos are selected.
Set default_per_page=25 to match issues list convention. Fix TS error
where repositoryIds[0] could be undefined. Replace purple400 (doesn't
exist in theme) with blue400 for the comments chart series.
Pass the cursor query param from the URL to the PR list API request so
clicking next/prev actually fetches the correct page. Enable count_hits
on the backend paginator to populate X-Hits header, and display a
"1-25 of N" caption next to the pagination buttons.
Merge the separate PR Status and Review Status columns into one Status
column showing the PR state tag on top and the review run status tag
below it.
Combine repository, PR number, title, and author into a single "Pull
Request" column. Make entire rows clickable (navigating to detail page)
with hover highlight, while preserving external link clicks via DOM
walk detection.
…axis

Replace MiniBarChart with full BarChart showing PRs+Skipped (stacked),
Reviews, and Comments as separate bar groups per day. Add distinct PR
count per time bucket to the stats API, support hourly intervals for
24h range, and show date-only x-axis labels with appropriate spacing
per time range.
Fix normalizeUrl import (moved from withDomainRequired to
url/normalizeUrl) which broke PR detail row clicks. Update tests
to match current table headers and add missing paginationCaption prop.
@vaind
Copy link
Copy Markdown
Owner Author

vaind commented Feb 19, 2026

@sentry review

@vaind
Copy link
Copy Markdown
Owner Author

vaind commented Feb 19, 2026

@sentry-ivan-dev review

fromisoformat can return a naive datetime when the input lacks
timezone info, triggering a Django RuntimeWarning. Apply the same
UTC fallback used elsewhere.
- Scope find_event_by_trigger_id query to (org, repo, trigger_id) to use
  the composite unique constraint instead of scanning by trigger_id alone
- Pass organization_id and repository_id through the task pipeline
- Move PR-level stats aggregation from Python loop to SQL count queries
- Scope Repository lookups by organization_id to prevent IDOR
- Add try/catch for parseStatsPeriod to handle invalid URL params
…nt, fix pr_state filter

- Replace string literals with CodeReviewEventStatus enum constants across
  all webhook handlers for type safety and grep-ability
- Simplify _enrich_groups by replacing correlated subquery with a single
  ordered query + Python dedup to pick latest event per PR
- Filter prState on latest event per PR group instead of all events, since
  pr_state is denormalized and only the latest event reflects current state
- Use Element.closest() instead of manual DOM walking for interactive
  element detection in row click handler
- Fix x-axis labels not rendering: XAxis default formatter returns '' for
  non-date category axes; override with identity formatter
- Fix bucket key mismatch for UTC+ users: generate date range using UTC
  arithmetic so keys match server-side TruncDay/TruncHour output
- Count each PR in its first-event bucket rather than every event bucket,
  so a PR appears exactly once in the time series
- Stack Skipped with Reviews (not PRs) to better represent outcomes
@vaind vaind marked this pull request as ready for review February 19, 2026 11:44
@vaind vaind force-pushed the feat/pr-review-dashboard branch from 41eb760 to 4282f97 Compare February 19, 2026 11:57
@vaind
Copy link
Copy Markdown
Owner Author

vaind commented Feb 19, 2026

@sentry review

@vaind vaind marked this pull request as draft February 19, 2026 12:45
@vaind vaind marked this pull request as ready for review February 19, 2026 12:45
@vaind
Copy link
Copy Markdown
Owner Author

vaind commented Feb 19, 2026

@sentry review

1 similar comment
@vaind
Copy link
Copy Markdown
Owner Author

vaind commented Feb 19, 2026

@sentry review

@vaind vaind force-pushed the feat/pr-review-dashboard branch 3 times, most recently from f1cfc1f to 1f528da Compare February 19, 2026 13:32
vaind added a commit to getsentry/sentry that referenced this pull request Feb 20, 2026
…#108531)

## Summary

> [!NOTE]  
> This is currently an internal/experimental tool, not meant to be
released to customers as is. Take it as a conversation starter and
source of internal UX feedback.

- Add `CodeReviewEvent` model to track Seer PR code review lifecycle
events (trigger, completion, status)
- Includes composite indexes for efficient querying by org, repo, and PR
number
- Unique constraint on `(organization_id, repository_id, trigger_id)` to
prevent duplicate events

**PR 1 of 3** for the PR Review Dashboard feature.

- Project overview:
https://linear.app/getsentry/project/pr-review-dashboard-dcc8b47f6d28/overview
- Full PR (being split): vaind#1

## Test plan

- Migration applies cleanly on a fresh database
- Backend logic and API endpoints (PR 2) will add tests exercising this
model
priscilawebdev pushed a commit to getsentry/sentry that referenced this pull request Feb 24, 2026
…#108531)

## Summary

> [!NOTE]  
> This is currently an internal/experimental tool, not meant to be
released to customers as is. Take it as a conversation starter and
source of internal UX feedback.

- Add `CodeReviewEvent` model to track Seer PR code review lifecycle
events (trigger, completion, status)
- Includes composite indexes for efficient querying by org, repo, and PR
number
- Unique constraint on `(organization_id, repository_id, trigger_id)` to
prevent duplicate events

**PR 1 of 3** for the PR Review Dashboard feature.

- Project overview:
https://linear.app/getsentry/project/pr-review-dashboard-dcc8b47f6d28/overview
- Full PR (being split): vaind#1

## Test plan

- Migration applies cleanly on a fresh database
- Backend logic and API endpoints (PR 2) will add tests exercising this
model
mchen-sentry pushed a commit to getsentry/sentry that referenced this pull request Feb 24, 2026
…#108531)

## Summary

> [!NOTE]  
> This is currently an internal/experimental tool, not meant to be
released to customers as is. Take it as a conversation starter and
source of internal UX feedback.

- Add `CodeReviewEvent` model to track Seer PR code review lifecycle
events (trigger, completion, status)
- Includes composite indexes for efficient querying by org, repo, and PR
number
- Unique constraint on `(organization_id, repository_id, trigger_id)` to
prevent duplicate events

**PR 1 of 3** for the PR Review Dashboard feature.

- Project overview:
https://linear.app/getsentry/project/pr-review-dashboard-dcc8b47f6d28/overview
- Full PR (being split): vaind#1

## Test plan

- Migration applies cleanly on a fresh database
- Backend logic and API endpoints (PR 2) will add tests exercising this
model
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant