You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Baseline data: block/buzz main is RED (14 flutter failures / 5 analyzer diagnostics) — relevant if fork-gates ever runs upstream suites #29
Hi — we maintain a separate long-running fork of block/buzz (mirror-main + product-main split) and
found your fork-contract work while surveying independent forks. The additive-only gate in scripts/fork-gates/check-additive-only.sh + .github/workflows/fork-gates.yml (as of 468705310)
is the machine-enforced version of something we currently do by convention only, so we read it closely.
Sharing a measurement we have that seems directly relevant to anyone gating a fork against upstream:
Upstream block/buzz main is not green. On an untouched origin/main worktree we measured:
flutter test (full mobile suite): 1647 pass / 14 fail on untouched upstream main.
Failures cluster in channel_detail_page_test.dart (4), channels_page_test.dart (4), search_page_test.dart (3), remind_me_later_sheet_test.dart (3).
flutter analyze: 5 diagnostics on untouched upstream main.
Rust relay side also carries a stable set of pre-existing failures (media/admin auth-gate tests,
plus 2 order-dependent flaky ones).
Measurement method, because this part bit us: Flutter's default text reporter truncates the failing
list (... and N more). Any grep-based comparison silently compares partial sets and produces false
greens. We now always run flutter test --reporter json, parse testDone events where result != success && hidden != true, and diff sorted name sets in both directions. For the
analyzer we compare normalized signatures (severity|rule|path|line) rather than counts — counts hide
one-for-one substitutions. We re-baseline whenever upstream churns; an upstream change to channel_detail_page_test.dart recently made an older baseline untrustworthy.
Practical implication for a fork gate: a red-on-upstream baseline means pass/fail or count-based
assertions can't distinguish "our change broke it" from "upstream is already broken", so the gate has
to diff failure name sets against a pinned upstream baseline rather than assert green.
Open question, genuinely curious rather than a suggestion: how do you intend fork-gates to treat
upstream test state? Your current gates assert structural properties (additive-only diff,
must-survive set) which sidestep this entirely — that may well be deliberate. If you ever extend the
gate to run upstream suites on contrib/* branches, does the CARRIED_PATCHES lane need a pinned
upstream baseline SHA to diff against, or would you keep verification purely structural?
Happy to share the exact JSON-reporter parsing snippet and our current baseline name set if useful.
Activity