Skip to content

Reconcile Bun shard calibration with measurement 74 and later corpus changes #1111

Open

Description

@taras

Measured incompatibility

Successful measurement74 attempt1 covers commit c128b655afa4e16fb509ba0b8d93a4bb7c850122 on ubuntu-latest with Bun1.4.0. Its assembled artifact records packages/cli/tests/adversarial-implementation.test.ts at 421949ms, above the documented420000ms target on its own.

The current corpus at main 97d7b44e5a43ecc0cdaa179103e90bffaf3f9ee1 also contains the later-added packages/cli/tests/repl-unfinished-entry.test.ts. It has no captured measurement, so the existing fallback charges it the heaviest applicable recorded value, also421949ms. No timing may be invented for it.

measured Bun corpus total:     3432032ms
later file's normal fallback:  421949ms
current applicable total:     3853981ms
documented floor: floor(3853981 / 420000) + 1 = 10
installed Bun matrix: 9 shards

Increasing the shard count alone cannot make an indivisible421949ms file finish below420000ms. This is a calibration incompatibility, not an invalid measurement record or missing partition coverage.

Boundary and evidence

Data-only import PR1110 keeps the exact assembled bytes and measured provenance. Its focused data/partition regressions pass10groups/52steps. Independent Codex Astra at signed 8d552b5c7e636bc4a5a5cf51b9da1f5f8145b180 passes the authorized artifact import and explicitly reports this separate calibration gap. No count change, new measurement, fixed-head calibration sequence or source repair was performed for this observation. A correctness CI pass does not establish the separate seven-minute claim.

The schema/discovery/partition rules remain in scripts/lib/test-weights.ts, scripts/lib/test-files.ts and scripts/lib/test-shards.ts; installed matrices remain19/16/9 in .github/workflows/ci.yml. AGENTS.md and specs/testing-spec.md define the measured count and five-consecutive-fixed-head calibration procedure. Follow that procedure rather than choosing a count by intuition or hand-editing milliseconds.

Follow-up

Establish why the measured workflow file exceeds the budget and obtain valid evidence for later-added tests. Preserve every substantive workflow journey and normal conservative fallback; do not exclude files or reduce coverage to satisfy the target. Make any bounded repair and calibration plan concrete before launching additional runs. Keep this issue open until the runtime/window/count claim has its required evidence.

Related investigations #1107 (lost oversized-prompt diagnostic) and #1108 (cache/outer serialization) remain unresolved. They may inform diagnosis; neither is asserted to cause or fix this timing observation. Creating this tracking issue starts no extra paid qualification, measurement or calibration.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions