Skip to content

Publish an opt-in adopters registry, and the rule that every adoption number comes from it (#475) - #531

Merged
pengfei-threemoonslab merged 4 commits into
mainfrom
claude/issue-475-review-360ece
Sep 6, 2026
Merged

pengfei-threemoonslab merged 4 commits into
mainfrom
claude/issue-475-review-360ece

Conversation

@pengfei-threemoonslab

@pengfei-threemoonslab pengfei-threemoonslab commented Sep 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Closes #475.

Shipgate collects nothing, deliberately. The price is that it cannot count its
own users: downloads measure CI caches, stars measure sentiment, and the repos
that quietly run the gate every day are invisible. This adds the counting
mechanism that stance requires — ADOPTERS.md,
opt-in and self-served, plus the rule that every adoption number this project
publishes comes from it.

  • One row per adopter, added by the adopter — through the new
    Adopter entry issue form or a
    pull request. Six fields: who, which repository (or the single word
    private), what it gates, the tier, the date, and a link to the public act
    where they asked. That link is the consent record, and it is what the count
    rests on.
  • Three tiers, defined tightly: local evaluation, advisory CI,
    blocking CI — the last one meaning the job fails on a verdict and the
    check is required on the protected branch.
  • An eight-rule claims policy. No entry, no claim; every number carries its
    as-of date; external and dogfooding are never summed; an entry is a dated
    statement rather than a measurement; a tier is self-reported and never
    upgraded; a badge is not an entry; the design-partner ledger (The design-partner pilot stops at one PR: measure reviewer value and the next eligible change #521) and the
    historical corpus (Gate 2: participant validation of the frozen corpus (#456) #511) stay separate; removal is unconditional.
  • One optional badge, deliberately tier-neutral, linking back to the
    registry. There is no blocking-CI variant and the file says why.
  • Today's numbers, published rather than omitted: 0 external entries, 1
    maintainer dogfooding entry.

The self-entry says advisory CI

Both self-gate workflows do fail the run on blocked and unknown — but
main carries no required status check, so a red run stops no merge. Under the
file's own definitions that is advisory, and a guard fails the build if the row
is ever raised without rewriting the paragraph that explains it. It is the
smallest available demonstration of the rule the registry runs on: the tier is
what is true, not what is flattering.

Type

  • Documentation only

Surface discipline. Not a new surface under
CONTRIBUTING §Surface discipline — no CLI
command, schema, report block, discovery surface or adapter, and no data
collection of any kind. ADOPTERS.md is registered in
NOT_A_DISTRIBUTION_SURFACE with its reason (it publishes self-reported
adopter claims, not an engine answer), which is what the top-level classifier
requires of any new root entry. Headline metric: attributable public adoption
claims, with dogfooding and external entries separated.

Verification

Local, all green: ruff check .; the full pytest -n auto -m "not perf" gate;
tests/test_adapter_static_only.py and tests/test_latency_budget.py -m perf
in their own runs, as CI splits them.

tests/test_adopters_registry.py (46 tests) is the "and verify" half of the
issue title. It fails on a row missing a field, a row whose cell count is wrong
(reported by name rather than as a zip() error during collection), an
undefined tier, a non-ISO or future date, an entry link that does not point
into this repository, this repository listed as an external adopter, counts
that disagree with the rows, an as-of date older than the newest entry, an
empty table that stops saying it is empty, a claims rule quietly dropped, a
second badge variant, a README that drops the privacy stance or drifts from the
registry's counts, an issue form that offers an undefined tier or stops
requiring consent — and any adopter number stated anywhere in the repository's
prose that these rows cannot source.

A 50-case perturbation sweep drove every one of those mutations against the
suite and confirmed each fails, that the legitimate edits still pass (adding a
real adopter correctly, adding a second maintainer row with its explanation,
writing the grammatically singular "1 external adopter entry"), and — via a
no-op detector — that every mutation string actually matched something. It
found two real defects while I was writing it, both fixed here:

  1. The badge guard folded case before looking for a tier token, so a
    blocking%20CI badge variant slipped through. It now also requires the
    section to carry exactly one shields.io URL.
  2. The claim scanner's first draft misfired on real prose in this repository —
    "5 sentences of user instructions", "27 national company registries" —
    because it accepted a preposition as filler. A number now counts as a claim
    only when it reads as one: a use verb in front of it, a use verb behind it,
    or a noun with no other meaning here. Both strings are pinned as
    must-not-fire cases.

A third came out of the sweep's own failure: the negative controls hard-coded
the number 1, which silently stopped being a control the moment a test
adopter row made 1 traceable. They now derive an unsourceable number from the
published counts.

Two review rounds followed, both with findings (commits 2 and 3):

  • Round 1, on the guards: the README count assertion demanded the plural
    "entries", so the first real adopter had to write "1 external adopter
    entries" or take a red build over grammar; the self-entry agreement guard
    hard-coded a sentence opener that cannot be written naturally once there are
    two maintainer rows; dates were compared against the runner's local today,
    failing a contributor east of UTC; and the claim scanner re-parsed
    ADOPTERS.md once per scanned file (294 today).
  • Round 2, on the prose: the self-entry's advisory CI claim had been
    checked against the wrong mechanism.
    Classic branch protection reports
    main as unprotected, but an active ruleset governs it. It requires a pull
    request and linear history and carries no required_status_checks rule — so
    the tier stands, and the file now names the command that shows it. ROADMAP.md
    was also restating the external count in prose, which is rule 2's staleness
    trap one file over; it points at the registry now instead.
  • Round 3, on review feedback (three P2s, all reproduced before and after, in
    5789ef07): a neighbouring sentence about dogfooding could authorize an
    explicitly external number — the exemption is now scoped to the claim's own
    sentence, and a claim that calls itself external never takes it at all;
    duplicate rows inflated the counts, since counts were compared to
    len() — rows now carry an adopter+repository identity checked across both
    tables; and only GitHub repositories were accepted, which would have told
    a GitLab or self-hosted adopter to write private about a public repository
    — repository host and consent record are now validated independently.

Not done here

The site half of the acceptance is not in this repository. "Linked from the
README and the site" — the README is done; threemoonslab.com is built from the
private ThreeMoonsLab/web repo, so the site link needs a change there. Happy
to file that as a tracking issue in web on request; it is the one acceptance
box this PR cannot tick.

Release-readiness notes

  • No user-code import added to default scan paths
  • No network access added to default scan paths
  • New or changed check IDs are documented in docs/checks.md — n/a, no check changes
  • Report/schema changes are additive or documented in STABILITY.md — n/a, no schema changes

🤖 Generated with Claude Code

…mes from it (#475)

Adoption evidence has been this project's #1 named risk in every strategic
review, and the reason is structural: Shipgate collects nothing, so downloads
measure CI caches and stars measure sentiment. Neither is adoption. This adds
the counting mechanism the no-telemetry stance requires.

`ADOPTERS.md` is opt-in and self-served: one row per adopter, added by the
adopter through an issue form or a pull request, naming what they gate, the
tier (local evaluation / advisory CI / blocking CI), the date, and a link to
the public act where they asked. Private repositories are listed at
organization granularity. Entries are removed on request, without a reason.

The deliverable is the claims policy, not the rows. Eight rules: no entry, no
claim; every number carries its as-of date; external and dogfooding are never
summed; an entry is a dated statement rather than a measurement; a tier is
self-reported and never upgraded; a badge is not an entry; the design-partner
ledger and the historical corpus stay separate; removal is unconditional.

Today's numbers are 0 external entries and 1 maintainer dogfooding entry. That
row reads `advisory CI`, not `blocking CI`, because `main` requires no status
check and a red run there stops no merge — the smallest available
demonstration that the tier is what is true rather than what is flattering.

`tests/test_adopters_registry.py` enforces it: row shape, closed tier
vocabulary, ISO dates that are not in the future, entry links that point into
this repository, this repository never listed as an external adopter, counts
that equal the rows, an as-of date not older than the newest entry, one
tier-neutral badge, an issue form that cannot drift from the registry's
vocabulary or stop requiring consent, and any adopter number in the
repository's prose that these rows cannot source. A 42-case perturbation sweep
confirmed each guard fails on the weakening it exists to catch, that a
correctly added adopter still passes, and that benchmark prose ("5 sentences
of user instructions", "27 national company registries") does not trip the
claim scanner.

Co-Authored-By: Claude Opus 5 <[email protected]>
Four findings from a review pass over the guards, all reproduced before
fixing and all pinned by new perturbation cases (sweep now 45 cases).

- The README external-count assertion was a bare substring requiring the
  plural "entries", so the first real adopter — count 1 — had to write
  "1 external adopter entries" or take a red build over grammar. Both counts
  now accept entry/entries, as the dogfooding assertion already did.
- The self-entry agreement guard demanded the literal opener "That row says
  `<tier>`" for every dogfooding row, which cannot be written naturally once
  there are two. It now pins the claim ("row says `<tier>`") rather than one
  sentence shape; a second maintainer row with no explanation still fails.
- Dates were compared against the runner's local `today`, so a contributor
  east of UTC dating an entry with their own today would fail CI in a UTC
  runner. One day of skew is allowed; "2027-03-01" still fails.
- `_untraceable_claims` re-read and re-parsed ADOPTERS.md once per scanned
  file (294 today). The counts are read once and passed in.

Co-Authored-By: Claude Opus 5 <[email protected]>
…g counts

Three findings from a content pass over the prose, which the first round did
not cover.

- **The self-entry's tier claim was checked against the wrong mechanism.**
  Classic branch protection reports `main` as unprotected, but an active
  ruleset governs it. The ruleset requires a pull request and linear history
  and carries no `required_status_checks` rule, so `advisory CI` stands — but
  the file now says how the claim was made, with the command, so the next
  reader can disagree with it rather than trust it.
- **ROADMAP.md restated the external count in prose.** Rules 1 and 2 say a
  number is published from the registry with its date; a second copy in a file
  nothing checks is how the first one goes stale. It now points at the file
  instead of repeating its number.
- The changelog claimed a 42-case sweep; it is 45 since the last round's
  fixes. Two added README lines were also left unwrapped.

Co-Authored-By: Claude Opus 5 <[email protected]>

@pengfei-threemoonslab pengfei-threemoonslab left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all 10 changed files at eb71c0b against db63e42. I found three validation issues worth fixing before merge; each inline comment includes a reproduced case.

Validation: all 830 tests in test_adopters_registry.py, test_distribution_surface_parity.py, and test_public_surface_contract.py passed; Ruff on the changed Python files and git diff --check passed. I also ran separate mutations against the new registry suite: a duplicated maintainer row and an unsupported external claim both passed, while a valid GitLab entry failed. The three GitHub Actions workflows report success for this head.

The static verifier reports control_state: complete, gate.merge_verdict: mergeable, and release_decision.decision: passed, with no blockers or review items. That result covers the configured capability surface; the findings below concern the new registry validation.

Comment thread tests/test_adopters_registry.py Outdated
Comment on lines +659 to +660
context = flat[max(0, match.start() - _CONTEXT) : match.end() + _CONTEXT].lower()
if number == dogfood and ("dogfood" in context or "maintainer" in context):

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Keep dogfooding context local to the matched claim

The 120-character window lets an unrelated dogfooding sentence authorize an explicitly external number. With the current counts (0 external, 1 maintainer), appending Agents Shipgate is used by 1 external team. Maintainer dogfooding is counted separately. to ROADMAP.md leaves all 46 registry tests passing: this branch accepts the external 1 using the maintainer total. This defeats the separation the scanner is supposed to enforce, especially in prose that discusses both counts together. Require the matched claim itself to be qualified as dogfooding, and never use that exemption for a claim explicitly qualified as external; add this adjacent-sentences case as a regression.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5789ef0, and your reproduction is now a regression test.

Reproduced first: appending that sentence pair to ROADMAP.md left all 46 tests green — the external 1 was authorized by the next sentence, which is the sum rule 3 exists to forbid.

Two changes, because the window was wrong in two independent ways:

  • The qualifier is scoped to the claim's own sentence (_sentence_around), not a character window. So used by 1 team. Maintainer dogfooding is separate. is flagged too, not just the explicitly-external form.
  • A claim that qualifies itself as external never takes the exemption, whatever its sentence says. That is checked on the matched claim text rather than the sentence, deliberately: 1 maintainer dogfooding adopter and 0 external adopters is a legitimate sentence that names both counts, and vetoing on the whole sentence would have made honest combined prose unwritable — which is the failure mode that gets a guard deleted rather than obeyed.

Regressions added: test_a_neighbouring_dogfooding_sentence_authorizes_nothing (your exact case, plus the unqualified variant) and test_an_externally_qualified_claim_never_borrows_the_dogfooding_count. Both are also perturbation-sweep cases, so the fix is checked from outside the suite as well.

Comment on lines +354 to +356
counts = _counts()
assert counts["External adopter entries"] == str(len(_entries(EXTERNAL_HEADING)))
assert counts["Maintainer dogfooding entries"] == str(len(_entries(DOGFOOD_HEADING)))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Reject duplicate adopter rows before counting them

The counts are compared only to list lengths, so the same adoption and consent record can be counted repeatedly. I duplicated the existing Three Moons Lab row byte for byte and changed the registry/README dogfooding count to 2; all 51 resulting registry tests passed, although there is still only one adopter and one public act. The same counting logic governs external entries, so accidental duplicate submissions can inflate the headline metric while every guard stays green. Add a uniqueness check for the adopter/entry identity (at minimum rejecting identical rows) before treating row counts as attributable adoption counts.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5789ef0. Confirmed your reproduction — a byte-for-byte duplicate of the maintainer row with the count moved to 2 passed everything, because the counts were only ever compared to len().

test_no_adopter_is_counted_twice now rejects it. The identity is adopter + repository link, not the whole row, so a near-duplicate (same repository, reworded description, different date) is caught too — an identical-row check alone would have missed the more likely accident. Identity is compared across both tables, since the same party listed as external and as dogfooding inflates the headline number from precisely the place rule 3 forbids.

Two consequences worth stating, and the file now states them: a private row is identified by its adopter alone — that is all the registry knows about it — so one organization holds at most one; and a second repository is a second row, which is also how an adopter with two different tiers records that honestly.

Comment thread tests/test_adopters_registry.py Outdated
Comment on lines +337 to +338
assert len(hrefs) == 1 and re.fullmatch(
r"https://github\.com/[\w.-]+/[\w.-]+", hrefs[0]

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Accept public repositories hosted outside github.com

This check restricts the adopter's repository to GitHub even though the registry and issue form require a public repository link or private, and the CLI can be used with repositories on other hosts. A correctly filled external row containing [acme/agent](https://gitlab.com/acme/agent), with its consent link still pointing into this project and both published counts updated, fails this assertion (49 passed, 1 skipped, 1 failed). Legitimate GitLab, Codeberg, or self-hosted adopters therefore cannot follow the documented contribution path without mislabeling their public repository as private. Validate the public repository URL independently of the GitHub-only consent-record requirement.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and fixed in 5789ef0 — this was the worst of the three, because the failure mode was the registry telling a truthful adopter to mislabel their public repository as private in order to fit the validator.

The two checks are now independent, which is the distinction the old regex collapsed:

  • Repository accepts any host — https://gitlab.com/…, https://codeberg.org/…, https://git.acme.example:8443/… — or the single word private. Shipgate runs on a checkout, not on a forge API, so there was never a reason for GitHub to be privileged here.
  • Entry stays restricted to an issue/PR/discussion in this repository. That one is not about hosting: it is the public act that makes the row attributable, and a consent record nobody outside the maintainers can open is not one.

Both rules are extracted into _repository_problem / _entry_record_problem and unit-tested with synthetic accept and reject cases, so the boundary is testable without adding rows to the registry. ADOPTERS.md and the issue form now say "on whichever host it lives", and a GitLab-hosted adopter is a must-pass perturbation case.

… accept any host

Three P2 findings from review of eb71c0b, each reproduced before and after.

- **A neighbouring sentence could authorize an external claim.** The
  dogfooding exemption used a 120-character window, so "Agents Shipgate is
  used by 1 external team. Maintainer dogfooding is counted separately."
  passed the whole suite: the external `1` borrowed the maintainer total,
  which is the sum rule 3 exists to forbid. A qualifier now only qualifies its
  own sentence, and a claim that calls itself external never takes the
  exemption at all — prose that discusses both counts together is exactly
  where this had to hold. Both halves are pinned as regressions.

- **Duplicate rows inflated the counts.** The counts were compared to list
  lengths, so the same adoption and the same consent record could be counted
  twice with every guard green. Rows now carry an identity — adopter plus
  repository link, compared across both tables — and a repeat fails. A
  `private` row is identified by its adopter alone, which is all the registry
  knows about it, so one organization holds at most one; the file says so, and
  says a second repository is a second row.

- **Only GitHub repositories were accepted.** A correctly filled row linking
  `https://gitlab.com/acme/agent` failed, leaving a GitLab, Codeberg or
  self-hosted adopter to write `private` about a public repository in order to
  be counted — a registry mislabelling its own rows to fit its validator. The
  repository link is now validated independently of the consent record: any
  host for the repository, still this repository for the public act that
  created the row. Both rules have their own accept/reject unit cases.

Sweep is 50 cases; the three reported reproductions are in it.

Co-Authored-By: Claude Opus 5 <[email protected]>
@pengfei-threemoonslab
pengfei-threemoonslab merged commit faeb91f into main Sep 6, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adoption evidence without telemetry: an opt-in adopters registry we can publish and verify

1 participant