Repository navigation
Publish an opt-in adopters registry, and the rule that every adoption number comes from it (#475) - #531
Conversation
…mes from it (#475) Adoption evidence has been this project's #1 named risk in every strategic review, and the reason is structural: Shipgate collects nothing, so downloads measure CI caches and stars measure sentiment. Neither is adoption. This adds the counting mechanism the no-telemetry stance requires. `ADOPTERS.md` is opt-in and self-served: one row per adopter, added by the adopter through an issue form or a pull request, naming what they gate, the tier (local evaluation / advisory CI / blocking CI), the date, and a link to the public act where they asked. Private repositories are listed at organization granularity. Entries are removed on request, without a reason. The deliverable is the claims policy, not the rows. Eight rules: no entry, no claim; every number carries its as-of date; external and dogfooding are never summed; an entry is a dated statement rather than a measurement; a tier is self-reported and never upgraded; a badge is not an entry; the design-partner ledger and the historical corpus stay separate; removal is unconditional. Today's numbers are 0 external entries and 1 maintainer dogfooding entry. That row reads `advisory CI`, not `blocking CI`, because `main` requires no status check and a red run there stops no merge — the smallest available demonstration that the tier is what is true rather than what is flattering. `tests/test_adopters_registry.py` enforces it: row shape, closed tier vocabulary, ISO dates that are not in the future, entry links that point into this repository, this repository never listed as an external adopter, counts that equal the rows, an as-of date not older than the newest entry, one tier-neutral badge, an issue form that cannot drift from the registry's vocabulary or stop requiring consent, and any adopter number in the repository's prose that these rows cannot source. A 42-case perturbation sweep confirmed each guard fails on the weakening it exists to catch, that a correctly added adopter still passes, and that benchmark prose ("5 sentences of user instructions", "27 national company registries") does not trip the claim scanner. Co-Authored-By: Claude Opus 5 <[email protected]>
Four findings from a review pass over the guards, all reproduced before
fixing and all pinned by new perturbation cases (sweep now 45 cases).
- The README external-count assertion was a bare substring requiring the
plural "entries", so the first real adopter — count 1 — had to write
"1 external adopter entries" or take a red build over grammar. Both counts
now accept entry/entries, as the dogfooding assertion already did.
- The self-entry agreement guard demanded the literal opener "That row says
`<tier>`" for every dogfooding row, which cannot be written naturally once
there are two. It now pins the claim ("row says `<tier>`") rather than one
sentence shape; a second maintainer row with no explanation still fails.
- Dates were compared against the runner's local `today`, so a contributor
east of UTC dating an entry with their own today would fail CI in a UTC
runner. One day of skew is allowed; "2027-03-01" still fails.
- `_untraceable_claims` re-read and re-parsed ADOPTERS.md once per scanned
file (294 today). The counts are read once and passed in.
Co-Authored-By: Claude Opus 5 <[email protected]>
…g counts Three findings from a content pass over the prose, which the first round did not cover. - **The self-entry's tier claim was checked against the wrong mechanism.** Classic branch protection reports `main` as unprotected, but an active ruleset governs it. The ruleset requires a pull request and linear history and carries no `required_status_checks` rule, so `advisory CI` stands — but the file now says how the claim was made, with the command, so the next reader can disagree with it rather than trust it. - **ROADMAP.md restated the external count in prose.** Rules 1 and 2 say a number is published from the registry with its date; a second copy in a file nothing checks is how the first one goes stale. It now points at the file instead of repeating its number. - The changelog claimed a 42-case sweep; it is 45 since the last round's fixes. Two added README lines were also left unwrapped. Co-Authored-By: Claude Opus 5 <[email protected]>
pengfei-threemoonslab
left a comment
There was a problem hiding this comment.
Reviewed all 10 changed files at eb71c0b against db63e42. I found three validation issues worth fixing before merge; each inline comment includes a reproduced case.
Validation: all 830 tests in test_adopters_registry.py, test_distribution_surface_parity.py, and test_public_surface_contract.py passed; Ruff on the changed Python files and git diff --check passed. I also ran separate mutations against the new registry suite: a duplicated maintainer row and an unsupported external claim both passed, while a valid GitLab entry failed. The three GitHub Actions workflows report success for this head.
The static verifier reports control_state: complete, gate.merge_verdict: mergeable, and release_decision.decision: passed, with no blockers or review items. That result covers the configured capability surface; the findings below concern the new registry validation.
| context = flat[max(0, match.start() - _CONTEXT) : match.end() + _CONTEXT].lower() | ||
| if number == dogfood and ("dogfood" in context or "maintainer" in context): |
There was a problem hiding this comment.
[P2] Keep dogfooding context local to the matched claim
The 120-character window lets an unrelated dogfooding sentence authorize an explicitly external number. With the current counts (0 external, 1 maintainer), appending Agents Shipgate is used by 1 external team. Maintainer dogfooding is counted separately. to ROADMAP.md leaves all 46 registry tests passing: this branch accepts the external 1 using the maintainer total. This defeats the separation the scanner is supposed to enforce, especially in prose that discusses both counts together. Require the matched claim itself to be qualified as dogfooding, and never use that exemption for a claim explicitly qualified as external; add this adjacent-sentences case as a regression.
There was a problem hiding this comment.
Fixed in 5789ef0, and your reproduction is now a regression test.
Reproduced first: appending that sentence pair to ROADMAP.md left all 46 tests green — the external 1 was authorized by the next sentence, which is the sum rule 3 exists to forbid.
Two changes, because the window was wrong in two independent ways:
- The qualifier is scoped to the claim's own sentence (
_sentence_around), not a character window. Soused by 1 team. Maintainer dogfooding is separate.is flagged too, not just the explicitly-external form. - A claim that qualifies itself as external never takes the exemption, whatever its sentence says. That is checked on the matched claim text rather than the sentence, deliberately:
1 maintainer dogfooding adopter and 0 external adoptersis a legitimate sentence that names both counts, and vetoing on the whole sentence would have made honest combined prose unwritable — which is the failure mode that gets a guard deleted rather than obeyed.
Regressions added: test_a_neighbouring_dogfooding_sentence_authorizes_nothing (your exact case, plus the unqualified variant) and test_an_externally_qualified_claim_never_borrows_the_dogfooding_count. Both are also perturbation-sweep cases, so the fix is checked from outside the suite as well.
| counts = _counts() | ||
| assert counts["External adopter entries"] == str(len(_entries(EXTERNAL_HEADING))) | ||
| assert counts["Maintainer dogfooding entries"] == str(len(_entries(DOGFOOD_HEADING))) |
There was a problem hiding this comment.
[P2] Reject duplicate adopter rows before counting them
The counts are compared only to list lengths, so the same adoption and consent record can be counted repeatedly. I duplicated the existing Three Moons Lab row byte for byte and changed the registry/README dogfooding count to 2; all 51 resulting registry tests passed, although there is still only one adopter and one public act. The same counting logic governs external entries, so accidental duplicate submissions can inflate the headline metric while every guard stays green. Add a uniqueness check for the adopter/entry identity (at minimum rejecting identical rows) before treating row counts as attributable adoption counts.
There was a problem hiding this comment.
Fixed in 5789ef0. Confirmed your reproduction — a byte-for-byte duplicate of the maintainer row with the count moved to 2 passed everything, because the counts were only ever compared to len().
test_no_adopter_is_counted_twice now rejects it. The identity is adopter + repository link, not the whole row, so a near-duplicate (same repository, reworded description, different date) is caught too — an identical-row check alone would have missed the more likely accident. Identity is compared across both tables, since the same party listed as external and as dogfooding inflates the headline number from precisely the place rule 3 forbids.
Two consequences worth stating, and the file now states them: a private row is identified by its adopter alone — that is all the registry knows about it — so one organization holds at most one; and a second repository is a second row, which is also how an adopter with two different tiers records that honestly.
| assert len(hrefs) == 1 and re.fullmatch( | ||
| r"https://github\.com/[\w.-]+/[\w.-]+", hrefs[0] |
There was a problem hiding this comment.
[P2] Accept public repositories hosted outside github.com
This check restricts the adopter's repository to GitHub even though the registry and issue form require a public repository link or private, and the CLI can be used with repositories on other hosts. A correctly filled external row containing [acme/agent](https://gitlab.com/acme/agent), with its consent link still pointing into this project and both published counts updated, fails this assertion (49 passed, 1 skipped, 1 failed). Legitimate GitLab, Codeberg, or self-hosted adopters therefore cannot follow the documented contribution path without mislabeling their public repository as private. Validate the public repository URL independently of the GitHub-only consent-record requirement.
There was a problem hiding this comment.
Agreed, and fixed in 5789ef0 — this was the worst of the three, because the failure mode was the registry telling a truthful adopter to mislabel their public repository as private in order to fit the validator.
The two checks are now independent, which is the distinction the old regex collapsed:
Repositoryaccepts any host —https://gitlab.com/…,https://codeberg.org/…,https://git.acme.example:8443/…— or the single wordprivate. Shipgate runs on a checkout, not on a forge API, so there was never a reason for GitHub to be privileged here.Entrystays restricted to an issue/PR/discussion in this repository. That one is not about hosting: it is the public act that makes the row attributable, and a consent record nobody outside the maintainers can open is not one.
Both rules are extracted into _repository_problem / _entry_record_problem and unit-tested with synthetic accept and reject cases, so the boundary is testable without adding rows to the registry. ADOPTERS.md and the issue form now say "on whichever host it lives", and a GitLab-hosted adopter is a must-pass perturbation case.
… accept any host Three P2 findings from review of eb71c0b, each reproduced before and after. - **A neighbouring sentence could authorize an external claim.** The dogfooding exemption used a 120-character window, so "Agents Shipgate is used by 1 external team. Maintainer dogfooding is counted separately." passed the whole suite: the external `1` borrowed the maintainer total, which is the sum rule 3 exists to forbid. A qualifier now only qualifies its own sentence, and a claim that calls itself external never takes the exemption at all — prose that discusses both counts together is exactly where this had to hold. Both halves are pinned as regressions. - **Duplicate rows inflated the counts.** The counts were compared to list lengths, so the same adoption and the same consent record could be counted twice with every guard green. Rows now carry an identity — adopter plus repository link, compared across both tables — and a repeat fails. A `private` row is identified by its adopter alone, which is all the registry knows about it, so one organization holds at most one; the file says so, and says a second repository is a second row. - **Only GitHub repositories were accepted.** A correctly filled row linking `https://gitlab.com/acme/agent` failed, leaving a GitLab, Codeberg or self-hosted adopter to write `private` about a public repository in order to be counted — a registry mislabelling its own rows to fit its validator. The repository link is now validated independently of the consent record: any host for the repository, still this repository for the public act that created the row. Both rules have their own accept/reject unit cases. Sweep is 50 cases; the three reported reproductions are in it. Co-Authored-By: Claude Opus 5 <[email protected]>
Summary
Closes #475.
Shipgate collects nothing, deliberately. The price is that it cannot count its
own users: downloads measure CI caches, stars measure sentiment, and the repos
that quietly run the gate every day are invisible. This adds the counting
mechanism that stance requires —
ADOPTERS.md,opt-in and self-served, plus the rule that every adoption number this project
publishes comes from it.
Adopter entry issue form or a
pull request. Six fields: who, which repository (or the single word
private), what it gates, the tier, the date, and a link to the public actwhere they asked. That link is the consent record, and it is what the count
rests on.
local evaluation,advisory CI,blocking CI— the last one meaning the job fails on a verdict and thecheck is required on the protected branch.
as-of date; external and dogfooding are never summed; an entry is a dated
statement rather than a measurement; a tier is self-reported and never
upgraded; a badge is not an entry; the design-partner ledger (The design-partner pilot stops at one PR: measure reviewer value and the next eligible change #521) and the
historical corpus (Gate 2: participant validation of the frozen corpus (#456) #511) stay separate; removal is unconditional.
registry. There is no blocking-CI variant and the file says why.
maintainer dogfooding entry.
The self-entry says
advisory CIBoth self-gate workflows do fail the run on
blockedandunknown— butmaincarries no required status check, so a red run stops no merge. Under thefile's own definitions that is advisory, and a guard fails the build if the row
is ever raised without rewriting the paragraph that explains it. It is the
smallest available demonstration of the rule the registry runs on: the tier is
what is true, not what is flattering.
Type
Surface discipline. Not a new surface under
CONTRIBUTING §Surface discipline — no CLI
command, schema, report block, discovery surface or adapter, and no data
collection of any kind.
ADOPTERS.mdis registered inNOT_A_DISTRIBUTION_SURFACEwith its reason (it publishes self-reportedadopter claims, not an engine answer), which is what the top-level classifier
requires of any new root entry. Headline metric: attributable public adoption
claims, with dogfooding and external entries separated.
Verification
Local, all green:
ruff check .; the fullpytest -n auto -m "not perf"gate;tests/test_adapter_static_only.pyandtests/test_latency_budget.py -m perfin their own runs, as CI splits them.
tests/test_adopters_registry.py(46 tests) is the "and verify" half of theissue title. It fails on a row missing a field, a row whose cell count is wrong
(reported by name rather than as a
zip()error during collection), anundefined tier, a non-ISO or future date, an entry link that does not point
into this repository, this repository listed as an external adopter, counts
that disagree with the rows, an as-of date older than the newest entry, an
empty table that stops saying it is empty, a claims rule quietly dropped, a
second badge variant, a README that drops the privacy stance or drifts from the
registry's counts, an issue form that offers an undefined tier or stops
requiring consent — and any adopter number stated anywhere in the repository's
prose that these rows cannot source.
A 50-case perturbation sweep drove every one of those mutations against the
suite and confirmed each fails, that the legitimate edits still pass (adding a
real adopter correctly, adding a second maintainer row with its explanation,
writing the grammatically singular "1 external adopter entry"), and — via a
no-op detector — that every mutation string actually matched something. It
found two real defects while I was writing it, both fixed here:
blocking%20CIbadge variant slipped through. It now also requires thesection to carry exactly one shields.io URL.
"5 sentences of user instructions", "27 national company registries" —
because it accepted a preposition as filler. A number now counts as a claim
only when it reads as one: a use verb in front of it, a use verb behind it,
or a noun with no other meaning here. Both strings are pinned as
must-not-fire cases.
A third came out of the sweep's own failure: the negative controls hard-coded
the number
1, which silently stopped being a control the moment a testadopter row made
1traceable. They now derive an unsourceable number from thepublished counts.
Two review rounds followed, both with findings (commits 2 and 3):
"entries", so the first real adopter had to write "1 external adopter
entries" or take a red build over grammar; the self-entry agreement guard
hard-coded a sentence opener that cannot be written naturally once there are
two maintainer rows; dates were compared against the runner's local
today,failing a contributor east of UTC; and the claim scanner re-parsed
ADOPTERS.mdonce per scanned file (294 today).advisory CIclaim had beenchecked against the wrong mechanism. Classic branch protection reports
mainas unprotected, but an active ruleset governs it. It requires a pullrequest and linear history and carries no
required_status_checksrule — sothe tier stands, and the file now names the command that shows it. ROADMAP.md
was also restating the external count in prose, which is rule 2's staleness
trap one file over; it points at the registry now instead.
5789ef07): a neighbouring sentence about dogfooding could authorize anexplicitly external number — the exemption is now scoped to the claim's own
sentence, and a claim that calls itself external never takes it at all;
duplicate rows inflated the counts, since counts were compared to
len()— rows now carry an adopter+repository identity checked across bothtables; and only GitHub repositories were accepted, which would have told
a GitLab or self-hosted adopter to write
privateabout a public repository— repository host and consent record are now validated independently.
Not done here
The site half of the acceptance is not in this repository. "Linked from the
README and the site" — the README is done; threemoonslab.com is built from the
private
ThreeMoonsLab/webrepo, so the site link needs a change there. Happyto file that as a tracking issue in
webon request; it is the one acceptancebox this PR cannot tick.
Release-readiness notes
docs/checks.md— n/a, no check changesSTABILITY.md— n/a, no schema changes🤖 Generated with Claude Code