Skip to content

Commit 0bda6b5

Browse files
feat(cli): track which registry items add installs (#3099)
* feat(cli): track which registry items `add` installs `cli_command` records that `add` ran and nothing about what it installed, and the registry is served from raw.githubusercontent.com, which gives no per-item counter either — so there is no way to tell which block or component people actually pull, and no way to know what is worth building more of. Emit one `registry_item_added` event per item written into a project, from `runAdd` after the install succeeds. That is the single choke point: the bulk `add <tag>` path re-enters it per item, and a failed or compatibility-refused install throws before it, so a refused install is never counted as a download. `requested` separates the item the user named from the transitive `registryDependencies` pulled in behind it; without it a popular dependency outranks everything that depends on it. Item names are public registry identifiers, never user content or project data, and the event goes through `trackEvent` — an install that opted out via `hyperframes telemetry disable`, `HYPERFRAMES_NO_TELEMETRY` or `DO_NOT_TRACK` sends nothing. * test(cli): cover `add` telemetry end to end against the built CLI The unit tests assert the emit seam and nothing past it. `shouldTrack()` short-circuits whenever `isDevMode()` is true, and that is true for any `.ts` entry, so under vitest a real event and no event are indistinguishable and the transport is never exercised at all. Drive the built CLI instead and assert on the HTTP body it actually produces: one event per installed item, the dependency reported with `requested: false`, an opted-out install sending no request at all (not merely one without this event), and a refused install counting nothing. Two fixtures, because neither case is reachable through the real registry. The registry origin is a first-class project setting, so a local one supplies the `registryDependencies` edge that no shipped catalog item declares today; and `globalThis.fetch` is wrapped to capture the batch rather than send it. The faked 200 is load-bearing: only a failed flush leaves events queued, and only a non-empty queue spawns the detached `flushSync` child that would bypass the hook and reach production analytics. Verified the check can fail — forcing `requested: true` for every item turns it red on exactly the dependency assertion.
1 parent 57ec008 commit 0bda6b5

6 files changed

Lines changed: 344 additions & 2 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -755,6 +755,13 @@ jobs:
755755
756756
test -s /tmp/hf-cli-inside/renders/inside.mp4
757757
758+
# Belongs in this job because it needs the BUILT CLI: `shouldTrack()`
759+
# short-circuits whenever `isDevMode()` is true, which it is for any `.ts`
760+
# entry, so under vitest a real event and no event look identical. Sends
761+
# nothing to PostHog — the script wraps fetch and captures the batch.
762+
- name: Smoke-test `add` telemetry end to end
763+
run: node scripts/ci/cli-telemetry-e2e.mjs
764+
758765
- name: Assert page.goto completes under 5s budget
759766
run: |
760767
set -euo pipefail

‎docs/packages/cli.mdx‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1063,7 +1063,8 @@ npx hyperframes telemetry status
10631063
Telemetry collects command names, render performance, render checkpoint and
10641064
error names, aggregate browser diagnostic counts, browser initialization
10651065
duration and tween count, aggregate video-extraction workload counts (extracted
1066-
frames, VFR preflights), example choices, and system info — including a coarse
1066+
frames, VFR preflights), example choices, the names of registry items `add`
1067+
installs, and system info — including a coarse
10671068
environment fingerprint: OS, kernel string, CPU and memory shape, sandbox
10681069
runtime such as gVisor or Docker, and the _name_ of a coding agent driving the
10691070
CLI when one is detected (`claude_code`, `codex`, `cursor`). That name is

‎packages/cli/src/commands/add.test.ts‎

Lines changed: 38 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,12 @@ import { tmpdir } from "node:os";
44
import { join } from "node:path";
55
import type { RegistryItem, RegistryManifest } from "@hyperframes/core";
66
import { AddError, buildSnippet, remapTarget, runAdd } from "./add.js";
7+
import { trackRegistryItemAdded } from "../telemetry/events.js";
8+
9+
// Assert the emitted payload rather than the transport: `shouldTrack()` is
10+
// already false under test (dev mode / no PostHog key), so a real call would
11+
// be indistinguishable from no call at all.
12+
vi.mock("../telemetry/events.js", () => ({ trackRegistryItemAdded: vi.fn() }));
713

814
// ── Fixtures ────────────────────────────────────────────────────────────────
915

@@ -250,7 +256,10 @@ describe("add command pure helpers", () => {
250256
});
251257

252258
describe("runAdd (integration, mocked registry)", () => {
253-
beforeEach(() => mockFetch());
259+
beforeEach(() => {
260+
vi.mocked(trackRegistryItemAdded).mockClear();
261+
mockFetch();
262+
});
254263
afterEach(() => {
255264
vi.unstubAllGlobals();
256265
});
@@ -313,6 +322,9 @@ describe("runAdd (integration, mocked registry)", () => {
313322
code: "incompatible-cli",
314323
});
315324
expect(existsSync(join(dir, "compositions/future-block.html"))).toBe(false);
325+
// Nothing was written, so nothing may be counted: an install count that
326+
// also counts refused installs is not a download count.
327+
expect(trackRegistryItemAdded).not.toHaveBeenCalled();
316328
} finally {
317329
rmSync(dir, { recursive: true, force: true });
318330
}
@@ -359,6 +371,31 @@ describe("runAdd (integration, mocked registry)", () => {
359371
}
360372
});
361373

374+
it("reports every installed item, marking only the requested one", async () => {
375+
const dir = tmp();
376+
try {
377+
writeRegistryConfig(dir);
378+
379+
await runAdd({ name: "dep-block", projectDir: dir, skipClipboard: true });
380+
381+
// A dependency dragged in behind the request must not read as a vote for
382+
// itself, or a popular dependency outranks everything that depends on it.
383+
expect(trackRegistryItemAdded).toHaveBeenCalledTimes(2);
384+
expect(trackRegistryItemAdded).toHaveBeenCalledWith({
385+
item: "base-component",
386+
itemType: "hyperframes:component",
387+
requested: false,
388+
});
389+
expect(trackRegistryItemAdded).toHaveBeenCalledWith({
390+
item: "dep-block",
391+
itemType: "hyperframes:block",
392+
requested: true,
393+
});
394+
} finally {
395+
rmSync(dir, { recursive: true, force: true });
396+
}
397+
});
398+
362399
it("throws AddError with code 'example-type' when asked to add an example", async () => {
363400
const dir = tmp();
364401
try {

‎packages/cli/src/commands/add.ts‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ import {
2828
writeProjectConfig,
2929
} from "../utils/projectConfig.js";
3030
import { copyToClipboard } from "../utils/clipboard.js";
31+
import { trackRegistryItemAdded } from "../telemetry/events.js";
3132

3233
// ── Target-path resolution ──────────────────────────────────────────────────
3334
// `registry-item.json` files specify `target` paths relative to the project
@@ -197,6 +198,17 @@ export async function runAdd(opts: RunAddArgs): Promise<RunAddResult> {
197198
// 5. Install — dependencies first, requested item last.
198199
const written = await installAll(installPlan, projectDir, config.registry);
199200

201+
// Report what landed, not what was asked for: a failed install throws above,
202+
// and the bulk `add <tag>` path re-enters here per item, so this one place
203+
// covers every way an item reaches a project.
204+
for (const planItem of installPlan) {
205+
trackRegistryItemAdded({
206+
item: planItem.name,
207+
itemType: planItem.type,
208+
requested: planItem.name === item.name,
209+
});
210+
}
211+
200212
// 6. Build include snippet + clipboard copy for the requested item.
201213
const itemForInstall = installPlan[installPlan.length - 1]!;
202214
const primaryFile =
@@ -254,6 +266,11 @@ export default defineCommand({
254266
description: "Print a machine-readable summary (written files + snippet) to stdout",
255267
},
256268
},
269+
// `run` is 28 cyclomatic and predates this change, which touches only
270+
// `runAdd`. Fallow scores it as new because the file changed. Splitting the
271+
// tag-fallback branch out would fix it honestly and is worth doing, but not
272+
// inside a telemetry change.
273+
// fallow-ignore-next-line complexity
257274
async run({ args }) {
258275
const projectDir = resolve(args.dir ?? process.cwd());
259276
const json = args.json === true;

‎packages/cli/src/telemetry/events.ts‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -498,6 +498,37 @@ export function trackInitTemplate(templateId: string, props?: { tailwind?: boole
498498
trackEvent("init_template", { template: templateId, tailwind: props?.tailwind });
499499
}
500500

501+
/**
502+
* One event per registry item written into a project.
503+
*
504+
* `cli_command` records that `add` ran, never what it installed, so the
505+
* catalog cannot be ranked by what people actually pull — and the registry is
506+
* served from raw.githubusercontent.com, which gives us no per-item counter
507+
* either. `add` is the only place an item lands in a project, so this is the
508+
* one signal that answers "which block is worth building more of".
509+
*
510+
* `requested` separates the item the user named from the transitive
511+
* `registryDependencies` dragged in behind it. A dependency installed
512+
* alongside something else is not a vote for itself, and collapsing the two
513+
* would rank a popular dependency above everything that depends on it.
514+
*
515+
* Item names are public registry identifiers, never user content or project
516+
* data. This routes through `trackEvent`, so an install that opted out
517+
* (`hyperframes telemetry disable`, `HYPERFRAMES_NO_TELEMETRY`, `DO_NOT_TRACK`)
518+
* emits nothing.
519+
*/
520+
export function trackRegistryItemAdded(props: {
521+
item: string;
522+
itemType: string;
523+
requested: boolean;
524+
}): void {
525+
trackEvent("registry_item_added", {
526+
item: props.item,
527+
item_type: props.itemType,
528+
requested: props.requested,
529+
});
530+
}
531+
501532
export function trackBrowserInstall(): void {
502533
trackEvent("browser_install", {});
503534
}

0 commit comments

Comments
 (0)