Repository navigation
fix(cli): read the registry from HeyGen's CDN first, and say why when no host answers - #7014
miguel-heygen wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
Requesting changes at 685af416.
1. No fails without the fix: line. This lane requires a fix PR to name, on a line starting fails without the fix:, a test that fails without the fix. The body says "The 5 new transport and manifest tests fail on main" but doesn't name any of them on that line. For example: fails without the fix: transport.test.ts › "<name>", catalog.test.ts › "<name>".
2. The mirror's 4 s cap also aborts the response body, with no fallback. AbortSignal.any([signal, AbortSignal.timeout(MIRROR_WAIT_MS)]) is attached to the fetch whose Response is returned to the caller, so the 4 s keeps running while the caller reads the body. I checked this on Node 22: a 200 whose body streams for about 3 s under AbortSignal.timeout(1000) gets headers, then text() throws TimeoutError. In this PR that error happens outside both the mirror→raw fallback and fetchWithRetry, which only wraps the fetch. So once the mirror serves files, any registry file that takes more than 4 s in total fails the install. Examples are the 2.5 MB macbook.glb models and the 2 MB wav/png assets on a slow link, and slow links are the networks this PR targets. Today those succeed from raw GitHub within the caller's 10 s. Possible fixes: cap only the wait for headers and clear the timer once they arrive, or read the mirror body fully inside the fallback try. Please add a test with a slow-body mirror.
3. The mirror isn't serving the registry. https://static.heygen.ai/hyperframes-oss/desktop/registry/main/registry.json and …/blocks/aurora/registry-item.json both return 403 AccessDenied, which is S3's missing-key answer. Raw GitHub serves registry.json (200, 37.5 KB). Neither this repo nor hyperframes-internal contains a workflow that writes hyperframes-oss/desktop/registry/main, so I can't confirm "published there on every change to registry/ on main". As merged, every default-registry read worldwide makes one failing request to the CDN first, and blocked networks still get nothing. Please point to the publisher, or land it first, and confirm the mirror's registry.json matches raw GitHub.
4. Comments fails on this head. catalog.ts goes from 159 to 160 comment lines, and transport.ts goes from 1/44 to 4/83.
Looks right: fetchRegistryManifest keeps its undefined contract by wrapping readRegistryManifest. A stale cached copy is still served before any error. A custom registry is never rewritten to the mirror.
|
Answered at
The three changed test files pass 3 runs in a row (111 tests); oxlint and oxfmt are clean. |
There was a problem hiding this comment.
Requesting changes at 2987c132. Three of my four points are fixed. The mirror point is not answered.
Fixed, and verified locally:
fails without the fix:: the line is now present, and the named tests fail at merge basecd598426and pass at head. The three changed test files pass 111/111.- Mirror cap: it now covers only the wait for the answer. The timer is cleared once the mirror answers, and the body reads under the caller's signal. With the
clearTimeoutafter the answer removed, "leaves a mirror answer that came in time readable past the mirror's cap" fails, so the new test pins the fix. Comments: passes.
Still blocking: the mirror doesn't exist yet, and the body still says it does. As of this review, https://static.heygen.ai/hyperframes-oss/desktop/registry/main/registry.json and …/blocks/aurora/registry-item.json still return 403 AccessDenied. The body still says "The registry folder is published there on every change to registry/ on main". I found no publisher in this repo or in hyperframes-internal, and the update doesn't name one.
Until it's published, this PR:
- adds one failing CDN round trip, of up to 4 s on a slow or hanging route, to every default-registry read for every user, which on installs means every item file;
- does nothing for the blocked networks it's for.
Any one of these clears it:
- Link the workflow or job that publishes
registry/to that prefix, with the mirror'sregistry.jsonreturning 200 and matching raw GitHub. - Land the publisher first and re-pin this PR.
- Correct the body to say the mirror is not live yet, and state that merging ahead of it is intended. I'll approve on that if whoever owns the CLI's network behaviour accepts the extra request in the meantime.
|
Agreed on the mirror: the publisher goes first. I corrected the body. It now says the mirror isn't live yet and that this PR merges only after |
2987c13 to
caf5249
Compare
jrusso1020
left a comment
There was a problem hiding this comment.
Reviewed at caf52499d7cc633c3434afd0f362654754b9d137.
The header-wait cap now clears when the mirror answers, preserving the caller's budget for the response body (packages/cli/src/registry/transport.ts:49–54). Custom registries remain direct, and valid stale manifest fallback is preserved. The required fails without the fix: line is present.
Important — preserve raw-GitHub failure semantics when adding the two-host diagnostic (transport.ts:65–67). The new wrapper always throws a plain Error, while existing callers depend on the original error:
registry/resolver.ts:67switches later item batches to cache-only onTimeoutError. With mirror 403 and raw GitHub timing out, the wrapper hides that name. ActualloadAllItemswith 100 uncached entries makes 100 raw + 100 mirror requests, compared with 32 raw requests on the unmirrored control. Only the 10-second timeout was accelerated to 25 ms; real abort signals drive the mock fetch. This changes one timeout wave into all waves. Extrapolation, not a production timing measurement: 448 uncached items at 32 concurrent could take roughly 140 seconds rather than one 10-second wave.registry/remote.ts:223–230scans the complete diagnostic for TLS errors to decide retryability. MirrorERR_TLS_CERT_ALTNAME_INVALIDplus a transient rawECONNRESETis classified as non-retryable, so the file fails after one raw request although a second would succeed. The unmirrored control retries and returns the file.
Fix: retain structured error metadata for the final raw attempt, including its timeout identity, while keeping both host reasons for display. Make retry classification use that raw failure rather than the mirror's diagnostic. A local control preserving the raw name and classifying the raw cause restores 32 requests and successful retry; it was restored afterward. Add two separate regressions through the real consumers: resolver stops new network batches after a raw timeout; file download retries raw ECONNRESET despite mirror TLS failure.
Important — publisher prerequisite remains unmet, as in Genesis's prior review and the author's explicit publisher-first agreement (transport.ts:33,43–51). Fresh HTTPS reads of both CDN registry.json and blocks/aurora/registry-item.json return 403 AccessDenied; raw GitHub's registry returns 200 (37,569 bytes). The body now correctly says the publisher must land first. Until then every uncached default-registry file gains a failing CDN request and the blocked-GitHub cohort still cannot load it. To clear: run the approved publisher, link its job, and show CDN registry 200 with identical bytes/SHA-256 to raw GitHub, plus blocks/aurora/registry-item.json and a file referenced by it returning the corresponding raw-GitHub content. No need to change the truthful body back.
Validation: full root bun run build passes; bun run --cwd packages/cli test src/registry/transport.test.ts src/registry/remote.test.ts src/commands/catalog.test.ts src/registry/resolver.test.ts reports 134 passed. Changed-file oxlint passes. The lane-named transport/remote diagnostic tests fail with mirror bypassed, and the slow-body-cap test fails with its timer cleanup removed; restored suite passes. Those check earlier fixes; the separate actual-consumer probes demonstrate the newly reported timeout and retry regressions. Fresh adversarial probes independently found the error-contract regression; I reran them and the proposed-fix control. Network availability is a point-in-time check, not a CDN publishing audit. No mainland-China or Desktop end-to-end run.
Need: yes · Size: ok · Reuse: existing resolver timeout guard and remote retry classifier · Simpler: keep final-attempt failure metadata separate from display diagnostics · Structure: fine
Risk: medium — changes shared registry transport and outage behavior.
Verdict: REQUEST CHANGES
Reasoning: The mirror cap is fixed, but the aggregate error breaks existing timeout/retry consumers, and the explicitly required mirror deployment is still absent.
— hf-review
|
Parked as a draft until the registry mirror on static.heygen.ai is live; that's the one open finding. It goes back to ready for review, with the mirror's REVISION and registry.json checked, the moment it is. |
|
Closing: a stale draft; main has since changed how the registry reports read failures. |
What
The CLI reads the default registry from a copy on HeyGen's CDN (
static.heygen.ai/hyperframes-oss/desktop/registry/main) first, then from raw.githubusercontent.com. When neither host answers,hyperframes catalogexits 1 and says why instead of printing an empty catalog.Why
Some networks can't reach raw.githubusercontent.com: it's blocked or DNS-poisoned in mainland China and on some ISPs elsewhere. On those networks:
hyperframes catalog --jsonprinted[]and exited 0, the same answer a reachable registry with no items would give. HyperFrames Desktop showed "The catalog is empty", and its retry could never help.addand item installs failed with a bare network error.HeyGen's CDN is reachable from those networks: HyperFrames Desktop already downloads and updates from it.
Not live yet, and this PR waits for it. The copy of
registry/on the CDN is made by a publishing job in HeyGen's own infrastructure that hasn't merged yet. Until it runs,registry.jsonat the mirror answers 403. Merging this before then would add a failing CDN round trip to every default-registry read and help no one. So this merges only afterhttps://static.heygen.ai/hyperframes-oss/desktop/registry/main/registry.jsonanswers 200 and matches raw GitHub's.To reproduce before this change: deny raw.githubusercontent.com (for example
127.0.0.1 raw.githubusercontent.comin the hosts file), clear~/.hyperframes/cache, runhyperframes catalog --json. It prints[]and exits 0.Related work
None.
How
fetchRegistryHttps(registry/transport.ts) is the one place every registry read goes through: the top-level manifest, item manifests and item files. A URL under the default registry is tried on the mirror first. The mirror gets 4 s to answer, so a hanging mirror still leaves the caller's time for raw GitHub. The cap covers only the wait for the answer: once the mirror answers, its body reads under the caller's own signal, as raw GitHub's does. Any mirror answer other than 2xx (S3 answers 403 for a key it doesn't hold), or any error, falls back to raw GitHub. A customregistryinhyperframes.jsonis read only where it was given.static.heygen.ai (ECONNREFUSED), raw.githubusercontent.com (ECONNREFUSED).readRegistryManifestthrows that reason when nothing is cached.fetchRegistryManifestkeeps its contract (undefined), so the picker's fallback to bundled items is unchanged.catalogusesreadRegistryManifestand exits 1 with the message.DEFAULT_REGISTRY_URLmoved totransport.ts, next to the mirror;remote.tsre-exports it, so imports are unchanged.Test plan
fails without the fix:
transport.test.ts› "names both hosts when neither answers",remote.test.ts› "throws why when neither registry host answers and nothing is cached",catalog.test.ts› "exits non-zero with the reason instead of printing an empty list"transport.test.ts(mirror first; fallback on a refused mirror and on a 404; both hosts refused names both; a custom registry is untouched),remote.test.ts(readRegistryManifestthrows why when neither host answers and nothing is cached),catalog.test.ts(an unreadable registry exits 1 with the reason and prints no[]). The new transport and manifest tests fail on main. Two more pin the mirror's cap: "leaves a mirror answer that came in time readable past the mirror's cap" (it fails when the cap's timer isn't cleared once the mirror answers) and "falls back to raw GitHub when the mirror gives no answer within its cap". Five existing tests that count fetch calls now count two per attempt (mirror, then raw GitHub).packages/clisuite has the same failures with and without this change (all from packages not built in my environment), plus the new tests passing. The three changed test files pass 3 runs in a row. oxlint and oxfmt are clean on the changed files.hyperframes catalog --jsonat 0.8.230 and 0.9.2 returns all 448 items from raw GitHub.hyperframes.jsonbehaves as before.transport.tsorcatalog.ts; the reasons are here.fails without the fix:
resolver.test.ts› "stops starting new batches after the raw host times out, whatever the mirror said" andremote.test.ts› "retries a transient raw GitHub reset even when the mirror failed on TLS". The two-host error keeps the raw GitHub failure's name and cause, and retry judges that failure; with that reverted, exactly these two fail and the other 59 pass.