Skip to content

Commit 2d9bf34

Browse files
feat(mobile): share native dev clients across machines by fingerprint
ensure now builds one device-independent client per Expo fingerprint (arm64 simulator .app, or one APK per emulator ABI), keeps it in ~/.cache/t3code/native-clients/artifacts, and installs it on any number of devices. Missing artifacts come from the shared task cache first; new builds are uploaded back when the task-cache login can write. The task-cache Worker gains native-client routes on a separate R2 bucket with 30-day retention and multipart uploads for >100 MB bodies. Co-Authored-By: Claude Opus 5.5 <[email protected]>
1 parent 7bafc40 commit 2d9bf34

7 files changed

Lines changed: 718 additions & 81 deletions

File tree

‎apps/mobile/README.md‎

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -31,11 +31,17 @@ node ../../scripts/mobile-native-client.ts ensure ios <simulator-udid>
3131
vp run dev:client
3232
```
3333

34-
The helper compares a local Expo fingerprint and the installed binary with its last successful
35-
build record. It builds and installs missing, stale, or unverified clients and reuses matching ones.
36-
Use `check` instead of `ensure` for a read-only decision: exit 0 means compatible, 2 means a build is
37-
needed, and 1 means an operational error. Run it on the simulator host; no EAS login is required.
38-
An externally installed client is unverified until the helper builds it once.
34+
The helper compares a local Expo fingerprint and the installed binary with its last install
35+
record. Missing, stale, or unverified clients are installed from a build for that fingerprint,
36+
looked up in `~/.cache/t3code/native-clients/artifacts`, then the shared task cache, and built only
37+
when neither has it. One iOS simulator build serves every iPhone and iPad simulator, and one APK
38+
per emulator ABI (x86_64 or arm64-v8a) serves every emulator of that ABI. Builds are shared back when your task-cache login
39+
(`bash infra/task-cache/install.sh`) can write. `T3_NATIVE_CLIENT_CACHE=read` skips the upload and
40+
`off` skips the shared cache.
41+
42+
Use `check` instead of `ensure` for a read-only decision: exit 0 means compatible, 2 means an install
43+
is needed, and 1 means an operational error. Run it on the simulator host; no EAS login is required.
44+
An externally installed client is unverified until the helper installs it once.
3945

4046
Start Metro for an already verified dev client:
4147

‎infra/task-cache/README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,10 @@ vp run --filter @t3tools/task-cache typecheck
5151

5252
The smoke check uploads a unique fixture, restores its output into a cold second directory, changes its input, and confirms recomputation. It also checks that anonymous reads and uploads are rejected. Cacheable commands must run through `vp run`; direct `vp test` or `vp check` invocations do not use this cache.
5353

54+
## Native clients
55+
56+
`/projects/t3code/native-clients/<platform>/<variant>/<expo-fingerprint>` stores simulator and emulator development clients for `scripts/mobile-native-client.ts` in a separate `-native-clients` R2 bucket with 30-day retention. Unlike Vite Task entries, the Expo fingerprint is portable, so a client built on any Mac or Linux machine serves every other one. Readers download; writers upload through R2 multipart in 64 MB parts, because Workers cap request bodies. The uploader's bundle digest is stored as object metadata and checked by the client before it installs anything.
57+
5458
## Current limits
5559

5660
- Vite Task separates operating systems and CPU architectures. macOS results do not satisfy Linux CI; Linux developer machines can populate compatible CI results.
Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
import { expect, it } from "@effect/vitest";
2+
import * as Effect from "effect/Effect";
3+
import * as Layer from "effect/Layer";
4+
import * as NativeClients from "./NativeClients.ts";
5+
6+
const tokenDigest = "210eb3d2c67a7207923e1999751300a87a2dd22839b258b80c3d805a6fe64984";
7+
const auth = (name: string) => ({
8+
authorization: `Basic ${btoa(`${name}:fixture-token-012345678901234567890123456789`)}`,
9+
});
10+
const key = `ios/development/${"a".repeat(40)}`;
11+
const base = `https://cache.example${NativeClients.routePrefix}${key}`;
12+
13+
function memoryBucket() {
14+
const objects = new Map<string, { bytes: ArrayBuffer; customMetadata: Record<string, string> }>();
15+
const uploads = new Map<
16+
string,
17+
{ key: string; customMetadata: Record<string, string>; parts: Map<number, ArrayBuffer> }
18+
>();
19+
const resume = (objectKey: string, uploadId: string) => ({
20+
uploadPart: async (partNumber: number, body: ReadableStream) => {
21+
uploads.get(uploadId)!.parts.set(partNumber, await new Response(body).arrayBuffer());
22+
return { partNumber, etag: `etag-${partNumber}` };
23+
},
24+
complete: async (parts: ReadonlyArray<{ partNumber: number }>) => {
25+
const upload = uploads.get(uploadId)!;
26+
const chunks = parts.map(({ partNumber }) => upload.parts.get(partNumber)!);
27+
objects.set(objectKey, {
28+
bytes: await new Blob(chunks).arrayBuffer(),
29+
customMetadata: upload.customMetadata,
30+
});
31+
uploads.delete(uploadId);
32+
},
33+
abort: async () => void uploads.delete(uploadId),
34+
});
35+
return {
36+
head: async (objectKey: string) => (objects.has(objectKey) ? {} : null),
37+
get: async (objectKey: string) => {
38+
const object = objects.get(objectKey);
39+
return object
40+
? {
41+
size: object.bytes.byteLength,
42+
body: new Blob([object.bytes]).stream(),
43+
customMetadata: object.customMetadata,
44+
}
45+
: null;
46+
},
47+
createMultipartUpload: async (
48+
objectKey: string,
49+
options: { customMetadata: Record<string, string> },
50+
) => {
51+
const uploadId = `upload/${uploads.size}+`;
52+
uploads.set(uploadId, {
53+
key: objectKey,
54+
customMetadata: options.customMetadata,
55+
parts: new Map(),
56+
});
57+
return { uploadId };
58+
},
59+
resumeMultipartUpload: resume,
60+
uploads,
61+
};
62+
}
63+
64+
const request = (path: string, init: RequestInit & { name: string }) =>
65+
Effect.gen(function* () {
66+
const native = yield* NativeClients.NativeClients;
67+
return yield* native.handle(
68+
new Request(`${base}${path}`, {
69+
...init,
70+
headers: { ...auth(init.name), ...init.headers },
71+
}),
72+
);
73+
});
74+
75+
it.effect("serves a writer's multipart upload to readers with its digest", () => {
76+
const bucket = memoryBucket();
77+
const bindings = {
78+
CLIENTS: bucket,
79+
developerKeys: JSON.stringify({ julius: tokenDigest }),
80+
readerKeys: JSON.stringify({ "ci-reader": tokenDigest }),
81+
readLimit: { limit: async () => ({ success: true }) },
82+
storeLimit: { limit: async () => ({ success: true }) },
83+
} as unknown as NativeClients.Bindings["Service"];
84+
const digest = "d".repeat(64);
85+
return Effect.gen(function* () {
86+
const denied = yield* request("/uploads", {
87+
name: "ci-reader",
88+
method: "POST",
89+
headers: { "x-t3-digest": digest },
90+
});
91+
expect(denied.status).toBe(403);
92+
93+
const created = yield* request("/uploads", {
94+
name: "julius",
95+
method: "POST",
96+
headers: { "x-t3-digest": digest },
97+
});
98+
const { uploadId } = (yield* Effect.promise(() => created.json())) as { uploadId: string };
99+
const parts = [];
100+
for (const [index, chunk] of ["native ", "client"].entries()) {
101+
const response = yield* request(`/uploads/${encodeURIComponent(uploadId)}/${index + 1}`, {
102+
name: "julius",
103+
method: "PUT",
104+
body: chunk,
105+
headers: { "content-length": String(chunk.length) },
106+
});
107+
const { etag } = (yield* Effect.promise(() => response.json())) as { etag: string };
108+
parts.push({ partNumber: index + 1, etag });
109+
}
110+
const completed = yield* request(`/uploads/${encodeURIComponent(uploadId)}/complete`, {
111+
name: "julius",
112+
method: "POST",
113+
body: JSON.stringify({ parts }),
114+
});
115+
expect(completed.status).toBe(204);
116+
117+
const download = yield* request("", { name: "ci-reader", method: "GET" });
118+
expect(download.status).toBe(200);
119+
expect(download.headers.get("x-t3-digest")).toBe(digest);
120+
expect(yield* Effect.promise(() => download.text())).toBe("native client");
121+
122+
const again = yield* request("/uploads", {
123+
name: "julius",
124+
method: "POST",
125+
headers: { "x-t3-digest": digest },
126+
});
127+
expect(yield* Effect.promise(() => again.json())).toEqual({ exists: true });
128+
expect(bucket.uploads.size).toBe(0);
129+
}).pipe(
130+
Effect.provide(
131+
NativeClients.layer.pipe(Layer.provide(Layer.succeed(NativeClients.Bindings, bindings))),
132+
),
133+
);
134+
});
Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
import type * as Cloudflare from "@cloudflare/workers-types";
2+
import * as Context from "effect/Context";
3+
import * as Effect from "effect/Effect";
4+
import * as Layer from "effect/Layer";
5+
import * as Schema from "effect/Schema";
6+
import { developerIdentity } from "./developerAuth.ts";
7+
8+
export const routePrefix = "/projects/t3code/native-clients/";
9+
10+
export class NativeClientsError extends Schema.TaggedError<NativeClientsError>()(
11+
"NativeClientsError",
12+
{ status: Schema.Number, reason: Schema.String },
13+
) {}
14+
15+
export class Bindings extends Context.Service<
16+
Bindings,
17+
{
18+
readonly CLIENTS: Cloudflare.R2Bucket;
19+
readonly developerKeys: string;
20+
readonly readerKeys: string;
21+
readonly readLimit: Cloudflare.RateLimit;
22+
readonly storeLimit: Cloudflare.RateLimit;
23+
}
24+
>()("@t3tools/task-cache/NativeClients/Bindings") {}
25+
26+
export class NativeClients extends Context.Service<
27+
NativeClients,
28+
{ readonly handle: (request: Request) => Effect.Effect<Response> }
29+
>()("@t3tools/task-cache/NativeClients") {}
30+
31+
const CompleteBody = Schema.fromJsonString(
32+
Schema.Struct({
33+
parts: Schema.Array(Schema.Struct({ partNumber: Schema.Number, etag: Schema.String })).check(
34+
Schema.isMinLength(1),
35+
Schema.isMaxLength(64),
36+
),
37+
}),
38+
);
39+
const decodeComplete = Schema.decodeUnknownEffect(CompleteBody);
40+
41+
// <platform>/<variant>/<fingerprint>[/uploads[/<uploadId>[/<part>|/complete]]]
42+
const route =
43+
/^((?:ios|android)\/[a-z0-9_-]{1,32}\/[a-f0-9]{40,64})(?:\/uploads(?:\/([^/]{1,1024})(?:\/(complete|[1-9][0-9]?))?)?)?$/;
44+
const digestPattern = /^[a-f0-9]{64}$/;
45+
46+
const make = Effect.gen(function* () {
47+
const bindings = yield* Bindings;
48+
49+
const r2 = <A>(run: () => Promise<A>) =>
50+
Effect.tryPromise({
51+
try: run,
52+
catch: () => new NativeClientsError({ status: 503, reason: "storage_unavailable" }),
53+
});
54+
55+
const limit = Effect.fn("NativeClients.limit")(function* (writing: boolean) {
56+
const allowed = yield* r2(() =>
57+
(writing ? bindings.storeLimit : bindings.readLimit).limit({
58+
key: writing ? "t3code:native-store" : "t3code:native-read",
59+
}),
60+
);
61+
if (!allowed.success)
62+
return yield* new NativeClientsError({ status: 429, reason: "rate_limit" });
63+
});
64+
65+
const handle = Effect.fn("NativeClients.handle")(
66+
function* (request: Request) {
67+
const url = new URL(request.url);
68+
const match = route.exec(url.pathname.slice(routePrefix.length));
69+
if (!match || url.search || !url.pathname.startsWith(routePrefix))
70+
return yield* new NativeClientsError({ status: 404, reason: "unknown_route" });
71+
const [, key, uploadId, part] = match as unknown as [
72+
string,
73+
string,
74+
string | undefined,
75+
string | undefined,
76+
];
77+
const operation =
78+
uploadId === undefined
79+
? url.pathname.endsWith("/uploads")
80+
? "create"
81+
: "download"
82+
: part === undefined
83+
? "abort"
84+
: part === "complete"
85+
? "complete"
86+
: "part";
87+
const method = {
88+
download: "GET",
89+
create: "POST",
90+
part: "PUT",
91+
complete: "POST",
92+
abort: "DELETE",
93+
}[operation];
94+
yield* Effect.annotateCurrentSpan({ "native.operation": operation, "native.key": key });
95+
if (request.method !== method)
96+
return yield* new NativeClientsError({ status: 405, reason: "method_not_allowed" });
97+
if (!request.headers.has("authorization"))
98+
return yield* new NativeClientsError({ status: 401, reason: "missing_identity" });
99+
100+
const developer = yield* developerIdentity(request, bindings.developerKeys);
101+
const writing = operation !== "download";
102+
if (writing) {
103+
if (typeof developer !== "string")
104+
return yield* new NativeClientsError({ status: 403, reason: "write_denied" });
105+
} else if (
106+
typeof developer !== "string" &&
107+
typeof (yield* developerIdentity(request, bindings.readerKeys)) !== "string"
108+
) {
109+
return yield* new NativeClientsError({ status: 403, reason: "read_denied" });
110+
}
111+
yield* limit(writing);
112+
113+
if (operation === "download") {
114+
const object = yield* r2(() => bindings.CLIENTS.get(key));
115+
if (!object) return yield* new NativeClientsError({ status: 404, reason: "missing" });
116+
return new Response(object.body as unknown as ReadableStream, {
117+
headers: {
118+
"Content-Type": "application/octet-stream",
119+
"Content-Length": String(object.size),
120+
"Cache-Control": "no-store",
121+
"X-T3-Digest": object.customMetadata?.digest ?? "",
122+
},
123+
});
124+
}
125+
if (operation === "create") {
126+
const digest = request.headers.get("x-t3-digest") ?? "";
127+
if (!digestPattern.test(digest))
128+
return yield* new NativeClientsError({ status: 400, reason: "invalid_digest" });
129+
if (yield* r2(() => bindings.CLIENTS.head(key))) return Response.json({ exists: true });
130+
const upload = yield* r2(() =>
131+
bindings.CLIENTS.createMultipartUpload(key, {
132+
customMetadata: { digest, writer: developer! },
133+
}),
134+
);
135+
return Response.json({ uploadId: upload.uploadId });
136+
}
137+
138+
const upload = bindings.CLIENTS.resumeMultipartUpload(key, decodeURIComponent(uploadId!));
139+
if (operation === "abort") {
140+
yield* r2(() => upload.abort());
141+
return new Response(null, { status: 204 });
142+
}
143+
if (operation === "part") {
144+
if (!request.body || !request.headers.has("content-length"))
145+
return yield* new NativeClientsError({ status: 411, reason: "length_required" });
146+
const uploaded = yield* r2(() =>
147+
upload.uploadPart(Number(part), request.body as unknown as Cloudflare.ReadableStream),
148+
);
149+
return Response.json({ etag: uploaded.etag });
150+
}
151+
const body = yield* decodeComplete(
152+
yield* r2(() => request.text().then((text) => text.slice(0, 64 * 1024))),
153+
).pipe(
154+
Effect.mapError(() => new NativeClientsError({ status: 400, reason: "invalid_parts" })),
155+
);
156+
yield* r2(() => upload.complete([...body.parts]));
157+
return new Response(null, { status: 204 });
158+
},
159+
Effect.catchTags({
160+
NativeClientsError: (error) =>
161+
Effect.gen(function* () {
162+
if (error.status >= 500)
163+
yield* Effect.logWarning("Native client request failed", { reason: error.reason });
164+
return Response.json({ error: error.reason }, { status: error.status });
165+
}),
166+
}),
167+
);
168+
169+
return NativeClients.of({ handle });
170+
});
171+
172+
export const layer = Layer.effect(NativeClients, make);

‎infra/task-cache/src/resources.ts‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,22 @@ export const Artifacts = cacheName.pipe(
3636
),
3737
);
3838

39+
// Fingerprint-keyed simulator/emulator dev clients; separate from the 9-day task-cache generations.
40+
export const NativeClients = cacheName.pipe(
41+
Effect.flatMap((name) =>
42+
Cloudflare.R2.Bucket("NativeClients", {
43+
name: `${name}-native-clients`,
44+
lifecycleRules: [
45+
{
46+
id: "native-client-retention",
47+
deleteObjectsTransition: { condition: { type: "Age", maxAge: 30 * 86400 } },
48+
abortMultipartUploadsTransition: { condition: { type: "Age", maxAge: 86400 } },
49+
},
50+
],
51+
}).pipe(retain()),
52+
),
53+
);
54+
3955
export const Savings = cacheName.pipe(
4056
Effect.flatMap((name) =>
4157
Cloudflare.D1.Database("Savings", {

0 commit comments

Comments
 (0)