Skip to content

Commit 6a6bcc6

Browse files
authored
fix(devin-cli): harden the imported CLI session against path, read, and cache defects (#4418)
* docs(devlog): plan Devin hardening and the cached-token companion Locks the roadmap for the devin-cli token transition, the cloud-direct transport and usage decode, and the cached-token display companion, before any implementation lands. * docs(devlog): fold the roadmap audit findings into the Devin hardening plan An independent review of the first draft failed it on three counts. Field 7 needs its own uint64 decoder and must suppress field 28 within a message, because both arrive together and the adapter last-writes usage. The input-versus-cache mapping is derived from the frame rather than assuming Cognition is exclusive, since guessing wrong inflates input and bills cache at the uncached rate. And missing_ui_screenshot is path-based on gui/src, so the display phase carries a real screenshot instead of avoiding the word. * fix(devin-cli): harden the imported CLI session against path, read, and cache defects Five defects on the credential-import path. An empty APPDATA or XDG_DATA_HOME resolved to a cwd-relative credentials path, because `??` treats an empty string as a set value; a file planted beside the proxy would have imported as the operator's own CLI session. Every read failure collapsed into `undefined`, so a permission error on an existing file was reported as 'not signed in' and sent the operator to `devin auth login`, which does not fix it. The outcome is now missing, unreadable, incomplete or ok, each with its own message, and the parse is bounded at 64 KiB. A bare JWT in Metadata.api_key went out without the `devin-session-token$` prefix and came back as an opaque permission_denied, which reads as a revoked account rather than a malformed credential. Only a three-segment JWT is reshaped, so a Codeium UUID, an sk-ws key and a cog_ key pass through untouched. Logout cleared the shared user-JWT and catalog cache only for provider `devin`, and account deletion never cleared it, so a CLI-imported key's JWT outlived its own logout. Both ids now clear on both paths. redactSecretString recognised neither a Devin session token nor a bare JWT, and a Connect trailer can quote the request that carried one.
1 parent f5b2a0d commit 6a6bcc6

12 files changed

Lines changed: 533 additions & 30 deletions

File tree

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# 260912 — Devin hardening and cached-token display
2+
3+
## Why this unit exists
4+
5+
`devin-cli` landed as a working provider in `devlog/_fin/260912_devin_cli_account_login/`:
6+
a signed-in local Devin CLI credentials.toml is imported as an OAuth account, and inference
7+
goes to the Cognition cloud endpoint through the cloud-direct adapter rather than through an
8+
ACP stdio loop. That unit proved the path works. It did not harden it.
9+
10+
Two things are outstanding.
11+
12+
The first is the auth and transport path itself. The import reads one file with two regexes,
13+
the session token has no modelled expiry, and the cloud-direct client's failure classification
14+
is thin enough that an operator cannot tell a revoked credential from a rate limit from a
15+
protocol drift. The adapter decodes a reverse-engineered protobuf frame, and a truncated or
16+
reshaped frame is a class of failure the current code does not name.
17+
18+
The second is unrelated to Devin and was raised alongside it: a cached request's token total
19+
is displayed without its cached companion on several surfaces. The logs table already renders
20+
a total with a stacked cached line, and the surfaces that do not do this look like they are
21+
reporting a different number rather than the same number without its breakdown.
22+
23+
## Reference material
24+
25+
can1357/oh-my-pi carries an independent Devin provider implementation
26+
(packages/ai/src/providers/devin.ts, packages/ai/src/usage/devin.ts,
27+
packages/catalog/src/discovery/devin.ts, packages/catalog/src/wire/devin.ts) plus generated
28+
proto descriptors for the same Cognition surface. It is cloned read-only into .tmp/ref/oh-my-pi
29+
and is never vendored, imported, or copied: it is a second observation of the same wire
30+
protocol, used to decide which of our assumptions are load-bearing and which are guesses that
31+
happened to hold. Its open pull requests are read the same way.
32+
33+
## Work phases
34+
35+
| Phase | Doc | Scope |
36+
|---|---|---|
37+
| wp1 | this file plus 010/020/030/040 | Lock the roadmap. Docs only. |
38+
| wp2 | 010_cli_token_transition.md | CLI credential import and token transition hardening. |
39+
| wp3 | 020_cloud_direct_hardening.md | Cloud-direct transport, usage, and catalog hardening. |
40+
| wp4 | 030_cached_token_display.md | Cached companion on every total-bearing surface. |
41+
| wp5 | 040_stacked_delivery.md | Stacked PR chain, exact-head CI, merge into dev. |
42+
43+
wp2 and wp3 are sequential because they share src/oauth/devin/api-base.ts and the account
44+
record shape. wp4 is independent of both and touches only gui/src and src/cli, so it is a
45+
sibling branch in the stack rather than a child.
46+
47+
## Out of scope
48+
49+
- The Devin session product (cog_ keys, agent VMs). credentials.toml carries devin_webapp_host
50+
and devin_api_url for it; neither is inference and neither is read.
51+
- Any change to src/adapters/devin-cli/acp.ts stdio behaviour beyond failure classification.
52+
The cloud-direct route is the one that serves traffic.
53+
- Vendoring anything from the reference clone.
54+
55+
## Constraints carried into every later phase
56+
57+
- Bun-native TypeScript. No Node-only API that Bun does not implement.
58+
- bun run privacy:scan stays green. A devin session token is not recognised by
59+
redactSecretString, so no error path may echo a request body or a parsed credential.
60+
- Behaviour changes in src/ get a focused regression test next to the existing
61+
tests/providers/devin-*.test.ts files.
62+
- Every new test file needs an entry in scripts/test-layout/layout.json and
63+
tests/fixtures/test-layout-expected.json.
Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# wp2 — Devin CLI token transition hardening
2+
3+
Branch: codex/260912-devin-cli-token-transition (base dev)
4+
5+
## What the path does today
6+
7+
ocx login devin-cli reads credentials.toml from the CLI data dir, pulls windsurf_api_key and
8+
api_server_url with two line regexes, validates the host, and stores an OAuth account whose
9+
expiry is Number.MAX_SAFE_INTEGER and whose refresh throws invalid_grant. Inference then runs
10+
through the cloud-direct Connect client, not through core's OAuth replay path.
11+
12+
## Defects to fix
13+
14+
1. The session token prefix is never normalized. Every Cognition RPC expects
15+
devin-session-token$<JWT>. A credential arriving without it (OPENCODEX_DEVIN_TEST_TOKEN, a
16+
pasted bare JWT, a provider apiKey typed by hand) is sent verbatim and returns an opaque
17+
permission_denied, which reads as a revoked account rather than a malformed credential.
18+
oh-my-pi normalizes at the metadata boundary (packages/catalog/src/wire/devin.ts). We do
19+
not. Fix: one normalizer applied where Metadata.apiKey is built, plus a unit test.
20+
21+
2. An empty APPDATA or XDG_DATA_HOME resolves to a cwd-relative path.
22+
src/oauth/devin-cli.ts uses env.APPDATA ?? join(homedir(), ...), and "" is a set value, so
23+
join("", "devin", "credentials.toml") yields devin/credentials.toml relative to whatever
24+
directory the proxy runs in. A file planted there imports as the operator's CLI session.
25+
Fix: treat an empty or whitespace-only value as unset.
26+
27+
3. The credential file is read whole with no bound and every I/O failure collapses to
28+
"not signed in". EACCES, EISDIR, and a missing file are indistinguishable, so the one error
29+
message the caller owns cannot name the actual recovery step. Fix: cap the read, and
30+
separate missing from unreadable without putting file bytes into any thrown value.
31+
32+
4. Logout clears the shared user-JWT and catalog cache only for provider "devin".
33+
src/server/management/oauth-account-routes.ts gates the clear on that exact id, so logging
34+
out of devin-cli leaves a cached api_key-bearing JWT in process memory for its whole TTL,
35+
and account deletion never clears it at all. devin and devin-cli share the same cache.
36+
Fix: cover both provider ids on both paths.
37+
38+
5. A Connect EOS trailer message is echoed verbatim into the client error and /api/logs.
39+
The HTTP-status paths deliberately refuse to echo bodies because a Connect error can quote
40+
the request that carries the key; the trailer path then does the opposite. redactSecretString
41+
recognises neither devin-session-token$... nor a bare JWT. Fix: add both patterns to the
42+
redactor so anything that does reach a log is masked.
43+
44+
## Non-goals
45+
46+
The app.devin.ai PKCE CLI OAuth flow. The import path is the intended substitute and a second
47+
login protocol is its own unit. Also excluded: probing the key at import time, which changes
48+
login latency and deserves its own decision.
49+
50+
## Verification
51+
52+
bun test tests/providers/devin-cli-login.test.ts tests/providers/devin-cli-authmode-migration.test.ts tests/providers/devin-hardening.test.ts
53+
plus bun run privacy:scan.
Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# wp3 — Devin cloud-direct hardening
2+
3+
Branch: codex/260912-devin-cloud-direct-hardening (base codex/260912-devin-cli-token-transition)
4+
5+
## 1. Usage is decoded from the display field, not the usage field
6+
7+
This is the defect the user can see, and it is confirmed against the reference proto.
8+
9+
decodeUsageBlock in src/adapters/devin/cloud-direct/chat.ts treats GetChatMessageResponse
10+
field 28 as a usage block keyed by metric-id strings. In the Cognition schema carried by
11+
can1357/oh-my-pi:
12+
13+
GetChatMessageResponse.usage = 7 (ModelUsageStats)
14+
GetChatMessageResponse.response_dimension_groups = 28 (repeated ResponseDimensionGroup)
15+
16+
ModelUsageStats.input_tokens = 2 uint64 varint
17+
ModelUsageStats.output_tokens = 3 uint64 varint
18+
ModelUsageStats.cache_write_tokens = 4 uint64 varint
19+
ModelUsageStats.cache_read_tokens = 5 uint64 varint
20+
21+
Field 28 is not an older usage shape. It is the current display message:
22+
ResponseDimensionGroup is {title, dimensions}, and ResponseDimension.uid is field 5 — which is
23+
exactly the sub-field today's decoder reads as metric_id. So the existing decoder works by
24+
reading presentation rows whose uid happens to spell the metric, and it yields cache numbers
25+
only when the server chose to render cache rows. Field 7 carries them unconditionally.
26+
27+
Three consequences the first draft of this plan got wrong, corrected after audit:
28+
29+
- Field 7 is uint64 varints. The existing entry walker only descends length-delimited
30+
sub-messages and reads a fixed32 float, so it cannot read field 7 at all. Field 7 needs its
31+
own decoder.
32+
- "Decode both, field 7 wins" is not what decoding both produces. Both fields arrive in the
33+
same response and src/adapters/devin.ts replaces usage on every usage event, so a naive
34+
addition lets field 28 land last and win. Within one message, field 7 must suppress
35+
field 28 outright; field 28 stays only as the fallback for a message that carries no field 7.
36+
- The adapter must merge usage fields across events rather than replacing the object, so a
37+
later partial frame cannot zero an earlier input count.
38+
39+
## 2. Whether input_tokens already includes cache is not known, so do not assume it
40+
41+
This repository's convention is inclusive: inputTokens covers the whole prompt, cachedInputTokens
42+
is the read subset, and totalTokens is input + output with no cache added on top. Adapters split
43+
on what the wire gives them — anthropic.ts and kiro-events.ts fold cache into input because their
44+
wire format is exclusive, while openai-responses.ts passes input_tokens through because it is
45+
already inclusive.
46+
47+
oh-my-pi summing input + output + cacheRead + cacheWrite is evidence that Devin might be
48+
exclusive. It is not proof, and guessing wrong in the inclusive direction silently inflates
49+
input and bills cache at the uncached rate, because normalizeCostTokens only rejects
50+
read + write > input.
51+
52+
So the mapping is derived from the frame rather than assumed:
53+
54+
if (input >= cacheRead + cacheWrite) inputTokens = input // already inclusive
55+
else inputTokens = input + cacheRead + cacheWrite
56+
57+
Both branches converge on the right answer for the case that prompted this work — a 58k prompt
58+
that is 57k cache read and 1k fresh reads as 58k total with a 57k cached subset whichever
59+
convention the wire uses — and neither branch can produce read + write > input. The heuristic
60+
is written down in the code with that reasoning, and replaced with a fixed mapping the moment a
61+
live ModelUsageStats frame settles the question.
62+
63+
## 3. An HTTP status never reaches the classifier
64+
65+
CloudChatError is thrown as "GetChatMessage failed (HTTP <status>)" with no status field, so a
66+
401 on a revoked import is a generic adapter failure rather than an authentication error, and
67+
inferHttpStatusFromAdapterMessage turns an HTTP 429 into a 502 — which means core's failover
68+
never rotates or backs off. Fix: carry status on the error and map 401, 403, 429 and 5xx.
69+
70+
## 4. A client abort is reported as an upstream failure
71+
72+
The adapter emits "Devin turn was aborted." with no status, and isClientClosedMessage does not
73+
recognise that wording, so a cancelled turn infers 502. Fix: emit the phrase the classifier
74+
already knows, with status 499.
75+
76+
## Verification
77+
78+
bun test tests/providers/devin-adapter.test.ts tests/providers/devin-hardening.test.ts
Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# wp4 — Cached-token companion on every total
2+
3+
Branch: codex/260912-cached-token-companion (base dev, sibling of the Devin chain)
4+
5+
## The complaint
6+
7+
A cached request whose total is 58,000 tokens is about 57,000 cache-read plus 1,000 fresh.
8+
The Logs table row already renders that as a total with a stacked "c 5.7만". Every other
9+
surface prints a bare 5.8만, which reads as a different, smaller request rather than the same
10+
request with its breakdown hidden. The conversation-totals banner sits directly above rows
11+
that do show the companion, so the mismatch is visible in one screenshot.
12+
13+
## Where the data already is
14+
15+
/api/logs forwards the whole usage object, and /api/usage already emits cache on summary,
16+
models, providers and day-models. No backend change is needed. The loss is client-side, and it
17+
is not only the GUI row types: Usage's UsageModel and UsageProvider, the dashboard's
18+
UsageSummary30d, summarizeFilteredLogs in Logs.tsx, and the CLI's CostRow each drop the fields
19+
before they reach a renderer.
20+
21+
## Approach
22+
23+
One shared helper beside formatTokens in gui/src/format-tokens.ts:
24+
25+
formatTokensWithCache(total, cached, locale) -> "5.8만 c5.7만"
26+
27+
It returns the bare total when cached is undefined or zero. It does not hide the companion when
28+
cached equals the total: an all-cache turn with no fresh input is exactly the case worth
29+
showing, and suppressing it would blank the most cached request on the page. The "c" marker
30+
matches the existing logs.tokens.cacheRead label, which already reads "cache read (c)", so no
31+
new i18n key is needed.
32+
33+
Surfaces to convert, in order of how visible the mismatch is:
34+
35+
1. Logs conversation-totals banner — summarizeFilteredLogs also sums cacheSplit(entry).read.
36+
2. Usage per-model and per-provider token columns — widen the row types to keep the cache
37+
fields the API already sends.
38+
3. Dashboard 30-day total tile — widen UsageSummary30d the same way.
39+
4. CLI usage report provider/model/account rows, matching the summary line that already
40+
prints "cached N".
41+
42+
The log detail panel is deliberately left alone: it already has separate cache read and cache
43+
write cells, so stacking the companion onto its total would duplicate them.
44+
45+
## CI gate
46+
47+
missing_ui_screenshot in .github/scripts/pr-quality.cjs is path-based: touching gui/src trips
48+
it whether or not the description says "gui". This PR therefore carries a real screenshot of
49+
the changed surface, produced from a build of this branch served by a throwaway proxy instance
50+
on its own port and its own OPENCODEX_HOME, so the operator's running service is untouched.
51+
52+
## Verification
53+
54+
bun test for the formatter and the CLI report, plus bun run lint:gui.
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# wp5 — Stacked delivery
2+
3+
Four branches, each one PR, chained so a reviewer sees one concern at a time.
4+
5+
dev
6+
└── codex/260912-devin-cli-token-transition (wp2)
7+
└── codex/260912-devin-cloud-direct-hardening (wp3)
8+
dev
9+
└── codex/260912-cached-token-companion (wp4)
10+
11+
wp4 is a sibling of the Devin chain, not a child: it touches `gui/src` and `src/cli` only and
12+
shares no file with wp2 or wp3.
13+
14+
Rules carried from the repository:
15+
16+
- Every PR fills `.github/PULL_REQUEST_TEMPLATE.md` in full and targets its parent branch;
17+
children retarget to `dev` once the parent lands.
18+
- Pushes use `--no-verify`; the local product suite is not run. Remote CI on the exact final
19+
head is the evidence, and any skipped local check is labelled NOT RUN.
20+
- Merges into `dev` are serialized, parent first, and each child is rebased onto the moved
21+
parent before its own merge.
22+
- A PR whose title or description mentions `gui` needs a screenshot, so wp4's description
23+
avoids that word unless a screenshot is attached.
24+

‎src/adapters/devin/cloud-direct/metadata.ts‎

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,33 @@ const CLOUD_CHAT_OS = 'windows';
5555
*/
5656
const DEVICE_FINGERPRINT_BYTES = 366;
5757

58+
/** Prefix every Cognition session key carries in `Metadata.api_key`. */
59+
const DEVIN_SESSION_TOKEN_PREFIX = 'devin-session-token$';
60+
61+
/** A bare JWT: three base64url segments. Nothing else is reshaped. */
62+
const BARE_JWT_PATTERN = /^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]*$/;
63+
64+
/**
65+
* Restore the `devin-session-token$` prefix on a bare JWT.
66+
*
67+
* Cognition reads `Metadata.api_key` as a prefixed session token. A key that
68+
* arrives without the prefix — a JWT pasted into `apiKey` by hand, or one
69+
* copied out of the CLI's file without its prefix — is sent verbatim and comes
70+
* back as an opaque `permission_denied`, which reads as a revoked account
71+
* rather than as a malformed credential.
72+
*
73+
* Only a bare JWT is reshaped. The other key formats this field has carried are
74+
* not JWTs and must pass through untouched: a Codeium-classic bare UUID, an
75+
* `sk-ws-01-…` Windsurf key, and a `cog_…` session key would all break if they
76+
* were prefixed. Anything already containing `$` is left alone for the same
77+
* reason.
78+
*/
79+
export function normalizeDevinSessionToken(apiKey: string): string {
80+
const trimmed = apiKey.trim();
81+
if (!trimmed || trimmed.includes('$')) return apiKey;
82+
return BARE_JWT_PATTERN.test(trimmed) ? `${DEVIN_SESSION_TOKEN_PREFIX}${trimmed}` : apiKey;
83+
}
84+
5885
export interface MetadataInput {
5986
/** Persistent api_key from OAuth (`devin-session-token$<JWT>`). */
6087
apiKey: string;
@@ -100,12 +127,14 @@ function osString(): string {
100127
export function buildMetadata(input: MetadataInput): Buffer {
101128
const version = input.windsurfVersion ?? WINDSURF_VERSION_STRING;
102129
const os = input.osName ?? osString();
130+
// One boundary, so no caller has to remember the prefix rule.
131+
const apiKey = normalizeDevinSessionToken(input.apiKey);
103132
if (input.cloudChatShape) {
104133
const clientVersion = input.windsurfVersion ?? CLOUD_CHAT_CLIENT_VERSION;
105134
return Buffer.concat([
106135
encodeString(1, CLOUD_CHAT_CLIENT_NAME),
107136
encodeString(2, clientVersion),
108-
encodeString(3, input.apiKey),
137+
encodeString(3, apiKey),
109138
encodeString(4, 'en'),
110139
encodeString(5, input.osName ?? CLOUD_CHAT_OS),
111140
encodeString(7, clientVersion),
@@ -117,7 +146,7 @@ export function buildMetadata(input: MetadataInput): Buffer {
117146
const parts: Buffer[] = [
118147
encodeString(1, 'windsurf'), // ide_name
119148
encodeString(2, version), // extension_version
120-
encodeString(3, input.apiKey), // api_key
149+
encodeString(3, apiKey), // api_key
121150
encodeString(4, 'en'), // locale
122151
encodeString(5, os), // os
123152
encodeString(7, version), // ide_version

‎src/lib/redact.ts‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -252,6 +252,13 @@ const SECRET_VALUE_PATTERNS: Array<[RegExp, string]> = [
252252
[/((?:"(?:api[_-]?key|access[_-]?token|refresh[_-]?token|id[_-]?token|client[_-]?secret|refreshToken|accessToken|clientSecret|apiKey)"\s*:\s*"))([^"]+)(")/gi, `$1${REDACTED_SECRET}$3`],
253253
// Raw JSON "token" field values (Copilot token exchange bodies echo the credential here).
254254
[/(("token"\s*:\s*"))([^"]+)(")/gi, `$1${REDACTED_SECRET}$4`],
255+
// Cognition/Devin session keys, and the bare JWTs several providers hand out.
256+
// A Connect EOS trailer can quote the request that carried the key, and the
257+
// rules above only fire on a label — `Bearer`, `api_key=`, `"token":` — which
258+
// a quoted proto field does not have. `eyJ` is the base64url of `{"`, so the
259+
// JWT rule needs a real three-segment shape and does not match ordinary prose.
260+
[/\bdevin-session-token\$[A-Za-z0-9._~+/=-]{8,}/g, REDACTED_SECRET],
261+
[/\beyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]*/g, REDACTED_SECRET],
255262
[/\b(arn:aws:[A-Za-z0-9_-]+:[A-Za-z0-9-]*:\d{12}:[A-Za-z0-9_/:+=,.@-]+)\b/g, REDACTED_SECRET],
256263
];
257264

0 commit comments

Comments
 (0)