Skip to content

Commit 7a849c5

Browse files
committed
A ChatGPT plan's requests name their session by Codex's session headers
A plan's requests read almost nothing from OpenAI's prompt cache: sessions 0eb4973f, 8facf414 and 0d89bd79 (gpt-6.1-sol, plan credential) read 0.9%, 5.6% and 8.1% of their prompt tokens from it, while dc3eb569, the same model at the same address with an API key, read 94%. nodwyn replay --window rebuilt every request of 0d89bd79 to its recorded hash, each its predecessor with messages appended, so the prefix was stable; only the credential differed. The plan's hits were partial older prefixes in 128-token multiples, the shape of replicas with their own caches. opencode found the same miss with a plan's token at api.openai.com/v1 (anomalyco/opencode#53765, 3.44%) and now sends the headers Codex names its session by, session-id, thread-id and x-client-request-id (anomalyco/opencode#53767). A request whose credential was granted chatgpt.tokens.use.direct now carries the session's cache key as those headers too, unless cache_key is off; an API key's requests are unchanged. TestPlanRequestsCarryCodexSessionHeaders fails on the unchanged code and passes here; make validate passes with no warnings. Whether the plan's server routes by these headers shows only in the next plan session's cache rate (ADR 9, amended 2026-10-07).
1 parent 49ae307 commit 7a849c5

7 files changed

Lines changed: 133 additions & 5 deletions

File tree

‎docs/adr/0009-context-and-cache-discipline.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -419,3 +419,43 @@ The RPC and log formats are unchanged; no usage is inferred for providers
419419
or older logs that do not report it. Browser regressions cover weighted
420420
aggregation, unknown versus zero, live replies, reopening, retained
421421
history and aggregation reuse over a large log, using fake providers only.
422+
423+
## Amendment 2026-10-07: a ChatGPT plan's requests name their session as Codex does
424+
425+
A ChatGPT plan's requests read almost nothing from the cache. Three
426+
sessions with gpt-6.1-sol on a plan's credential (20261007T182910Z-0eb4973f,
427+
20261007T181306Z-8facf414 and 20261007T191149Z-0d89bd79) read 0.9%, 5.6% and
428+
8.1% of their prompt tokens from it; 20261007T183700Z-dc3eb569, the same
429+
model at the same address with an API key, read 94%, each request reading
430+
its predecessor's whole prompt but three tokens. Rebuilt with `nodwyn replay
431+
--window`, every request of 0d89bd79 matched its recorded hash and was its
432+
predecessor with messages appended, under the same system prompt and 29
433+
tools, so the prefix the cache needs was there. The plan's hits were partial
434+
and matched older requests, in multiples of 128 tokens (2,560, 6,784, 24,448
435+
and 56,704): what replicas with caches of their own look like, the shape this
436+
ADR's 2026-10-02 amendment saw at OpenRouter. The plan's replies also took two
437+
to five times as long.
438+
439+
The requests differed only in their credential: both carried the session's
440+
`prompt_cache_key` and headers. Codex, signed in to the same plan, read 95%
441+
of a 257-reply session's prompt tokens from the cache that day, but over its
442+
own endpoint (`chatgpt.com/backend-api/codex`), which a plan's token may not
443+
reach (`checkOpenAISink`). opencode reported the same miss with a plan's token at
444+
`api.openai.com/v1` (3.44%, anomalyco/opencode#53765) and now sends the
445+
session headers Codex sends there, `session-id`, `thread-id` and
446+
`x-client-request-id`, which a maintainer saw raise the hits
447+
(anomalyco/opencode#53767). Codex dropped the underscored `session_id` for
448+
the hyphenated name (anomalyco/opencode#29471).
449+
450+
A request whose credential was granted ChatGPT plan usage
451+
(`chatgpt.tokens.use.direct`, a scope the keychain requires of a plan's
452+
sign-in) now carries the session's cache key as those three headers as well,
453+
whenever it carries the key at all: `cache_key` `session` or `header`, never
454+
`off`. The scope comes from the token response the keychain stored, not from
455+
a file in the repository, and the headers say only what `prompt_cache_key`
456+
already says. An API key's requests, which the server caches, are unchanged,
457+
as are the provider-neutral request, its hash and replay. The test is the
458+
cached share of the next plan session's append-only requests. If it stays
459+
low, what Codex and opencode also send back, the reply's encrypted reasoning
460+
(`include: ["reasoning.encrypted_content"]`), is the remaining difference;
461+
that is not built.

‎docs/book/source/catalogue.tex‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ \part{The source catalogue}\label{part:source-catalogue}
44

55
\chapter{Coverage and reading conventions}
66

7-
This inventory includes 230 of 456 tracked Go files: every non-test Go file, 51,445 of 94,468 Go source lines (54.5\% by lines). It also includes all 35 tracked Scheme files (2,979 lines). Counts include comments and blank lines; Go test files are omitted, not silently shortened.
7+
This inventory includes 230 of 456 tracked Go files: every non-test Go file, 51,467 of 94,552 Go source lines (54.4\% by lines). It also includes all 35 tracked Scheme files (2,979 lines). Counts include comments and blank lines; Go test files are omitted, not silently shortened.
88

99
Each file is printed in full from the checkout at build time, including build constraints, imports and comments. The navigation map in Part~\ref{part:source-map} supplies routes through these listings. Some Unicode symbols are rendered with equivalent typesetting glyphs; consult the source file for exact bytes.
1010

‎docs/manual.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1789,7 +1789,11 @@ gets them. Each request to a chat-completions server therefore names its
17891789
session: a key derived from the session id, the same for every request of
17901790
the session and resumed ones, saying nothing else about it. It goes as
17911791
OpenRouter's `X-Session-Id` header (Baseten's `X-Session-Affinity` carries the
1792-
same key) and as `prompt_cache_key`, the field OpenAI routes by. A server that
1792+
same key) and as `prompt_cache_key`, the field OpenAI routes by. A ChatGPT
1793+
plan's requests also carry it as the `session-id`, `thread-id` and
1794+
`x-client-request-id` headers Codex sends: with `prompt_cache_key` alone,
1795+
the plan's server read under a tenth of a session's prompt from its cache,
1796+
where an API key's same requests read nineteen twentieths. A server that
17931797
refuses fields it does not know takes the headers alone, and `off` sends
17941798
none:
17951799

‎internal/config/config.go‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -210,7 +210,9 @@ type Model struct {
210210
// holds its prompt cache (ADR 9): "session" (the default) sends a key
211211
// derived from the session id as the X-Session-Id header and as
212212
// prompt_cache_key, "header" only as the header, for a server that
213-
// refuses fields it does not know, and "off" sends neither. openai only.
213+
// refuses fields it does not know, and "off" sends neither. A ChatGPT
214+
// plan's requests carry it as Codex's session headers too, unless "off"
215+
// (ADR 9, amended 2026-10-07). openai only.
214216
CacheKey string `toml:"cache_key"`
215217
// CacheControl asks a chat-completions provider to cache the prompt:
216218
// "ephemeral" for the provider's own lifetime (five minutes) or "1h". A

‎internal/core/credentials.go‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ import (
99
"fmt"
1010
"net/http"
1111
"net/url"
12+
"slices"
1213
"strings"
1314
"time"
1415

@@ -293,6 +294,9 @@ func (s *Session) endpoint(m config.Model, effort string) loop.Endpoint {
293294
if m.CacheKey != config.CacheKeyOff {
294295
oai.CacheKey = cacheKey(s.id)
295296
oai.CacheKeyHeaderOnly = m.CacheKey == config.CacheKeyHeader
297+
// A ChatGPT plan's token is the one granted plan usage, a scope
298+
// the keychain requires of a plan sign-in.
299+
oai.CodexHeaders = m.Credential != "" && slices.Contains(s.credentialScopes(m.Credential, ""), "chatgpt.tokens.use.direct")
296300
}
297301
p = oai
298302
case "anthropic":

‎internal/core/models_test.go‎

Lines changed: 64 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@ package core
22

33
import (
44
"bytes"
5+
"context"
56
"encoding/json"
67
"io"
78
"net/http"
@@ -21,6 +22,7 @@ import (
2122
"github.com/fclaude/nodwyn/internal/llm"
2223
"github.com/fclaude/nodwyn/internal/log"
2324
"github.com/fclaude/nodwyn/internal/window"
25+
fakeoauth "github.com/fclaude/nodwyn/test/fakes/oauth"
2426
fake "github.com/fclaude/nodwyn/test/fakes/ollama"
2527
)
2628

@@ -33,7 +35,10 @@ type fakeOpenAI struct {
3335
requests []seenRequest
3436
}
3537

36-
type seenRequest struct{ body, auth, session string }
38+
type seenRequest struct {
39+
body, auth, session string
40+
header http.Header
41+
}
3742

3843
// oaiText and oaiCall are assistant messages in the protocol's JSON.
3944
func oaiText(s string) string { return `{"role":"assistant","content":` + quote(s) + `}` }
@@ -48,7 +53,7 @@ func newFakeOpenAI(t *testing.T, replies ...string) *fakeOpenAI {
4853
body, _ := io.ReadAll(r.Body)
4954
f.mu.Lock()
5055
defer f.mu.Unlock()
51-
f.requests = append(f.requests, seenRequest{string(body), r.Header.Get("Authorization"), r.Header.Get("X-Session-Id")})
56+
f.requests = append(f.requests, seenRequest{string(body), r.Header.Get("Authorization"), r.Header.Get("X-Session-Id"), r.Header.Clone()})
5257
if len(f.replies) == 0 {
5358
http.Error(w, `{"error":"script exhausted"}`, http.StatusInternalServerError)
5459
return
@@ -169,6 +174,63 @@ func TestModelRequestsCarryTheSessionsCacheKey(t *testing.T) {
169174
}
170175
}
171176

177+
// A ChatGPT plan's requests name the session by the headers Codex sends as
178+
// well, since by prompt_cache_key alone the plan's server read almost none of
179+
// a session from its cache. An API key's requests, which it does cache, are
180+
// as they were, and cache_key = "off" sends neither (ADR 9, amended
181+
// 2026-10-07).
182+
func TestPlanRequestsCarryCodexSessionHeaders(t *testing.T) {
183+
t.Parallel()
184+
issuer := fakeoauth.New("nodwyn-test")
185+
t.Cleanup(issuer.Close)
186+
plan := []string{"openid", "resource.invoke", "chatgpt.tokens.use.direct"}
187+
for _, tc := range []struct {
188+
credential, setting string
189+
named bool
190+
}{
191+
{"plan", "", true},
192+
{"plan", config.CacheKeyHeader, true},
193+
{"plan", config.CacheKeyOff, false},
194+
{"key", "", false},
195+
} {
196+
kc, err := keychain.Create(filepath.Join(t.TempDir(), "kc"), keychain.FastKDF, []byte("p"))
197+
if err != nil {
198+
t.Fatal(err)
199+
}
200+
g := keychain.Grant{Profile: keychain.Custom(issuer.AuthURL(), issuer.TokenURL(), ""), ClientID: issuer.ClientID, Scopes: plan}
201+
if _, err := kc.AddOAuth2(context.Background(), "plan", g, issuer.Browser); err != nil {
202+
t.Fatal(err)
203+
}
204+
if _, err := kc.Add("key", []byte("sk-test-key")); err != nil {
205+
t.Fatal(err)
206+
}
207+
table := binding.Table{Bind: []binding.Binding{
208+
{Class: "plan", Dest: "127.0.0.1", Positions: []string{"header:Authorization:Bearer"}, Methods: []string{"POST"}, Scopes: plan},
209+
{Class: "key", Dest: "127.0.0.1", Positions: []string{"header:Authorization:Bearer"}, Methods: []string{"POST"}},
210+
}}
211+
impl := newFakeOpenAI(t, oaiText("hello"))
212+
r := modelRig(t, `(define (agent-turn request) (model-turn "impl"))`,
213+
map[string]config.Model{"impl": {Provider: "openai", URL: impl.URL + "/v1", Model: "m", Credential: tc.credential, CacheKey: tc.setting}},
214+
Options{Keychain: kc, Bindings: table})
215+
if e := r.turn("hi"); e != "" {
216+
t.Fatal(e)
217+
}
218+
want := ""
219+
if tc.named {
220+
want = cacheKey(r.s.ID())
221+
}
222+
seen := impl.seen()
223+
if len(seen) != 1 || seen[0].auth == "" {
224+
t.Fatalf("%s, cache_key %q: requests %+v", tc.credential, tc.setting, seen)
225+
}
226+
for _, name := range []string{"session-id", "thread-id", "x-client-request-id"} {
227+
if got := seen[0].header.Get(name); got != want {
228+
t.Errorf("%s, cache_key %q: %s %q, want %q", tc.credential, tc.setting, name, got, want)
229+
}
230+
}
231+
}
232+
}
233+
172234
func TestModelTurnUnknownName(t *testing.T) {
173235
t.Parallel()
174236
r := modelRig(t, `(define (agent-turn request) (model-turn "gpt-9"))`,

‎internal/llm/openai.go‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,12 @@ type OpenAI struct {
4040
// OpenRouter also reads). "" sends none.
4141
CacheKey string
4242
CacheKeyHeaderOnly bool
43+
// CodexHeaders sends CacheKey also as the session-id, thread-id and
44+
// x-client-request-id headers Codex sends, for a ChatGPT plan's requests:
45+
// with prompt_cache_key alone, the plan's server read 1% to 8% of a
46+
// session's prompt tokens from its cache where an API key's read 94% of
47+
// the same requests (ADR 9, amended 2026-10-07).
48+
CodexHeaders bool
4349
// CacheControl asks the provider to cache the prompt, as the request's
4450
// top-level cache_control: "ephemeral" or "1h". A Claude model reached
4551
// through OpenRouter, Bedrock or Vertex reads and writes its cache only
@@ -441,6 +447,11 @@ func cacheAsk(setting string) *oaiCacheControl {
441447
return &oaiCacheControl{Type: setting}
442448
}
443449

450+
// codexHeaders are the headers Codex names its session by, hyphenated as it
451+
// now sends them; opencode sends the same three with a plan's token
452+
// (anomalyco/opencode#53767).
453+
var codexHeaders = []string{"session-id", "thread-id", "x-client-request-id"}
454+
444455
// send makes one authorized request and returns a 200 response. A redirect
445456
// is an error (NoRedirect), and a permanent one: asking again gets the same
446457
// answer. A request with a body, a chat completion, carries the cache key.
@@ -456,6 +467,11 @@ func (o *OpenAI) send(ctx context.Context, method, path string, body []byte) (*h
456467
// Baseten routes by the same key under its own name; a server
457468
// that knows neither header ignores them.
458469
hreq.Header.Set("X-Session-Affinity", o.CacheKey)
470+
if o.CodexHeaders {
471+
for _, name := range codexHeaders {
472+
hreq.Header.Set(name, o.CacheKey)
473+
}
474+
}
459475
}
460476
}
461477
if o.Authorize != nil {

0 commit comments

Comments
 (0)