Repository navigation
[FEATURE]: /api/experimental/generate accepts only {prompt, model} while LLM.request already accepts system, messages and generation #49549
Description
The stateless generate endpoint is the only way a plugin can run a model call without
creating a session. It is useful exactly as designed, but it can only send a bare prompt on
default generation settings, and the layer directly beneath it already accepts more.
packages/protocol/src/groups/generate.ts:8-16 defines the payload as:
payload: Schema.Struct({
prompt: Schema.String,
model: Model.Ref.pipe(Schema.optional),
}),packages/core/src/generate.ts forwards exactly those two:
const response = yield* llm.generate(LLM.request({ model: resolved.model, prompt: input.prompt }))But LLM.request at packages/ai/src/llm.ts:42-62 destructures system, prompt, messages,
tools, toolChoice, generation, providerOptions and http. And prompt is itself sugar
for a single user message (llm.ts:59):
messages: [...(messages?.map(Message.make) ?? []), ...(prompt === undefined ? [] : [Message.user(prompt)])],The session path already exercises this. packages/core/src/session/model-request.ts:229-231
splits a prepared options object into generation (keys in GenerationOptions.fields) and
providerOptions, and attaches both at :252-253. So the capability is present and used; the
stateless entry simply does not offer it.
Request: widen the generate.text payload to accept messages? and generation?, forwarding
them to the LLM.request call that already accepts them. messages? rather than system?
because a real [system, user] pair is the general shape and prompt is the sugar over it.
Why it matters: a plugin that needs a system prompt today has to concatenate it into the user
turn. For anything calibrated against a fixed system/user split that is a behaviour change, not a
formatting preference. A downstream consumer building on this — the Magic Context plugin — runs a
hidden summarisation pass calibrated on a [system, user] shape at temperature 0.1 with a 32k
output cap; none of those three survive the current payload. Their wire capture against a
two-model mock provider confirms the request goes out with no instructions, no temperature and
no max_output_tokens.
I am happy to open a PR for this if the shape is agreeable.
Separately, and only if you want it answered here: is the /api/experimental/ prefix a
"may change" or a "will be removed" signal? session.import/session.export and
experimental.terminal live under it and look load-bearing, so I would rather ask than assume.
Activity
Trollgeir commentedon Oct 24, 2026
I maintain Lacuna, a plugin that summarizes consumed conversation without modifying session history. Accepting
messagesandgenerationwould also help this use case: it needs structured conversation input and control over generation settings, rather than a flattened prompt.Alongside those fields, these capabilities would be useful:
system. Session request preparation already keeps it separate frommessages; exposing the same shape would avoid reshaping captured context.One related consideration is prepared-request parity. For same-model summarization, reusing the session’s cached prefix matters. That requires preserving not only the messages but also the system, tool definitions, settings, and applicable metadata, then appending the summarization prompt. A way to reuse a captured, prepared request could address this separately; it needn’t block the smaller payload expansion proposed here.
Structured output would be a useful optional capability, but isn’t required for Lacuna.
opencodejr commentedon Oct 28, 2026
Thanks for the detailed write-up! You're asking for the stateless
generate.textendpoint to accept optionalmessagesandgenerationand pass them through toLLM.request, which already supports them. Plugins could then send a real[system, user]pair with settings such as temperature and an output token cap, instead of putting the system prompt into the user turn.I checked the code, and it matches your description: the payload in
packages/protocol/src/groups/generate.tsandTextInputinpackages/core/src/generate.tsonly carrypromptandmodel. A plugin can't work around this, because the limit is in the server endpoint's request schema. One thing to keep in mind if you do a PR: the same payload is also used by the deprecated/api/experimental/generatealias.Maintainers will consider the request and the proposed shape, and they're the best people to say how stable the
/api/experimental/prefix is meant to be. It may be worth waiting for their feedback on the shape before you open a PR.