-
Notifications
You must be signed in to change notification settings - Fork 14.1k
Webui/server config defaults #18028
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Webui/server config defaults #18028
Conversation
|
It work |
allozaur
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking good on my end
ggerganov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can merge after rebase and green CI. @ngxson up to you
|
Also, you need to re-generate the server docs via |
|
I will apply the optimizations and rebase / test |
ngxson
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can be merged once you update the docs 👍
Add CLI arguments --webui-config (inline JSON) and --webui-config-file (file path) to configure WebUI default settings from server side. Backend changes: - Parse JSON once in server_context::load_model() for performance - Cache parsed config in webui_settings member (zero overhead on /props) - Add proper error handling in router mode with try/catch - Expose webui_settings in /props endpoint for both router and child modes Frontend changes: - Add 14 configurable WebUI settings via parameter sync - Add tests for webui settings extraction - Fix subpath support with base path in API calls Addresses feedback from @ngxson and @ggerganov
2977afc to
f93fee0
Compare
|
Docs regenerated. Fun fact: llama-gen-docs initializes my GPU just to write markdown😂 |
|
By the way, this works great! Thank you for implementing this and helping me finally get it through! @mashdragon you will be glad to see this! If I find the time, I can write up an example for the webui readme (in a separate PR perhaps). |

Make sure to read the contributing guidelines before submitting a PR
server/webui: add CLI support for configuring WebUI defaults
Summary
Implements issue #17940: adds '--webui-config-file' CLI argument and 'LLAMA_WEBUI_CONFIG' environment variable to configure WebUI default settings from the server side.
Problem
WebUI settings are hardcoded in the frontend with no server-side configuration. This causes problems:
Example: Issue #17940 requests ability to set 'pasteLongTextToFileLen: 0' by default, but currently every user must manually configure this in every new browser session.
The TODO that guided this implementation
While exploring the codebase, I found this intentional placeholder in 'server-models.cpp' (lines 6-16):
This empty JSON was left by the original developers as a clear signal for where webui settings should go. This PR completes that TODO.
Backend Changes (C++)
What was added
1. CLI argument ('common/arg.cpp')
2. Environment variable ('server.cpp')
3. JSON parsing logic ('server.cpp')
4. Expose in /props ('server-models.cpp' + 'server-context.cpp')
How it works
The C++ backend is completely agnostic: it doesn't know what settings exist. It just:
This means adding new WebUI settings never requires C++ changes.
Frontend Changes (TypeScript)
What was added
1. API type ('types/api.d.ts')
2. Syncable parameters ('services/parameter-sync.ts')
3. Store integration ('stores/server.svelte.ts' + 'stores/settings.svelte.ts')
4. Tests ('parameter-sync.spec.ts')
Settings intentionally excluded
These are user-specific and should NOT be server-configurable:
Priority Order
Server operators set defaults, but users keep control:
This means:
Usage Examples
Config file
Environment variable
LLAMA_WEBUI_CONFIG='{"pasteLongTextToFileLen":0}' ./llama-serverBoth (env var overrides file)
LLAMA_WEBUI_CONFIG='{"pdfAsImage":true}' \ ./llama-server --webui-config-file webui-defaults.jsonCheck it works
curl http://localhost:8080/props | jq .webui_settingsTesting
Backend
Frontend
Backward Compatibility
No breaking changes:
What this enables
Now server operators can:
And adding new configurable settings only requires TypeScript changes: no C++ recompilation needed.
Fixes #17940