Repository navigation
docs(discovery): make the file discovery example run as written - #3681
Merged
Merged
Conversation
The example in "Running EPP with file discovery (no Kubernetes)" did not start with the v1.0.0 epp: the config had no apiVersion/kind header, and it listed single-profile-handler inside the scheduling profile, which only accepts filters, scorers and pickers. With a valid config, Envoy's plaintext epp cluster failed every request because the EPP serves ext_proc over TLS by default. Add the header, reference the scorer and picker from the profile, and start the EPP with --secure-serving=false. Also describe the startup log lines the EPP emits in this mode. Signed-off-by: YoungjuKim <[email protected]>
ahg-g
approved these changes
Oct 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind documentation
What this PR does / why we need it:
The walkthrough in
docs/discovery.md, "Running EPP with file discovery (no Kubernetes)", does not work as written with the v1.0.0epp. Following it with the v1.0.0epp, Envoy 1.33.2 and threellm-d-inference-simv1.0.0 backends on127.0.0.1:8001-8003(all in one Linux container), it fails in three places:apiVersion/kindheader, so the EPP exits:single-profile-handleralso takes no parameters (its factory ignores them), so thescorer/pickerparameters in the example have no effect.ext_procover TLS by default (server TLSlog line withtls: true), and theeppcluster in the Envoy config connects in plaintext. For 12 requests, Envoy reportedext_proc.streams_failed: 12anddownstream_rq_5xx: 12.Changes, all in that section:
apiVersion: llm-d.ai/v1andkind: EndpointPickerConfigto the config.single-profile-handlerentry. With one profile the EPP adds that handler itself; it shows up in theEffective raw configurationlog line.--secure-serving=falseto theeppcommand, with one paragraph on the TLS default and a link totls.md.watching endpoints file for changeswhenwatchFileistrue. The sentence now names that line.Tests:
127.0.0.1:8001-8003) start the EPP and Envoy. 30POST /v1/completionsrequests through:8080all return 200, spread over the three backends (10/11/9 by the simulator'sx-inference-podheader). Envoy reportsstreams_failed: 0.The runs were made on 2026-10-26 (UTC) with the v1.0.0 release
epp(commit b9393d1).docs/discovery.mdand the config loader, scheduler profile and--secure-servingcode are unchanged on main at 6e5f3f0 (2026-10-27), and the patch applies to it as is.make presubmitwas not run because this change only touches Markdown. The TLS path (Envoy with anUpstreamTlsContextto the EPP) was not tested.Which issue(s) this PR fixes:
None. This is a documentation correction. It is unrelated to #3615 / #3630 (file-discovery reload after atomic replace on Docker Desktop).
Release note (write
NONEif no user-facing change):AI disclosure: this change was prepared with the help of an AI coding assistant (Claude Code). Claude ran the reproduction and the checks above in a container in my home lab and drafted the diff and this description.
🤖 Generated with Claude Code