feat: extend secret references to all plugins with central resolution#13312
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR centralizes $secret:// / $env:// reference resolution so all plugins automatically receive resolved values, and updates docs/tests to reflect the new behavior.
Changes:
- Resolve secret references for all HTTP/stream plugin confs in
plugin.filter()/plugin.stream_filter()using a weak-key cache for stable resolved table identities. - Add secret-ref detection/collection helpers in
apisix.secret, and remove plugin-localfetch_secrets()calls from several plugins. - Bypass schema constraints for secret-ref fields during config load; add new tests and update secret terminology docs.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
apisix/plugin.lua |
Central secret resolution + cache; adds secret-aware schema validation bypass logic. |
apisix/secret.lua |
Adds is_secret_ref, has_secret_ref, and collect_secret_values helpers used by central resolver/validator. |
apisix/plugins/openid-connect.lua |
Removes per-plugin secret resolution in rewrite. |
apisix/plugins/limit-count.lua |
Removes per-plugin secret resolution in access. |
apisix/plugins/clickhouse-logger.lua |
Removes per-call secret resolution in log sending path. |
apisix/plugins/authz-keycloak.lua |
Removes per-plugin secret resolution in access. |
apisix/plugins/ai-aws-content-moderation.lua |
Removes per-plugin secret resolution in rewrite. |
t/secret/central-secret-refs.t |
Adds test coverage for centrally resolved secret refs and schema-bypass behavior. |
docs/en/latest/terminology/secret.md |
Documents secret refs working for any plugin + schema-bypass behavior. |
docs/zh/latest/terminology/secret.md |
Same as EN docs, for ZH. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
nic-6443
force-pushed
the
feat/central-secret-refs
branch
from
April 29, 2026 07:40
c6d6f54 to
39ae26b
Compare
Closed
5 tasks
Previously, only a handful of plugins (authz-keycloak, openid-connect, limit-count, clickhouse-logger, ai-aws-content-moderation) explicitly called fetch_secrets() to resolve $secret:// and $env:// references. Plugins that didn't call fetch_secrets() would silently ignore secret references, causing confusing failures. This commit makes secret references work automatically for ALL plugins by injecting resolution at the end of plugin.filter() and stream_filter(). Key design decisions: 1. Single injection point in plugin.filter() covers all code paths (HTTP requests, consumer merge, global rules, stream plugins). 2. Stable table references via weak-keyed cache: on most requests (no secret rotation), resolved values haven't changed, so the same resolved table reference is returned. This preserves plugin-internal caching that uses conf as cache key. 3. Schema validation is made secret-aware: when a plugin conf contains secret refs, the validator strips those fields from both conf and schema copies before checking, so constraints like enum/pattern/minLength/maxLength don't reject the $secret://... placeholder strings. 4. Remove explicit fetch_secrets() calls from 5 plugins that previously had them. Non-plugin call sites (consumer.lua, radixtree_sni.lua) are unchanged as they operate on different code paths.
- Extract strip_secret_refs as module-level function for reuse - Add secret-aware validation to stream_check_schema - Handle scalar array items (string arrays with secret refs) - Fix test: add no_shuffle(), remove unused require, improve assertions
Instead of bypassing plugin_obj.check_schema() entirely when secret refs are detected, move the strip logic into core.schema.check(). This preserves all custom code validation in each plugin's check_schema (regex_uri compilation, headers checks, group field validation, claim_schema, etc.) while only bypassing jsonschema constraints for the specific fields containing secret references.
- Handle anyOf/oneOf/allOf at property level for string type detection - Recurse into object-typed branches of anyOf/oneOf/allOf - Adjust minItems when stripping secret ref elements from arrays - Add warning log when secret resolution fails - Add code comment explaining stable reference cache rationale - Remove empty more_headers directive in test
- Adjust minProperties when stripping object properties (fixes proxy-rewrite and nested object validations) - Extend strip_required to handle if/then/else conditional schemas (fixes SSL cert/key secret ref validation)
openid-connect check_schema requires either bearer_only=true or session.secret to be set.
- Rename TEST 8/9 to match actual assertions - Add TEST 11: limit-count redis_host as env ref exercises if/then conditional required stripping - Renumber old TEST 11 to TEST 12
When core.schema.check() validates a copy of the json (to strip secret ref fields before jsonschema validation), the validator sets default values on the copy but not the original. This caused: - upstream.scheme defaulting to nil instead of 'http' (balancer crash) - plugin defaults like redis_database not being set - SSL and other resource defaults being lost Add merge_defaults() to copy newly-set fields from the validated copy back to the original json after successful validation.
…handle dependencies keyword - has_secret_ref and collect_secret_values now use a visited set to prevent stack overflow on tables with circular references - strip_required now recurses into 'dependencies' keyword to handle schemas like jwt-auth that use dependencies.field.oneOf[].required
Several plugins perform Lua-level validation in check_schema that would reject $secret:// or $env:// reference strings: - jwt-auth: base64 decode of secret - response-rewrite: base64 decode of body - proxy-rewrite: regex compilation of regex_uri elements - redirect: regex compilation of regex_uri elements - ai-proxy: JSON decode of gcp service_account_json - ai-proxy-multi: JSON decode of gcp service_account_json - openid-connect: jsonschema compilation of claim_schema - request-validation: schema validation of body_schema/header_schema Add is_secret_ref() guard before each check so secret reference strings pass through check_schema and get resolved at runtime.
…lidation hook Replace the complex schema stripping approach (deepcopy + strip_secret_refs + strip_required + merge_defaults, ~180 lines) with a single skip_validation hook in jsonschema. The hook tells the validator to accept secret reference strings as-is, bypassing type/enum/format/pattern constraints since they will be resolved at runtime. Also bump jsonschema dependency from 0.9.9 to 0.9.13 which adds the skip_validation feature.
The jsonschema skip_validation hook now passes (value, schema) instead of just (value). Update is_secret_ref to accept the schema parameter. Also fix stale test descriptions referencing old 'stripping' approach.
The skip_validation hook should only bypass validation when the schema expects a string type. For non-string fields (integer, boolean, etc.), secret ref strings are not allowed at schema validation time.
For string-typed schemas, only skip validation if the value would fail a constraint (enum, pattern, minLength, format). This prevents breaking oneOf validators (like SSL key/cert) where one branch already accepts secret ref patterns natively. For non-string schemas (integer, boolean, etc.), always skip since a string secret ref would fail the type check.
The jsonschema skip_validation hook now handles secret references automatically, making the oneOf workaround in SSL cert/key fields unnecessary. Simplify is_secret_ref to a straightforward prefix check. Also update secret docs to describe the full supported scope: - All plugin config string fields - SSL cert/key/certs/keys fields - Consumer auth config string fields
ed25519 private keys in PEM format are ~119 bytes, shorter than the previous minLength of 128. Lower to 64 for future compatibility.
Only skip validation when the schema declares type='string' or has no explicit type constraint. Fields with type='object', 'integer', etc. will no longer be bypassed even if the value looks like a secret ref.
In redirect.lua and proxy-rewrite.lua, still validate the regex pattern when only the replacement is a secret ref (using empty string as test replacement). Only skip validation when the pattern itself is a secret ref.
Workflow plugin calls action handlers directly, bypassing plugin.filter() where central secret resolution happens. Add fetch_secrets() in workflow handlers for limit-conn and limit-count to ensure secret refs are resolved when these plugins are used as workflow actions.
This reverts commit e31c069.
fetch_secrets() always returns a table when given a table input, so the nil check was unreachable dead code.
Add _no_secret_ref weak-keyed cache so configs without secret refs are only scanned once. Subsequent requests skip has_secret_ref() entirely.
nic-6443
force-pushed
the
feat/central-secret-refs
branch
from
April 30, 2026 08:22
33f0653 to
03bb120
Compare
nic-6443
requested review from
AlinsRan,
Baoyuantop,
membphis,
moonming and
shreemaan-abhishek
April 30, 2026 08:48
moonming
approved these changes
Apr 30, 2026
membphis
approved these changes
Apr 30, 2026
Baoyuantop
approved these changes
Apr 30, 2026
5 tasks
This was referenced Jun 3, 2026
This was referenced Jun 11, 2026
hiades-devops
pushed a commit
to hiades-devops/apisix
that referenced
this pull request
Jun 15, 2026
- Remove no-op ssl_trusted_certificate option from resty.http request (the api7 fork only honors ssl_verify/ssl_server_name/ssl_send_status_req); document that apisix.ssl.ssl_trusted_certificate must be set in config.yaml to the cluster CA for in-cluster TLS verification to work - Add optional 'endpoint' field to schema to allow a full base URL override (e.g. http://127.0.0.1:1984 for test mocks), replacing the hardcoded https:// - Fix tests 8-12: use endpoint instead of kubernetes_host/kubernetes_port so plain HTTP mock servers are reachable without a TLS handshake - Fix TEST 13: add service_account_file to manager registration so TEST 14 does not fall back to the non-existent in-cluster token path in CI - Revert authz-keycloak maxLength bump (100 -> 4096): the issue is already addressed on master since apache#13312 via skip_validation for secret refs
wistefan
pushed a commit
to wistefan/apisix
that referenced
this pull request
Jun 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Extend secret references (
$secret://,$env://) to work with all plugins, SSL certificates, and consumer auth configurations automatically, removing the need for each plugin to callfetch_secrets()individually.Changes
Central secret resolution in
plugin.filter(): Addedresolve_plugin_conf()with a weak-keyed cache that resolves all secret references in any plugin configuration before plugin execution. Removed explicitfetch_secrets()calls from jwt-auth, authz-keycloak, openid-connect, csrf, and limit-count plugins.jsonschema
skip_validationhook: Leverages the newskip_validation(value, schema)hook in jsonschema v0.9.13 to automatically bypass schema validation constraints (enum, pattern, minLength, maxLength, format) for$secret:///$env://values. This allows secret references in any string field without schema validation failures.Removed legacy SSL
oneOfworkaround: The SSL schema previously usedoneOf: [{certificate_scheme}, {secret_uri_schema}]to accept both real certificates and secret URIs. This workaround is no longer needed — the jsonschema hook handles it automatically. SSL cert/key/certs/keys fields now use plaincertificate_scheme/private_key_schemadirectly.check_schemaguards: Addedis_secret_refbypass in plugins with Lua-based validation that would reject$secret://values: response-rewrite, proxy-rewrite, redirect, ai-proxy, ai-proxy-multi, openid-connect, request-validation, jwt-auth.Documentation: Updated English and Chinese secret manager docs to describe the full supported scope (all plugin string fields, SSL cert/key, consumer auth fields).
Supported scope
cert,key,certs,keysfieldsFixes #13493