TT-17297: support file based kv references#8267
Conversation
…rsing logic without base path provided
… to expose this arg
…e if its provided
…d update the docs
🎯 Recommended Merge TargetsBased on JIRA ticket TT-17297: Support File Based KV Referencing. Fix Version: Tyk 5.14.0
Required:
📋 Workflow
|
|
This PR introduces a file-based Key-Value (KV) provider to the Tyk Gateway. This feature enables the gateway to resolve configuration values and secrets directly from local files, which is particularly useful for securely consuming Kubernetes secrets mounted as volumes. It enhances security by allowing private key material to remain on data-plane pods instead of being stored in the management plane. Two distinct mechanisms for file-based referencing are implemented:
A new configuration section, Files Changed AnalysisThe changes introduce a new, self-contained file KV provider and integrate it into key parts of the gateway:
Architecture & Impact Assessment
VisualizationThis diagram illustrates the two distinct flows for resolving file-based secrets: graph TD
subgraph "Load-Time Resolution (Gateway/API Config)"
A[tyk.conf / api_definition.json] --|contains "file:///path/to/secret"|--> B{Config/API Loader};
B -- calls --> C[gateway.ResolveFileKV];
C -- reads --> D[Filesystem];
B -- injects value --> E[Live Gateway Config / API Spec];
end
subgraph "Request-Time Resolution (Middleware)"
F[HTTP Request] --> G{URL Rewrite Middleware};
G --|template has "$secret_file.key"|--> H[gateway.ReplaceTykVariables];
H -- calls --> C[gateway.ResolveFileKV];
H -- injects value --> I[Rewritten URL / Header];
end
Scope Discovery & Context Expansion
Metadata
Powered by Visor from Probelabs Last updated: 2026-06-15T06:12:16.540Z | Triggered by: pr_updated | Commit: 691325d 💡 TIP: You can chat with Visor using |
Security Issues (3)
Architecture Issues (2)
Performance Issues (1)
Security Issues (3)
✅ Quality Check PassedNo quality issues found – changes LGTM. Powered by Visor from Probelabs Last updated: 2026-06-15T06:12:14.028Z | Triggered by: pr_updated | Commit: 691325d 💡 TIP: You can chat with Visor using |
|
/visor
Once a file is read, subsequent reads hit the kernel's buffer cache — not the disk. For a small secret file (bytes to a few KB), that's microseconds. File reads are not the same as disk I/O in practice. |
|
/visor
using Replacer is a super overkill in our case |
…and restrict processing if it is
|
/visor
This is the way any other kv ref is resolved, no hard fail but skip and log |
|
/visor
False positive, the IsLocal was added on 1.20+ |
…mlinks due to race condition..
|
SentinelOne CNS Hardcoded Secret Detector SentinelOne CNS is a cloud-agnostic, agentless CSPM & CWPP solution that continuously detects and prevents vulnerabilities that have the highest probability of being exploited in Azure, AWS, Google Cloud, and Kubernetes. |
🚨 Jira Linter FailedCommit: The Jira linter failed to validate your PR. Please check the error details below: 🔍 Click to view error detailsNext Steps
This comment will be automatically deleted once the linter passes. |
|



Description
Ticket: https://tyktech.atlassian.net/browse/TT-17297
Adds a file-based KV provider to Tyk Gateway so that KV references in gateway config and API definitions can resolve to the contents of local files. This is the primary mechanism for consuming Kubernetes secrets mounted as volume files on data-plane pods - enabling certificates and tokens to be referenced by name without ever storing private key material in the management plane.
Related Issue
Part of a two-ticket solution:
TT-17297 (this PR) — file-based KV provider: file:// URI scheme for load-time resolution, $secret_file. prefix for runtime resolution
TT-17295 (dependency) — PEM string support in API definitions: allows a certificate field to contain the raw PEM content rather than a file path
The mTLS client certificate use case (K8s secret → API definition → mTLS handshake) requires both PRs to be merged. This PR resolves the KV reference to the PEM content; TT-17295 makes the gateway accept that PEM content string as a certificate.
Motivation and Context
Currently the management plane stores private key material for every TLS certificate used in mTLS. This creates a security risk by centralising sensitive keys. The proposed solution: the management plane stores only a reference (file://client.crt), and the data-plane gateway resolves it from a K8s secret mounted as a volume at runtime - the private key never leaves the pod.
How This Has Been Tested
NOTE:
DEMO
Given the api def with client cert added with using file:// syntax to allowed list. Then client send request with using valid cert, then cert is updated with new one and temporaly old cert is still valid until gateway is hot-reloaded. Then we trigger the hot-reload with api def update and old cert stopped working but new cert value was used without gateway process shutdown
Screen.Recording.2026-06-03.at.08.36.06.mov
Given the config file with file:// reference by using relative path
Screen.Recording.2026-06-03.at.13.27.27.mov
Given the config file with file:// reference by using absolute path
Screen.Recording.2026-06-03.at.14.00.41.mov
Given api def with rewrite URL pattern to add query param with file referencing
Screen.Recording.2026-06-03.at.13.55.00.mov
Types of changes
Checklist
TODO: VALIDATE DOCS WITH ANDY
Ticket Details
TT-17297
Generated at: 2026-06-03 12:17:15