feat: add data-mask plugin#13347
Merged
Merged
Conversation
The data-mask plugin masks sensitive fields in request data (query parameters, headers, and body) before they appear in access logs or logger plugins. It supports three masking actions: - remove: completely remove the field - replace: replace the field value with a fixed string - regex: apply a regex substitution to the field value For body masking, it supports both JSON (via JSONPath) and URL-encoded formats. Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
AlinsRan
requested review from
Baoyuantop,
membphis,
nic-6443 and
shreemaan-abhishek
May 12, 2026 09:06
nic-6443
previously approved these changes
May 12, 2026
…king The data-mask plugin writes masked query params to ctx.var.request_line so that access logs reflect the redacted request line. Three supporting changes are required: - apisix/cli/ngx_tpl.lua: declare $request_line nginx variable in both http server blocks so nginx knows the variable exists - apisix/core/ctx.lua: register request_line as a writable ctx.var field - apisix/init.lua: copy the read-only $request nginx variable to the writable $request_line at the start of the access phase Also remove the EE-specific uri_before_strip fallback in data-mask.lua (strip_path_prefix is an EE-only feature; upstream always uses ctx.var.uri). Co-authored-by: Copilot <[email protected]>
Add two new tests: - TEST 15: create a route with data-mask removing 'password' and replacing 'token' query parameters - TEST 16: verify that the access log records the masked request line (password absent, token replaced with *****) via $request_line Also update the main log_format in t/APISIX.pm to use $request_line instead of $request, matching the EE configuration. $request_line is a writable copy of the read-only $request variable initialized in the access phase so that data-mask (and other plugins) can update it. Co-authored-by: Copilot <[email protected]>
- Add `set $request_line ''` declarations to server blocks in t/APISIX.pm to fix nginx startup failure (unknown variable error) - Add warn log when body masking is skipped due to max_body_size limit - Use `uri_before_strip or uri` for request_uri to handle proxy-rewrite URI stripping correctly Co-authored-by: Copilot <[email protected]>
get_http_version() returns a number (e.g. 1.1), not the string "HTTP/1.1", so the prefix must be prepended explicitly. Co-authored-by: Copilot <[email protected]>
There was a problem hiding this comment.
Pull request overview
Adds a new data-mask plugin to redact/mask sensitive request data (query params, headers, body) before it is emitted by APISIX logging surfaces, and wires it into the default plugin list, docs, and test suite.
Changes:
- Introduces
apisix/plugins/data-mask.luaimplementing masking actions (remove/replace/regex) for query/header/body (JSONPath + urlencoded). - Adds
request_linectx var support so access log formats can log a maskable request line, and updates test harness log format accordingly. - Adds documentation (EN/ZH), config registration, and a comprehensive test file covering query/header/body + access-log masking.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
apisix/plugins/data-mask.lua |
New plugin implementation to redact request query/header/body for log outputs. |
apisix/init.lua |
Initializes ctx.var.request_line from $request so it can be overridden later for masked access-log output. |
apisix/core/ctx.lua |
Allows request_line to be stored in ctx.var. |
apisix/cli/ngx_tpl.lua |
Declares $request_line Nginx var in generated config so it can be set from Lua. |
apisix/cli/config.lua |
Adds data-mask to the default enabled plugin list. |
conf/config.yaml.example |
Registers data-mask in the example plugin priority list. |
docs/en/latest/plugins/data-mask.md |
English plugin documentation and usage examples. |
docs/zh/latest/plugins/data-mask.md |
Chinese plugin documentation and usage examples. |
docs/en/latest/config.json |
Adds the new plugin doc page to the English docs sidebar/config. |
docs/zh/latest/config.json |
Adds the new plugin doc page to the Chinese docs sidebar/config. |
t/plugin/data-mask.t |
New test coverage for query/header/body masking + access-log masking behavior. |
t/APISIX.pm |
Test harness log format switched to $request_line to validate access-log masking. |
t/admin/plugins.t |
Ensures data-mask is present in the admin plugins list test. |
Comments suppressed due to low confidence (1)
t/plugin/data-mask.t:664
- In the failure branch, this test prints
matchwhich is never defined (likely meant to printmatch10/match11or the request body). This makes debugging failures harder if the assertion fails.
local match10, err = ngx.re.match(log.request.body, "arg10=10")
local match11, err = ngx.re.match(log.request.body, "arg11=11")
os.remove("mask-urlencoded-body.log")
if match10 and not match11 then
ngx.say("success")
return
end
ngx.say(match)
ngx.say(err)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Do not log sensitive values in regex_replace error messages - Handle multi-value query/form params in mask_table (regex action) - Fix mask_json to skip regex on non-string values with a warning - Fix mask_json to not assign nil when regex_replace fails - Fix header regex action to not clear header when regex fails - Defer get_post_args until a urlencoded rule is actually matched - Add warn log when request body is stored in a temporary file - Add docs note about access_log_format $request_line requirement - Fix test: remove extra backtick in TEST 12 title - Fix test: replace undefined 'match' variable with meaningful debug output Co-authored-by: Copilot <[email protected]>
shreemaan-abhishek
previously approved these changes
May 13, 2026
- regex_replace() now returns substitution count n; callers only set masked=true when n > 0 to avoid false-positive masking when the regex does not match - mask_table(): add type guards before calling regex_replace() to handle valueless boolean query args (e.g. ?token) without crashing ngx.re.sub - get_post_args(): capture and log error return; warn on truncation - Header regex: only update header when substitution actually matched Co-authored-by: Copilot <[email protected]>
membphis
approved these changes
May 13, 2026
nic-6443
approved these changes
May 14, 2026
shreemaan-abhishek
approved these changes
May 14, 2026
wistefan
pushed a commit
to wistefan/apisix
that referenced
this pull request
Jun 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
data-maskplugin masks or redacts sensitive fields in HTTP request data — query parameters, headers, and body — before they appear in access logs or logger plugin outputs (e.g.file-logger,http-logger,kafka-logger).Problem
API gateways sit in front of every service and naturally observe all request data, including credentials, tokens, PII, and payment card numbers. When access logging is enabled, this sensitive data is routinely written to log files or shipped to log aggregation systems, creating compliance risk under GDPR, PCI-DSS, HIPAA, and similar regulations.
Currently, APISIX has no built-in mechanism to sanitize sensitive fields before logging. Operators must either:
Scenarios
Scenario 1 — Credential sanitization in query parameters
A legacy API accepts credentials as query parameters (e.g.
/api?password=secret&token=abc). With access logging enabled, these values appear in the request URI logged by every logger plugin.data-maskcan remove thepasswordfield and replace thetokenwith*****before any logger sees the request.Scenario 2 — Authorization header redaction
A service passes
Authorization: Bearer <token>on every request. Log pipelines shipped to third-party SIEM systems must not contain bearer tokens.data-maskremoves or redacts the header in the log phase; upstream communication is unaffected.Scenario 3 — Partial masking of payment card numbers
An e-commerce API receives card numbers in JSON bodies or query strings. PCI-DSS requires only the first 6 and last 4 digits to be visible.
data-mask'sregexaction can transform1234-5678-9012-3456→1234-****-****-3456using a single rule.Scenario 4 — Nested JSON body field masking via JSONPath
A batch API receives a JSON body with user records, each containing a
tokenand nestedcredit.card. JSONPath expressions ($.users[*].token,$.users[*].credit.card) let a single rule mask all matching nested fields without enumerating each one.Design
The plugin runs in the log phase and modifies the request context (
ctx.var) used by logger plugins. Upstream communication is unaffected — the plugin never alters the actual request sent to the upstream.Three masking actions are supported:
remove— deletes the field entirely from the logged datareplace— substitutes the field value with a fixed stringregex— applies anngx.re.subregex substitution; capture groups are available as$1,$2, etc.Three field types are supported:
query— URL query parametersheader— HTTP request headersbody— request body (jsonformat supports JSONPath;urlencodedfor form data)Example
Mask sensitive query parameters and log with
file-logger:Send a request:
curl "http://127.0.0.1:9080/anything?password=secret&token=abc123&card=1234-5678-9012-3456"The logged URI in
logs/access.logwill be:The
passwordfield is absent;tokenis replaced; card digits are partially masked.Changes
apisix/plugins/data-mask.lua: Plugin implementationconf/config.yaml.example: Register plugin at priority 1500apisix/cli/config.lua: Add to default plugin listdocs/en/latest/plugins/data-mask.md: English documentation with full examplesdocs/zh/latest/plugins/data-mask.md: Chinese documentationChecklist