Skip to content

fix(opentelemetry): coerce additional_attributes values to string#13146

Merged
shreemaan-abhishek merged 4 commits into
apache:masterfrom
janiussyafiq:fix/ot-handle-nonstring
Apr 7, 2026
Merged

fix(opentelemetry): coerce additional_attributes values to string#13146
shreemaan-abhishek merged 4 commits into
apache:masterfrom
janiussyafiq:fix/ot-handle-nonstring

Conversation

@janiussyafiq

Copy link
Copy Markdown
Contributor

Description

When using the opentelemetry plugin with additional_attributes that reference Nginx variables returning numeric values (e.g. request_time, upstream_response_time, bytes_sent), APISIX crashes with the following error during span export:

lua entry thread aborted: runtime error: .../opentelemetry/trace/exporter/otlp.lua: bad argument #2 to 'encode' (string expected for field 'string_value', got number)

This happens because inject_attributes() always calls attr.string(key, val) regardless of the actual Lua type of the value. The resty.ngxvar library returns Lua numbers for certain Nginx variables, and the OTLP protobuf encoder strictly expects a string for string_value, causing it to abort.

The fix wraps all values with tostring() before passing them to attr.string(). This is correct because Nginx variables are conceptually always strings — tostring() simply restores that invariant that resty.ngxvar breaks for numeric variables.

Which issue(s) this PR fixes:

Fixes #9921

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. bug Something isn't working labels Apr 2, 2026
@janiussyafiq
janiussyafiq force-pushed the fix/ot-handle-nonstring branch from a9adf61 to b2afa2e Compare April 3, 2026 03:23
Comment thread t/plugin/opentelemetry.t Outdated
--- response_body
opentracing
--- no_error_log
bad argument #2 to 'encode'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this specific error message and simply assert that there are no error logs at all. Because if a specific error message is set, problems may fail to be detected due to changes in the error message.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch! fixed

@shreemaan-abhishek
shreemaan-abhishek merged commit eb2eb86 into apache:master Apr 7, 2026
19 checks passed
@janiussyafiq
janiussyafiq deleted the fix/ot-handle-nonstring branch April 7, 2026 08:26
shreemaan-abhishek added a commit that referenced this pull request May 7, 2026
…ghten test (#13315)

Three follow-ups to #13146 (which started coercing additional_attributes
values to strings before sending them to OTel attr.string):

1. inject_attributes() gated insertion on `if val then`, which silently
   drops boolean false (Lua falsy) — regressing any user whose nginx
   variable resolves to a boolean false flag. Switch to
   `if val ~= nil then` so false survives.

2. ngx.req.get_headers() returns a Lua table for multi-value headers
   (e.g. multiple Set-Cookie / X-Forwarded-For entries). tostring()
   over a table emits 'table: 0x...' and that's what was landing in
   the span attribute. Extract a coerce_attr_value() helper that joins
   multi-value entries with ', ' and skips when the source is missing
   entirely.

3. TEST 22's response_body regex was qr/.*opentelemetry-lua.*/ which
   matches the OTLP exporter marker even when the numeric attributes
   are entirely absent from the export — i.e. the test would still
   pass with the regression #13146 was supposed to fix. Tighten it to
   assert each of request_time / bytes_sent appears as a stringValue.
   (upstream_response_time is intentionally not asserted: TEST 21
   serves /opentracing directly without proxying, so
   $upstream_response_time is nil and the plugin correctly omits the
   attribute.)

Signed-off-by: Shreemaan Abhishek <[email protected]>
Signed-off-by: Abhishek Choudhary <[email protected]>
wistefan pushed a commit to wistefan/apisix that referenced this pull request Jun 16, 2026
…ghten test (apache#13315)

Three follow-ups to apache#13146 (which started coercing additional_attributes
values to strings before sending them to OTel attr.string):

1. inject_attributes() gated insertion on `if val then`, which silently
   drops boolean false (Lua falsy) — regressing any user whose nginx
   variable resolves to a boolean false flag. Switch to
   `if val ~= nil then` so false survives.

2. ngx.req.get_headers() returns a Lua table for multi-value headers
   (e.g. multiple Set-Cookie / X-Forwarded-For entries). tostring()
   over a table emits 'table: 0x...' and that's what was landing in
   the span attribute. Extract a coerce_attr_value() helper that joins
   multi-value entries with ', ' and skips when the source is missing
   entirely.

3. TEST 22's response_body regex was qr/.*opentelemetry-lua.*/ which
   matches the OTLP exporter marker even when the numeric attributes
   are entirely absent from the export — i.e. the test would still
   pass with the regression apache#13146 was supposed to fix. Tighten it to
   assert each of request_time / bytes_sent appears as a stringValue.
   (upstream_response_time is intentionally not asserted: TEST 21
   serves /opentracing directly without proxying, so
   $upstream_response_time is nil and the plugin correctly omits the
   attribute.)

Signed-off-by: Shreemaan Abhishek <[email protected]>
Signed-off-by: Abhishek Choudhary <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

help request: OpenTelemetry plugin has problems handling Nginx variables of non-string type

4 participants