Skip to content

bugfix: Fix otlp translator for foreign characters#15249

Merged
ArthurSens merged 2 commits intomainfrom
otlp-fix-sanitiza
Oct 31, 2024
Merged

bugfix: Fix otlp translator for foreign characters#15249
ArthurSens merged 2 commits intomainfrom
otlp-fix-sanitiza

Conversation

@ArthurSens
Copy link
Copy Markdown
Member

@ArthurSens ArthurSens commented Oct 30, 2024

Description

  • Foreign letters like japanese, russian, etc, aren't supported by Prometheus legacy characters set.
  • OTLP endpoint always convert all characters to the legacy set. I'm working on allowing all UTF8 but the default behavior will be converting to legacy still.
  • The bug is that foreign characters aren't being translated to _ even with no support for the whole UTF8 charset

The fix was originally proposed in #14974, but we're moving it to a separate PR to keep the scope of changes smaller.

In the end, I've also decided to go in a different direction than proposed in #14974 for this fix. I've looked up the unicode table and just choose to use IsUpper and IsLower instead of the whole Letter category.

@ArthurSens ArthurSens changed the title Otlp fix sanitiza bugfix: Fix otlp translator for colons and foreign characters Oct 30, 2024
@ArthurSens ArthurSens force-pushed the otlp-fix-sanitiza branch 2 times, most recently from 94bccd7 to 8df09f3 Compare October 30, 2024 22:01
@ArthurSens ArthurSens changed the title bugfix: Fix otlp translator for colons and foreign characters bugfix: Fix otlp translator for foreign characters Oct 30, 2024
@ArthurSens ArthurSens changed the title bugfix: Fix otlp translator for foreign characters bugfix: Fix otlp translator for colons and foreign characters Oct 31, 2024
@ArthurSens ArthurSens changed the title bugfix: Fix otlp translator for colons and foreign characters bugfix: Fix otlp translator for foreign characters Oct 31, 2024
@ArthurSens
Copy link
Copy Markdown
Member Author

Lol apologies for being so indecisive if I'm fixing colons in this PR or not 😅

@ArthurSens ArthurSens merged commit ccaff31 into main Oct 31, 2024
@ArthurSens ArthurSens deleted the otlp-fix-sanitiza branch October 31, 2024 22:05
@aknuds1
Copy link
Copy Markdown
Contributor

aknuds1 commented Nov 2, 2024

Can you add a changelog entry for the changes in this PR? I still lack the context as to why the changes are necessary, and what the consequences are.

I think the PR description should have contained all the context about the changes, instead of back-referencing a previous PR where they were not the main topic.

Copy link
Copy Markdown
Contributor

@aknuds1 aknuds1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest fixing some test typos.

{"label:with:colons", "label_with_colons"}, // Without UTF-8 support, colons are only allowed in metric names
{"LabelWithCapitalLetters", "LabelWithCapitalLetters"},
{"label!with&special$chars)", "label_with_special_chars_"},
{"label_with_foreign_characteres_字符", "label_with_foreign_characteres___"},
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{"label_with_foreign_characteres_字符", "label_with_foreign_characteres___"},
{"label_with_foreign_characters_字符", "label_with_foreign_characters___"},

require.Equal(t, ":foo::bar", BuildCompliantName(createCounter(":foo::bar", ""), "", false))
require.Equal(t, "foo_bar", BuildCompliantName(createGauge("foo.bar", "1"), "", false))
require.Equal(t, "system_io", BuildCompliantName(createCounter("system.io", "foo/bar"), "", false))
require.Equal(t, "metric_with___foreign_characteres", BuildCompliantName(createCounter("metric_with_字符_foreign_characteres", ""), "", false))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
require.Equal(t, "metric_with___foreign_characteres", BuildCompliantName(createCounter("metric_with_字符_foreign_characteres", ""), "", false))
require.Equal(t, "metric_with___foreign_characters", BuildCompliantName(createCounter("metric_with_字符_foreign_characters", ""), "", false))

@ArthurSens ArthurSens mentioned this pull request Nov 2, 2024
julienduchesne pushed a commit to julienduchesne/prometheus that referenced this pull request Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants