Skip to content

mtest: fix unencodable XML chars - #11977

Merged
dcbaker merged 1 commit into
mesonbuild:masterfrom
tristan957:xml-fix
Jul 13, 2023
Merged

dcbaker merged 1 commit into
mesonbuild:masterfrom
tristan957:xml-fix

Conversation

@tristan957

Copy link
Copy Markdown
Member

Replace unencodable XML chars with their printable representation, so that, xmllint can parse test outputs without error.

Closes #9894
Closes #10680

@tristan957
tristan957 requested a review from jpakkane as a code owner July 13, 2023 00:48
@tristan957
tristan957 force-pushed the xml-fix branch 8 times, most recently from 415b47f to 8c01bc7 Compare July 13, 2023 01:04
Replace unencodable XML chars with their printable representation, so
that, xmllint can parse test outputs without error.

Closes mesonbuild#9894

Co-authored-by: Tristan Partin <[email protected]>
@tristan957
tristan957 requested a review from eli-schwartz July 13, 2023 03:05
@tristan957 tristan957 added this to the 1.2.0 milestone Jul 13, 2023
@dcbaker
dcbaker merged commit bd3d2cf into mesonbuild:master Jul 13, 2023
@tristan957
tristan957 deleted the xml-fix branch July 13, 2023 16:39
@benoit-pierre

benoit-pierre commented Aug 14, 2023 •

Copy link
Copy Markdown
Contributor

This PR also changes the JSON and text logs (meson-logs/testlog.json & meson-logs/testlog.txt) and the meson test command standard output, breaking, for example, color output (console codes). Was that intended?

@tristan957

Copy link
Copy Markdown
Member Author

@benoit-pierre let me take a look because that is most certainly not intended

@benoit-pierre

Copy link
Copy Markdown
Contributor

Glad to hear, tentative patch:

 mesonbuild/mtest.py | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git i/mesonbuild/mtest.py w/mesonbuild/mtest.py
index eb56c42be..8d83d17bd 100644
--- i/mesonbuild/mtest.py
+++ w/mesonbuild/mtest.py
@@ -867,10 +867,10 @@ class JunitBuilder(TestLogger):
                     et.SubElement(testcase, 'system-out').text = subtest.explanation
             if test.stdo:
                 out = et.SubElement(suite, 'system-out')
-                out.text = test.stdo.rstrip()
+                out.text = replace_unencodable_xml_chars(test.stdo.rstrip())
             if test.stde:
                 err = et.SubElement(suite, 'system-err')
-                err.text = test.stde.rstrip()
+                err.text = replace_unencodable_xml_chars(test.stde.rstrip())
         else:
             if test.project not in self.suites:
                 suite = self.suites[test.project] = et.Element(
@@ -893,10 +893,10 @@ class JunitBuilder(TestLogger):
                 suite.attrib['failures'] = str(int(suite.attrib['failures']) + 1)
             if test.stdo:
                 out = et.SubElement(testcase, 'system-out')
-                out.text = test.stdo.rstrip()
+                out.text = replace_unencodable_xml_chars(test.stdo.rstrip())
             if test.stde:
                 err = et.SubElement(testcase, 'system-err')
-                err.text = test.stde.rstrip()
+                err.text = replace_unencodable_xml_chars(test.stde.rstrip())
 
     async def finish(self, harness: 'TestHarness') -> None:
         """Calculate total test counts and write out the xml result."""
@@ -1180,9 +1180,9 @@ def decode(stream: T.Union[None, bytes]) -> str:
     if stream is None:
         return ''
     try:
-        return replace_unencodable_xml_chars(stream.decode('utf-8'))
+        return stream.decode('utf-8')
     except UnicodeDecodeError:
-        return replace_unencodable_xml_chars(stream.decode('iso-8859-1', errors='ignore'))
+        return stream.decode('iso-8859-1', errors='ignore')
 
 async def read_decode(reader: asyncio.StreamReader,
                       queue: T.Optional['asyncio.Queue[T.Optional[str]]'],

tristan957 added a commit to tristan957/meson that referenced this pull request Aug 14, 2023
This was an unintended consequence of the original patch in mesonbuild#11977.

Co-authored-by: Benoit Pierre <[email protected]>
tristan957 added a commit to tristan957/meson that referenced this pull request Aug 14, 2023
This was an unintended consequence of the original patch in mesonbuild#11977.

Co-authored-by: Benoit Pierre <[email protected]>
tristan957 added a commit to tristan957/meson that referenced this pull request Aug 14, 2023
This was an unintended consequence of the original patch in mesonbuild#11977.

Co-authored-by: Benoit Pierre <[email protected]>
tristan957 added a commit to tristan957/meson that referenced this pull request Aug 14, 2023
This was an unintended consequence of the original patch in mesonbuild#11977.

Co-authored-by: Benoit Pierre <[email protected]>
benoit-pierre added a commit to benoit-pierre/meson that referenced this pull request Aug 14, 2023
This was an unintended consequence of the original patch in mesonbuild#11977.

Co-authored-by: Benoit Pierre <[email protected]>
benoit-pierre added a commit to benoit-pierre/meson that referenced this pull request Aug 14, 2023
This was an unintended consequence of the original patch in mesonbuild#11977.

Co-authored-by: Benoit Pierre <[email protected]>
tristan957 added a commit to tristan957/meson that referenced this pull request Aug 14, 2023
This was an unintended consequence of the original patch in mesonbuild#11977.

Co-authored-by: Benoit Pierre <[email protected]>
xclaesse pushed a commit that referenced this pull request Aug 17, 2023
This was an unintended consequence of the original patch in #11977.

Co-authored-by: Benoit Pierre <[email protected]>
benoit-pierre added a commit to benoit-pierre/meson that referenced this pull request Aug 21, 2023
This was an unintended consequence of the original patch in mesonbuild#11977.

Co-authored-by: Benoit Pierre <[email protected]>
bruchar1 pushed a commit to bruchar1/meson that referenced this pull request Aug 25, 2023
This was an unintended consequence of the original patch in mesonbuild#11977.

Co-authored-by: Benoit Pierre <[email protected]>
nirbheek pushed a commit that referenced this pull request Sep 27, 2023
This was an unintended consequence of the original patch in #11977.

Co-authored-by: Benoit Pierre <[email protected]>
Volker-Weissmann pushed a commit to Volker-Weissmann/meson that referenced this pull request May 1, 2025
This was an unintended consequence of the original patch in mesonbuild#11977.

Co-authored-by: Benoit Pierre <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Invalid XML generated in testlog.junit.xml with \x07 in test output

5 participants