Skip to content

tests: avoid "_csv.Error: field larger than field limit (131072)" error#38030

Merged
tavplubix merged 2 commits intoClickHouse:masterfrom
azat:tests-fix-parser
Jun 15, 2022
Merged

tests: avoid "_csv.Error: field larger than field limit (131072)" error#38030
tavplubix merged 2 commits intoClickHouse:masterfrom
azat:tests-fix-parser

Conversation

@azat
Copy link
Copy Markdown
Member

@azat azat commented Jun 13, 2022

Changelog category (leave one):

  • Not for changelog (changelog entry is not required)

Ignores tests stdout (and hence hung check stdout) to avoid too long lines in csv.

@robot-ch-test-poll robot-ch-test-poll added the pr-not-for-changelog This PR should not be mentioned in the changelog label Jun 13, 2022
@azat azat mentioned this pull request Jun 13, 2022
Merged
@azat azat force-pushed the tests-fix-parser branch from 589e900 to 35c0a12 Compare June 14, 2022 10:49
@tavplubix
Copy link
Copy Markdown
Member

tavplubix commented Jun 14, 2022

stderr may be quite long too...
It it possible to capture, for example, first 1KB of stderr and first 1KB of stdout? I was thinking about adding a column that contains part of test output to CI database and it would be really helpful

@tavplubix tavplubix self-assigned this Jun 14, 2022
@azat
Copy link
Copy Markdown
Member Author

azat commented Jun 14, 2022

stderr may be quite long too...

Actually both stderr and stdout are limited to 100 lines.

ClickHouse/tests/clickhouse-test

Line 739 in f03119e

open(self.stdout_file).read().splitlines()[:100]

And the main source of this error is hung check report, suppressing everything after stdout: suppresses it too.

It it possible to capture, for example, first 1KB of stderr and first 1KB of stdout? I was thinking about adding a column that contains part of test output to CI database and it would be really helpful

This is a great idea! OK, I will leave first 100 lines from the stdout/stderr and filter out hung check report.

@azat azat force-pushed the tests-fix-parser branch from 35c0a12 to fa77b69 Compare June 14, 2022 15:16
@azat
tests: avoid "_csv.Error: field larger than field limit (131072)" error
23494e9 
v2: apply black
v3: ignore only hung check report
v4: ignore everything after "Database: "
Signed-off-by: Azat Khuzhin <[email protected]>
@azat azat force-pushed the tests-fix-parser branch from fa77b69 to 23494e9 Compare June 14, 2022 15:17
@tavplubix
Copy link
Copy Markdown
Member

tavplubix commented Jun 15, 2022

Stateless tests (thread, actions) [1/3] - checks with tsan are broken after upgrading to llvm-14

@tavplubix tavplubix merged commit f6f4d78 into ClickHouse:master Jun 15, 2022
@azat azat deleted the tests-fix-parser branch June 15, 2022 20:49
@tavplubix
Copy link
Copy Markdown
Member

tavplubix commented Jun 28, 2022

Seems like it did not help: https://github.com/ClickHouse/ClickHouse/runs/7085200751?check_suite_focus=true

This was referenced Jun 28, 2022
Merged
Merged
@tavplubix tavplubix mentioned this pull request Aug 10, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tavplubix tavplubix tavplubix approved these changes

Assignees

@tavplubix tavplubix

Labels

pr-not-for-changelog This PR should not be mentioned in the changelog

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@azat @tavplubix @robot-ch-test-poll