Skip to content

SpaceSaving remove last element from map fix#17845

Merged
alexey-milovidov merged 14 commits intoClickHouse:masterfrom
kitaisreal:space-saving-remove-last-element-from-map-fix
Dec 13, 2020
Merged

SpaceSaving remove last element from map fix#17845
alexey-milovidov merged 14 commits intoClickHouse:masterfrom
kitaisreal:space-saving-remove-last-element-from-map-fix

Conversation

@kitaisreal
Copy link
Copy Markdown
Contributor

@kitaisreal kitaisreal commented Dec 6, 2020

I hereby agree to the terms of the CLA available at: https://yandex.ru/legal/cla/?lang=en

Mentioned in #17404.

Changelog category (leave one):

  • Bug Fix

Changelog Entry:
Fix possible segfault in topK aggregate function. This closes #17404.

@robot-clickhouse robot-clickhouse added the pr-bugfix Pull request with bugfix, not backported by default label Dec 6, 2020
@alexey-milovidov alexey-milovidov self-assigned this Dec 6, 2020
@kitaisreal kitaisreal force-pushed the space-saving-remove-last-element-from-map-fix branch 2 times, most recently from 32c7e22 to a053e80 Compare December 10, 2020 19:55
@kitaisreal kitaisreal force-pushed the space-saving-remove-last-element-from-map-fix branch from a053e80 to b70b98c Compare December 11, 2020 13:54
Copy link
Copy Markdown
Member

@alexey-milovidov alexey-milovidov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

15061972 
00453_top_k.reference

The reference file is too large.
You can wrap topK calls with length.

#include <Common/HashTable/HashSet.h>


int main(int, char **)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, it's better to "unitize" this test (see gtest_*.cpp files), so it will be run in CI.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

@@ -1,5 +1,5 @@
#include <iostream>
#include <iomanip>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, this test is also not unitized (as it is very old), it will be better to convert it to unit test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

/// We need to guarantee loop termination because there will be empty position
assert(m_size < grower.bufSize());

size_t j = i;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i, j, k are cryptic names, maybe invent some better names?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed.


size_t k = grower.place(buf[j].getHash(*this));

if (i <= j ? ((k <= i) || (k > j)) : ((k <= i) && (k > j)))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

memcpy implies i != j but here we check for i <= j that is confusing.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's strange to use ?: operator inside if condition.

&& and || will look better.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed.

@alexey-milovidov
Copy link
Copy Markdown
Member

alexey-milovidov commented Dec 12, 2020

alexey-milovidov added a commit that referenced this pull request Dec 13, 2020
Backport #17845 to 20.12: SpaceSaving remove last element from map fix
alexey-milovidov added a commit that referenced this pull request Dec 13, 2020
Backport #17845 to 20.11: SpaceSaving remove last element from map fix
alexey-milovidov added a commit that referenced this pull request Dec 13, 2020
Backport #17845 to 20.9: SpaceSaving remove last element from map fix
alexey-milovidov added a commit that referenced this pull request Dec 13, 2020
Backport #17845 to 20.10: SpaceSaving remove last element from map fix
kitaisreal added a commit that referenced this pull request Feb 19, 2021
Backport #17845 to 20.8: SpaceSaving remove last element from map fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-bugfix Pull request with bugfix, not backported by default

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Query Fuzzer, topK, segfault.

3 participants