[vectorized](improving) (exec) optimize VDataStreamSender's send() performance #7747 by zuochunwei · Pull Request #7751 · apache/doris

zuochunwei · 2022-01-13T12:07:22Z

Proposed changes

gather channel's rows, and then send together
handle each column's different rows at a time is good for improving cache hit
comparing test data, wait @zenoyang to provide

Issue Number: close #xxx

Problem Summary:

Describe the overview of changes.

Checklist(Required)

Does it affect the original behavior: (Yes/No/I Don't know)
Has unit tests been added: (Yes/No/No Need)
Has document been added or modified: (Yes/No/No Need)
Does it need to update dependencies: (Yes/No)
Are there any changes that cannot be rolled back: (Yes/No)

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

HappenLee · 2022-01-13T12:14:45Z

+        auto& dst = _columns[i];
+        auto& src = *src_columns_with_schema[i].column.get();
+        for (const int* row = row_begin; row != row_end; ++row) {
+            dst->insert_from(src, *row);


here will be virtual function call. so we should implement a function in column.

yes, that 's what i will todo next

zenoyang · 2022-01-13T14:03:46Z

	before PR (s)	after PR (s)
Q2.1	4.22、4.15、4.55	4.28、4.21、4.18
Q3.1	3.99、3.88、4.14	4.00、4.04、4.12

Since DataStreamSender is fast, the complete query time may not reflect the PR difference.

zuochunwei · 2022-01-14T04:14:46Z

before PR (s) after PR (s)
Q2.1 4.22、4.15、4.55 4.28、4.21、4.18
Q3.1 3.99、3.88、4.14 4.00、4.04、4.12
Since DataStreamSender is fast, the complete query time may not reflect the PR difference.

please test for the recently commit, thanks!

zenoyang · 2022-01-14T06:29:22Z

new test:

	before PR (s)	after PR (s)
Q2.1	4.22、4.15、4.55	4.26、4.23、4.51
Q3.1	3.99、3.88、4.14	3.93、3.90、3.70

note: hash join

zenoyang · 2022-01-14T10:22:46Z

test sql: select count(distinct lo_partkey) from lineorder

before pr

Total: 15s147ms

VDataStreamSender (dst_id=3, dst_fragments=[{"0297b49f50144c67-8175137460f3732a", "0297b49f50144c67-8175137460f3732b", "0297b49f50144c67-8175137460f3732c"}]):(Active: 13s155ms, % non-child: 86.86%)
VDataStreamSender (dst_id=3, dst_fragments=[{"0297b49f50144c67-8175137460f3732a", "0297b49f50144c67-8175137460f3732b", "0297b49f50144c67-8175137460f3732c"}]):(Active: 13s87ms, % non-child: 86.41%)
VDataStreamSender (dst_id=3, dst_fragments=[{"0297b49f50144c67-8175137460f3732a", "0297b49f50144c67-8175137460f3732b", "0297b49f50144c67-8175137460f3732c"}]):(Active: 12s833ms, % non-child: 84.73%)

VDataStreamSender cost: 
13155*0.8686=11426.433 ms
13870*0.8641=11985.067 ms
12883*0.8473=10915.766 ms

after pr

Total: 13s414ms

VDataStreamSender (dst_id=3, dst_fragments=[{"f191f98c5e644df6-a05e405fde783be2", "f191f98c5e644df6-a05e405fde783be3", "f191f98c5e644df6-a05e405fde783be4"}]):(Active: 11s129ms, % non-child: 82.97%)
VDataStreamSender (dst_id=3, dst_fragments=[{"f191f98c5e644df6-a05e405fde783be2", "f191f98c5e644df6-a05e405fde783be3", "f191f98c5e644df6-a05e405fde783be4"}]):(Active: 10s322ms, % non-child: 76.95%)
VDataStreamSender (dst_id=3, dst_fragments=[{"f191f98c5e644df6-a05e405fde783be2", "f191f98c5e644df6-a05e405fde783be3", "f191f98c5e644df6-a05e405fde783be4"}]):(Active: 9s120ms, % non-child: 67.99%)

VDataStreamSender cost: 
11129*0.8297=9233.731 ms
10322*0.7695=7942.779 ms
9120*0.6799=6200.688 ms

HappenLee · 2022-01-17T03:27:20Z

+
+    void insert_indices_from(const IColumn& src, const int* indices_begin, const int* indices_end) override {
+        for (auto x = indices_begin; x != indices_end; ++x) {
+            Self::insert_from(src, *x);


same to the column_vector

HappenLee · 2022-01-17T03:28:31Z


+template <typename T>
+void ColumnVector<T>::insert_indices_from(const IColumn& src, const int* indices_begin, const int* indices_end) {
+    const Self& src_vec = assert_cast<const Self&>(src);


use push_back_without_reserve

HappenLee · 2022-01-17T03:29:07Z

 }

+void ColumnNullable::insert_indices_from(const IColumn& src, const int* indices_begin, const int* indices_end) {
+    for (auto x = indices_begin; x != indices_end; ++x) {


which will cause to many virtural function call, refactor it

HappenLee · 2022-01-17T03:35:39Z

        }

+        // channel2rows' subscript means channel id 
+        std::vector<vectorized::UInt64> hash_vals(rows);


we should reuse the mem

HappenLee · 2022-01-17T03:35:57Z

        // vectorized caculate hash val
        int rows = block->rows();
        // for each row, we have a hash_val
        std::vector<size_t> hash_vals(rows);


we should reuse the mem

HappenLee · 2022-01-17T03:37:33Z

+
+template <typename Channels, typename HashVals>
+Status VDataStreamSender::channel_add_rows(Channels& channels, int num_channels, const HashVals& hash_vals, int rows, Block* block) {
+    std::vector<int> channel2rows[num_channels];


same to the up, we should reuse the mem

channel2rows is used for aggregate rows for channels, it's a temporary data served for the function of channel_add_rows, it's not used in other member functions, it's not has getter/setter, it's not visiable to outer, so i don't think make it as member data is a good idea.

for performance, if you can prove it with rewriting code, please tell me

…rformance #7747 (#7751)

zuochunwei added 3 commits January 13, 2022 20:03

optimize VDataStreamSender::send

4a888c7

optimize VDataStreamSender::send() compile error

16fc7d5

reduce redundant codes

06a8423

HappenLee reviewed Jan 13, 2022

View reviewed changes

add insert_indices_from interface for IColumn

ef5a652

HappenLee reviewed Jan 17, 2022

View reviewed changes

modify to statisfy Code review suggestions

5d19e7b

HappenLee merged commit 778fa8d into apache:vectorized Jan 17, 2022

HappenLee pushed a commit that referenced this pull request Jan 17, 2022

[Vectorized](improving) (exec) optimize VDataStreamSender's send() pe…

f45d2a2

…rformance #7747 (#7751)

Conversation

zuochunwei commented Jan 13, 2022

Proposed changes

Problem Summary:

Checklist(Required)

Further comments

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zenoyang commented Jan 13, 2022

Uh oh!

zuochunwei commented Jan 14, 2022

Uh oh!

zenoyang commented Jan 14, 2022

Uh oh!

zenoyang commented Jan 14, 2022

before pr

after pr

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants