Skip to content

[vectorized](improving) (exec) optimize VDataStreamSender's send() performance - #7747

Closed
zuochunwei wants to merge 27 commits into
apache:vectorizedfrom
zuochunwei:StreanSenderImproving
Closed

[vectorized](improving) (exec) optimize VDataStreamSender's send() performance#7747
zuochunwei wants to merge 27 commits into
apache:vectorizedfrom
zuochunwei:StreanSenderImproving

Conversation

@zuochunwei

@zuochunwei zuochunwei commented Jan 13, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor

Proposed changes

  1. gather channel's rows, and then send together
  2. handle each column's different rows at a time is good for improving cache hit

comparing test data, wait @zenoyang to provide

Issue Number: close #xxx

Problem Summary:

Describe the overview of changes.

Checklist(Required)

  1. Does it affect the original behavior: (Yes/No/I Don't know)
  2. Has unit tests been added: (Yes/No/No Need)
  3. Has document been added or modified: (Yes/No/No Need)
  4. Does it need to update dependencies: (Yes/No)
  5. Are there any changes that cannot be rolled back: (Yes/No)

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

lihaopeng and others added 27 commits December 31, 2021 11:31
lihaopeng @HappenLee
@stdpain @yangzhg @wangbo @EmmyMiao87 @BiteTheDDDDt @zhangstar333 @zuochunwei @zenoyang @wangshuo128 @zbtzbtzbt @Gabriel39 @xinghuayu007 @weizuo93 yiguolei @anneji-dev @awakeljw @taberylyang @azurenake
[Vectorized Exec Engine] Support Vectorized Exec Engine In Doris
e0cadc4 
Co-authored-by: HappenLee <[email protected]>
Co-authored-by: stdpain <[email protected]>
Co-authored-by: Zhengguo Yang <[email protected]>
Co-authored-by: wangbo <[email protected]>
Co-authored-by: emmymiao87 <[email protected]>
Co-authored-by: Pxl <[email protected]>
Co-authored-by: zhangstar333 <[email protected]>
Co-authored-by: thinker <[email protected]>
Co-authored-by: Zeno Yang <[email protected]>
Co-authored-by: Wang Shuo <[email protected]>
Co-authored-by: zhoubintao <[email protected]>
Co-authored-by: Gabriel <[email protected]>
Co-authored-by: xinghuayu007 <[email protected]>
Co-authored-by: weizuo93 <[email protected]>
Co-authored-by: yiguolei <[email protected]>
Co-authored-by: anneji-dev <[email protected]>
Co-authored-by: awakeljw <[email protected]>
Co-authored-by: taberylyang <[email protected]>
Co-authored-by: Cui Kaifeng <[email protected]>
@HappenLee lihaopeng
[Bug] Fix negative function error result and sort node eos (apache#7555)
ad71c92 
Co-authored-by: lihaopeng <[email protected]>
@zhangstar333
[Function] Fix error about rank/dense_rank/row_number return always n…
27d0b07 
…ot nullable (apache#7561)
@HappenLee lihaopeng
[Bug] Fix bug of concat function and fold const expr (apache#7608)
8d440c6 
Co-authored-by: lihaopeng <[email protected]>
@BiteTheDDDDt
[Bug] Change parser string to int (apache#7595)
19249d6
@BiteTheDDDDt
[Vectorized][Function] Support function and (apache#7618)
5d711ce
@zuochunwei zuochunwei
[Refactor] Cow refactor: giveup using boost (apache#7567)
4a213f2 
Co-authored-by: zuochunwei <[email protected]>
@HappenLee lihaopeng
[Bug] Fix bug of cast expr nullable and ifnull function (apache#7626)
5229418 
Co-authored-by: lihaopeng <[email protected]>
@BiteTheDDDDt
[Vectorized][Feature] fix core dump when using function override and …
5830710 
…function alias at the same time && support substr(str,int) override (apache#7640)
@zhangstar333
[Function][Vec] add function coalesce (apache#7632)
23e1518
@HappenLee lihaopeng
[Bug] Fix function nulllable not match and largetint cast failed (apa…
3b09779 
…che#7659)

Co-authored-by: lihaopeng <[email protected]>
@BiteTheDDDDt
[Vectorizd] [Function] Add string type vec support at doris_builtins_…
9b3897a 
…functions[D (apache#7661)
@BiteTheDDDDt
[Vectorized][Feature] support money_format/ucase/character_length (ap…
55a6a80 
…ache#7649)
@zuochunwei zuochunwei
[vectorized] [block] Add new method get_data_type to avoid unnecessar…
4ec9f0c 
…y copy by the method get_data_type (apache#7600)

Co-authored-by: zuochunwei <[email protected]>
@zenoyang
[Vectorized] Support bloom filter predicate on vectorized engine stor…
f8bf00f 
…age layer (apache#7557)
@BiteTheDDDDt
[Vectorized] [Function] Support do not fold constant at vectorized (a…
9bc346f 
…pache#7668)
@HappenLee lihaopeng
[Feature][Vectorized] Support String in vec exe engine (apache#7670)
a7ba883 
Co-authored-by: lihaopeng <[email protected]>
@zuochunwei zuochunwei
[Vectorized] (olap) Optimize BlockReader's performance (apache#7642)
5b0997b 
Co-authored-by: zuochunwei <[email protected]>
@BiteTheDDDDt
[Vectorized][Bug] fix 'negative' function ut run fail && fix testIsBu…
ca35232 
…cketShuffleJoin run fail && fix some compile fail (apache#7688)
@BiteTheDDDDt
[Vectorized][Enhancement] fix some bug & improve some code (apache#7714)
cc1094b
@HappenLee lihaopeng
[Vectorized][Enhancement] use simd to speed up coalesce and if_not_nu…
1759125 
…ll function (apache#7722)

Co-authored-by: lihaopeng <[email protected]>
@BiteTheDDDDt
[Vectorized][Bug] Fix get wrong result when select random column && f…
1ac2205 
…ix get wrong has_null_tag (apache#7728)
@BiteTheDDDDt
[Vectorized][Feature] upport function conv (apache#7693)
881b15d 
* support function conv()

* add document
@HappenLee lihaopeng
[Vectorized][Bug] Bitmap/HLL type no support cast to varchar/char (ap…
ba09d3d 
…ache#7737)

Co-authored-by: lihaopeng <[email protected]>
zuochunwei
optimize VDataStreamSender::send
f1f60f4
zuochunwei
optimize VDataStreamSender::send() compile error
0d238f8
zuochunwei
reduce redundant codes
0a01138
@zuochunwei zuochunwei closed this Jan 13, 2022
HappenLee pushed a commit that referenced this pull request Jan 17, 2022
@zuochunwei
[Vectorized](improving) (exec) optimize VDataStreamSender's send() pe…
778fa8d 
…rformance #7747 (#7751)
HappenLee pushed a commit that referenced this pull request Jan 17, 2022
@zuochunwei lihaopeng
[Vectorized](improving) (exec) optimize VDataStreamSender's send() pe…
f45d2a2 
…rformance #7747 (#7751)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@zuochunwei @HappenLee @zhangstar333 @BiteTheDDDDt @zenoyang