Repository navigation
Commit 0491437
authored
[Opt](scanner-scheduler) Optimize
## Proposed changes
- Optimize `BlockingQueue`, `BlockingPriorityQueue` by swapping `notify` and `unlock` to reduce lock competition. Ref: https://www.boost.org/doc/libs/1_54_0/boost/thread/sync_bounded_queue.hpp
- Change remote scan thread pool to `PriorityQueue`.
### Test result
Before:
```
mysql> select sum(lo_partkey) from lineorder;
+-----------------+
| sum(lo_partkey) |
+-----------------+
| 300021444265405 |
+-----------------+
1 row in set (1.11 sec)
```
After:
```
mysql> select sum(lo_partkey) from lineorder;
+-----------------+
| sum(lo_partkey) |
+-----------------+
| 300021444265405 |
+-----------------+
1 row in set (0.80 sec)
```BlockingQueue, BlockingPriorityQueue and change remote scan thread pool. (apache#26784)1 parent c1c8376 commit 0491437
4 files changed
Lines changed: 82 additions & 32 deletions
File tree
- be/src
- util
- vec/exec/scan
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
45 | | - | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
46 | 48 | | |
47 | 49 | | |
48 | 50 | | |
| |||
55 | 57 | | |
56 | 58 | | |
57 | 59 | | |
58 | | - | |
59 | | - | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
60 | 69 | | |
61 | | - | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
62 | 74 | | |
63 | 75 | | |
64 | 76 | | |
65 | 77 | | |
66 | 78 | | |
67 | 79 | | |
68 | | - | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
69 | 83 | | |
70 | 84 | | |
71 | 85 | | |
| |||
95 | 109 | | |
96 | 110 | | |
97 | 111 | | |
98 | | - | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
99 | 117 | | |
100 | 118 | | |
101 | 119 | | |
| |||
131 | 149 | | |
132 | 150 | | |
133 | 151 | | |
134 | | - | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
135 | 157 | | |
136 | 158 | | |
137 | 159 | | |
| |||
145 | 167 | | |
146 | 168 | | |
147 | 169 | | |
| 170 | + | |
148 | 171 | | |
149 | 172 | | |
150 | 173 | | |
| |||
154 | 177 | | |
155 | 178 | | |
156 | 179 | | |
157 | | - | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
158 | 185 | | |
159 | 186 | | |
160 | 187 | | |
| |||
163 | 190 | | |
164 | 191 | | |
165 | 192 | | |
166 | | - | |
167 | | - | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
168 | 198 | | |
169 | 199 | | |
170 | 200 | | |
| |||
204 | 234 | | |
205 | 235 | | |
206 | 236 | | |
| 237 | + | |
| 238 | + | |
207 | 239 | | |
208 | 240 | | |
209 | | - | |
| 241 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
44 | | - | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
45 | 47 | | |
46 | 48 | | |
47 | 49 | | |
| |||
50 | 52 | | |
51 | 53 | | |
52 | 54 | | |
53 | | - | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
54 | 59 | | |
55 | 60 | | |
56 | 61 | | |
57 | 62 | | |
58 | 63 | | |
59 | | - | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
60 | 69 | | |
61 | 70 | | |
62 | 71 | | |
| |||
70 | 79 | | |
71 | 80 | | |
72 | 81 | | |
73 | | - | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
74 | 86 | | |
75 | 87 | | |
76 | 88 | | |
77 | 89 | | |
78 | 90 | | |
79 | 91 | | |
80 | 92 | | |
81 | | - | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
82 | 98 | | |
83 | 99 | | |
84 | 100 | | |
| |||
98 | 114 | | |
99 | 115 | | |
100 | 116 | | |
101 | | - | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
102 | 122 | | |
103 | 123 | | |
104 | 124 | | |
| |||
136 | 156 | | |
137 | 157 | | |
138 | 158 | | |
| 159 | + | |
| 160 | + | |
139 | 161 | | |
140 | 162 | | |
141 | 163 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
97 | 97 | | |
98 | 98 | | |
99 | 99 | | |
100 | | - | |
| 100 | + | |
101 | 101 | | |
102 | 102 | | |
103 | 103 | | |
| |||
123 | 123 | | |
124 | 124 | | |
125 | 125 | | |
126 | | - | |
127 | | - | |
128 | | - | |
129 | | - | |
130 | | - | |
131 | | - | |
132 | | - | |
133 | | - | |
134 | | - | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
135 | 129 | | |
136 | 130 | | |
137 | 131 | | |
| |||
273 | 267 | | |
274 | 268 | | |
275 | 269 | | |
276 | | - | |
| 270 | + | |
| 271 | + | |
277 | 272 | | |
278 | | - | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
279 | 276 | | |
280 | 277 | | |
281 | 278 | | |
| |||
486 | 483 | | |
487 | 484 | | |
488 | 485 | | |
489 | | - | |
| 486 | + | |
490 | 487 | | |
491 | 488 | | |
492 | 489 | | |
| |||
507 | 504 | | |
508 | 505 | | |
509 | 506 | | |
510 | | - | |
511 | 507 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
111 | 111 | | |
112 | 112 | | |
113 | 113 | | |
114 | | - | |
| 114 | + | |
115 | 115 | | |
116 | 116 | | |
117 | 117 | | |
| |||
0 commit comments