Repository navigation
Commit a08d695
committed
[improvement](paimon) Fix Paimon JNI profile snapshot aggregation
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Paimon JNI scanner statistics include both scanner-local cumulative values and process-wide snapshot values such as active scanner counts, async reader thread counts, and JVM memory usage. Exporting all of them through the additive JNI profile counter path made multi-split scans multiply snapshot values by the number of collected splits. This change extends the JNI statistics profile collectors to support non-additive gauge and peak metric types, then reports process-wide snapshots as set/max metrics while keeping scanner-local rows, calls, encoded lengths, and timers additive.
### Release note
None
### Check List (For Author)
- Test: Unit Test
- JAVA_HOME=/usr/local/opt/openjdk@17/libexec/openjdk.jdk/Contents/Home mvn test -pl be-java-extensions/paimon-scanner -am -Dtest=PaimonJniScannerTest -DfailIfNoTests=false -Ddoris.thrift.executable=/usr/local/bin/thrift
- python3 build-support/run_clang_format.py --clang-format-executable $(brew --prefix llvm@16)/bin/clang-format --style file --inplace true be/src/format/jni/jni_reader.cpp be/src/format_v2/jni/jni_table_reader.cpp be/src/format_v2/jni/jni_table_reader.h
- git diff --check
- Behavior changed: No
- Does this need documentation: No1 parent 61eb6bf commit a08d695
4 files changed
Lines changed: 89 additions & 30 deletions
File tree
- be/src
- format_v2/jni
- format/jni
- fe/be-java-extensions/paimon-scanner/src
- main/java/org/apache/doris/paimon
- test/java/org/apache/doris/paimon
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
424 | 424 | | |
425 | 425 | | |
426 | 426 | | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
427 | 430 | | |
428 | 431 | | |
429 | 432 | | |
430 | 433 | | |
431 | 434 | | |
432 | 435 | | |
433 | 436 | | |
434 | | - | |
| 437 | + | |
435 | 438 | | |
436 | 439 | | |
437 | 440 | | |
438 | 441 | | |
| 442 | + | |
439 | 443 | | |
440 | 444 | | |
441 | 445 | | |
| 446 | + | |
442 | 447 | | |
443 | 448 | | |
444 | 449 | | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
445 | 475 | | |
446 | | - | |
| 476 | + | |
| 477 | + | |
447 | 478 | | |
448 | 479 | | |
449 | | - | |
450 | 480 | | |
451 | 481 | | |
452 | 482 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
200 | 200 | | |
201 | 201 | | |
202 | 202 | | |
| 203 | + | |
203 | 204 | | |
204 | 205 | | |
205 | 206 | | |
206 | 207 | | |
207 | 208 | | |
208 | 209 | | |
209 | 210 | | |
210 | | - | |
| 211 | + | |
211 | 212 | | |
212 | 213 | | |
213 | 214 | | |
214 | 215 | | |
| 216 | + | |
215 | 217 | | |
216 | 218 | | |
217 | 219 | | |
| 220 | + | |
218 | 221 | | |
219 | 222 | | |
220 | 223 | | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
221 | 249 | | |
222 | | - | |
| 250 | + | |
| 251 | + | |
223 | 252 | | |
224 | 253 | | |
225 | | - | |
226 | 254 | | |
227 | 255 | | |
228 | 256 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
405 | 405 | | |
406 | 406 | | |
407 | 407 | | |
408 | | - | |
409 | | - | |
410 | | - | |
411 | | - | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
412 | 412 | | |
413 | | - | |
| 413 | + | |
414 | 414 | | |
415 | | - | |
| 415 | + | |
416 | 416 | | |
417 | 417 | | |
418 | | - | |
| 418 | + | |
419 | 419 | | |
420 | | - | |
421 | | - | |
| 420 | + | |
| 421 | + | |
422 | 422 | | |
423 | 423 | | |
424 | 424 | | |
| |||
451 | 451 | | |
452 | 452 | | |
453 | 453 | | |
454 | | - | |
455 | | - | |
456 | | - | |
457 | | - | |
458 | | - | |
459 | | - | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
460 | 461 | | |
461 | 462 | | |
462 | 463 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
92 | 92 | | |
93 | 93 | | |
94 | 94 | | |
95 | | - | |
| 95 | + | |
96 | 96 | | |
97 | 97 | | |
98 | 98 | | |
| |||
108 | 108 | | |
109 | 109 | | |
110 | 110 | | |
111 | | - | |
112 | | - | |
| 111 | + | |
| 112 | + | |
113 | 113 | | |
114 | 114 | | |
115 | | - | |
| 115 | + | |
116 | 116 | | |
117 | | - | |
118 | | - | |
119 | | - | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
120 | 120 | | |
121 | 121 | | |
122 | 122 | | |
123 | | - | |
124 | | - | |
| 123 | + | |
| 124 | + | |
125 | 125 | | |
126 | 126 | | |
127 | 127 | | |
| |||
0 commit comments