Repository navigation
Commit dc69242
refactor(proto): migrate ArrowSource serde (apache#24189)
## Which issue does this PR close?
- Part of apache#23516.
## Rationale for this change
Part of epic apache#23494. Moves `ArrowSource` protobuf serialization from the
central dispatch into the source implementation.
## What changes are included in this PR?
Add protobuf serialization and deserialization to `ArrowSource` and add
the required `proto` feature wiring to `datafusion-datasource-arrow`.
Repoint the live decode arm to `ArrowSource::try_from_proto` and remove
the old central encode arm. Keep `try_into_arrow_scan_physical_plan` as
a deprecated compatibility wrapper that delegates to the new
implementation.
The protobuf wire format remains unchanged.
## Are these changes tested?
Yes. The existing `roundtrip_arrow_scan` coverage passes through the new
hooks.
## Are there any user-facing changes?
The existing `PhysicalPlanNodeExt` method remains available as a
deprecated compatibility wrapper. There is no immediate API removal or
wire-format change.1 parent a66175e commit dc69242
5 files changed
Lines changed: 96 additions & 51 deletions
File tree
- datafusion
- datasource-arrow
- src
- proto
- src/physical_plan
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| 45 | + | |
45 | 46 | | |
46 | 47 | | |
47 | 48 | | |
| |||
65 | 66 | | |
66 | 67 | | |
67 | 68 | | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
392 | 392 | | |
393 | 393 | | |
394 | 394 | | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| 436 | + | |
| 437 | + | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
| 441 | + | |
| 442 | + | |
| 443 | + | |
| 444 | + | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
395 | 453 | | |
396 | 454 | | |
397 | 455 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
62 | 62 | | |
63 | 63 | | |
64 | 64 | | |
65 | | - | |
| 65 | + | |
66 | 66 | | |
67 | 67 | | |
68 | 68 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
29 | | - | |
30 | 29 | | |
31 | 30 | | |
32 | 31 | | |
| |||
86 | 85 | | |
87 | 86 | | |
88 | 87 | | |
89 | | - | |
90 | | - | |
91 | | - | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
| 88 | + | |
| 89 | + | |
96 | 90 | | |
97 | 91 | | |
98 | 92 | | |
| |||
115 | 109 | | |
116 | 110 | | |
117 | 111 | | |
118 | | - | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
119 | 115 | | |
120 | 116 | | |
121 | 117 | | |
| |||
1103 | 1099 | | |
1104 | 1100 | | |
1105 | 1101 | | |
1106 | | - | |
1107 | | - | |
| 1102 | + | |
| 1103 | + | |
1108 | 1104 | | |
1109 | 1105 | | |
1110 | 1106 | | |
| |||
1233 | 1229 | | |
1234 | 1230 | | |
1235 | 1231 | | |
1236 | | - | |
1237 | | - | |
1238 | | - | |
1239 | | - | |
1240 | | - | |
1241 | | - | |
1242 | | - | |
1243 | | - | |
1244 | | - | |
1245 | | - | |
1246 | 1232 | | |
1247 | 1233 | | |
1248 | 1234 | | |
| |||
1386 | 1372 | | |
1387 | 1373 | | |
1388 | 1374 | | |
| 1375 | + | |
| 1376 | + | |
| 1377 | + | |
| 1378 | + | |
1389 | 1379 | | |
1390 | 1380 | | |
1391 | 1381 | | |
1392 | 1382 | | |
1393 | 1383 | | |
1394 | 1384 | | |
1395 | | - | |
1396 | | - | |
1397 | | - | |
1398 | | - | |
1399 | | - | |
1400 | | - | |
| 1385 | + | |
| 1386 | + | |
| 1387 | + | |
| 1388 | + | |
1401 | 1389 | | |
1402 | 1390 | | |
1403 | | - | |
1404 | | - | |
1405 | | - | |
| 1391 | + | |
| 1392 | + | |
| 1393 | + | |
1406 | 1394 | | |
1407 | 1395 | | |
1408 | 1396 | | |
| |||
2459 | 2447 | | |
2460 | 2448 | | |
2461 | 2449 | | |
| 2450 | + | |
| 2451 | + | |
| 2452 | + | |
| 2453 | + | |
2462 | 2454 | | |
2463 | 2455 | | |
2464 | 2456 | | |
2465 | 2457 | | |
2466 | 2458 | | |
2467 | | - | |
2468 | | - | |
2469 | | - | |
2470 | | - | |
2471 | | - | |
2472 | | - | |
2473 | | - | |
2474 | | - | |
2475 | | - | |
2476 | | - | |
2477 | | - | |
2478 | | - | |
2479 | | - | |
2480 | | - | |
2481 | | - | |
2482 | | - | |
2483 | | - | |
2484 | | - | |
2485 | | - | |
2486 | | - | |
| 2459 | + | |
| 2460 | + | |
| 2461 | + | |
| 2462 | + | |
| 2463 | + | |
| 2464 | + | |
2487 | 2465 | | |
2488 | 2466 | | |
2489 | 2467 | | |
| |||
0 commit comments