Repository navigation
Commit 918013e
authored
refactor(proto): migrate AvroSource serde (#24190)
## Which issue does this PR close?
- Closes #23516.
## Rationale for this change
Part of epic #23494. Moves `AvroSource` protobuf serialization from the
central dispatch into the source implementation.
## What changes are included in this PR?
Add protobuf serialization and deserialization to `AvroSource` and add
the required `proto` feature wiring to `datafusion-datasource-avro`.
Repoint the feature-gated live decode arm to
`AvroSource::try_from_proto` and remove the old central encode arm. Keep
`try_into_avro_scan_physical_plan` as a deprecated compatibility wrapper
that delegates to the new implementation.
The protobuf wire format remains unchanged.
## Are these changes tested?
Yes. Added `roundtrip_avro_scan`.
## Are there any user-facing changes?
The existing `PhysicalPlanNodeExt` method remains available as a
deprecated compatibility wrapper. There is no immediate API removal or
wire-format change.1 parent ddb0250 commit 918013e
6 files changed
Lines changed: 101 additions & 28 deletions
File tree
- datafusion
- datasource-avro
- src
- proto
- src/physical_plan
- tests/cases
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
33 | 42 | | |
34 | 43 | | |
35 | 44 | | |
| |||
39 | 48 | | |
40 | 49 | | |
41 | 50 | | |
| 51 | + | |
42 | 52 | | |
43 | 53 | | |
44 | 54 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
168 | 168 | | |
169 | 169 | | |
170 | 170 | | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
171 | 222 | | |
172 | 223 | | |
173 | 224 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
63 | 63 | | |
64 | 64 | | |
65 | 65 | | |
66 | | - | |
| 66 | + | |
67 | 67 | | |
68 | 68 | | |
69 | 69 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1090 | 1090 | | |
1091 | 1091 | | |
1092 | 1092 | | |
1093 | | - | |
1094 | | - | |
| 1093 | + | |
| 1094 | + | |
| 1095 | + | |
| 1096 | + | |
| 1097 | + | |
| 1098 | + | |
| 1099 | + | |
| 1100 | + | |
| 1101 | + | |
1095 | 1102 | | |
1096 | 1103 | | |
1097 | 1104 | | |
| |||
1429 | 1436 | | |
1430 | 1437 | | |
1431 | 1438 | | |
| 1439 | + | |
| 1440 | + | |
| 1441 | + | |
| 1442 | + | |
1432 | 1443 | | |
1433 | 1444 | | |
1434 | 1445 | | |
| |||
1437 | 1448 | | |
1438 | 1449 | | |
1439 | 1450 | | |
1440 | | - | |
1441 | | - | |
1442 | | - | |
1443 | | - | |
| 1451 | + | |
| 1452 | + | |
| 1453 | + | |
| 1454 | + | |
1444 | 1455 | | |
1445 | 1456 | | |
1446 | | - | |
1447 | | - | |
1448 | | - | |
| 1457 | + | |
| 1458 | + | |
| 1459 | + | |
1449 | 1460 | | |
1450 | 1461 | | |
1451 | 1462 | | |
| |||
2472 | 2483 | | |
2473 | 2484 | | |
2474 | 2485 | | |
2475 | | - | |
2476 | | - | |
2477 | | - | |
2478 | | - | |
2479 | | - | |
2480 | | - | |
2481 | | - | |
2482 | | - | |
2483 | | - | |
2484 | | - | |
2485 | | - | |
2486 | | - | |
2487 | | - | |
2488 | | - | |
2489 | | - | |
2490 | | - | |
2491 | | - | |
2492 | | - | |
2493 | 2486 | | |
2494 | 2487 | | |
2495 | 2488 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1378 | 1378 | | |
1379 | 1379 | | |
1380 | 1380 | | |
| 1381 | + | |
| 1382 | + | |
| 1383 | + | |
| 1384 | + | |
| 1385 | + | |
| 1386 | + | |
| 1387 | + | |
| 1388 | + | |
| 1389 | + | |
| 1390 | + | |
| 1391 | + | |
| 1392 | + | |
| 1393 | + | |
| 1394 | + | |
| 1395 | + | |
| 1396 | + | |
| 1397 | + | |
| 1398 | + | |
1381 | 1399 | | |
1382 | 1400 | | |
1383 | 1401 | | |
| |||
0 commit comments