Repository navigation
Commit 853b07c
prototype: migrate MemorySourceConfig; DataSourceExec fully on the hook (apache#22419)
MemorySourceConfig rides DataSource::try_to_proto (in datafusion-datasource,
where the type actually lives). Record-batch IPC serde is pure Arrow, inlined
locally like CsvSource's byte helpers — no codec.
With every DataSource migrated (5 file formats + memory), try_from_data_source_exec
and the DataSourceExec encode downcast arm are deleted entirely: DataSourceExec
now serializes purely via the ExecutionPlan hook, delegating through
DataSource -> FileSource with zero downcasts in the central encode path.
roundtrip_memory_source + the 5 scan tests stay green (192 total).
GenerateSeries/LazyMemoryExec remain typed arms (their generator types live in
datafusion-functions-table, above physical-plan — the hook can't name them
without a crate cycle). Extension stays as the terminal codec.try_encode fallback.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01SfBPC2yYCwHUn4V8pQCVj31 parent c6a26d8 commit 853b07c
2 files changed
Lines changed: 174 additions & 76 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
256 | 256 | | |
257 | 257 | | |
258 | 258 | | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
259 | 347 | | |
260 | 348 | | |
261 | 349 | | |
| |||
607 | 695 | | |
608 | 696 | | |
609 | 697 | | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
| 710 | + | |
| 711 | + | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
| 731 | + | |
| 732 | + | |
| 733 | + | |
| 734 | + | |
| 735 | + | |
| 736 | + | |
| 737 | + | |
| 738 | + | |
| 739 | + | |
| 740 | + | |
| 741 | + | |
| 742 | + | |
| 743 | + | |
| 744 | + | |
| 745 | + | |
| 746 | + | |
| 747 | + | |
| 748 | + | |
| 749 | + | |
| 750 | + | |
| 751 | + | |
| 752 | + | |
| 753 | + | |
| 754 | + | |
| 755 | + | |
| 756 | + | |
| 757 | + | |
| 758 | + | |
| 759 | + | |
| 760 | + | |
| 761 | + | |
| 762 | + | |
| 763 | + | |
| 764 | + | |
| 765 | + | |
| 766 | + | |
| 767 | + | |
| 768 | + | |
| 769 | + | |
| 770 | + | |
| 771 | + | |
| 772 | + | |
| 773 | + | |
| 774 | + | |
| 775 | + | |
| 776 | + | |
| 777 | + | |
| 778 | + | |
| 779 | + | |
610 | 780 | | |
611 | 781 | | |
612 | 782 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
| 37 | + | |
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| |||
111 | 111 | | |
112 | 112 | | |
113 | 113 | | |
114 | | - | |
115 | | - | |
| 114 | + | |
116 | 115 | | |
117 | 116 | | |
118 | 117 | | |
| |||
781 | 780 | | |
782 | 781 | | |
783 | 782 | | |
784 | | - | |
785 | | - | |
| 783 | + | |
| 784 | + | |
786 | 785 | | |
787 | 786 | | |
788 | 787 | | |
| |||
914 | 913 | | |
915 | 914 | | |
916 | 915 | | |
917 | | - | |
918 | | - | |
919 | | - | |
920 | | - | |
921 | | - | |
922 | | - | |
923 | | - | |
924 | | - | |
925 | | - | |
926 | | - | |
927 | 916 | | |
928 | 917 | | |
929 | 918 | | |
| |||
3337 | 3326 | | |
3338 | 3327 | | |
3339 | 3328 | | |
3340 | | - | |
3341 | | - | |
3342 | | - | |
3343 | | - | |
3344 | | - | |
3345 | | - | |
3346 | | - | |
3347 | | - | |
3348 | | - | |
3349 | | - | |
3350 | | - | |
3351 | | - | |
3352 | | - | |
3353 | | - | |
3354 | | - | |
3355 | | - | |
3356 | | - | |
3357 | | - | |
3358 | | - | |
3359 | | - | |
3360 | | - | |
3361 | | - | |
3362 | | - | |
3363 | | - | |
3364 | | - | |
3365 | | - | |
3366 | | - | |
3367 | | - | |
3368 | | - | |
3369 | | - | |
3370 | | - | |
3371 | | - | |
3372 | | - | |
3373 | | - | |
3374 | | - | |
3375 | | - | |
3376 | | - | |
3377 | | - | |
3378 | | - | |
3379 | | - | |
3380 | | - | |
3381 | | - | |
3382 | | - | |
3383 | | - | |
3384 | | - | |
3385 | | - | |
3386 | | - | |
3387 | | - | |
3388 | | - | |
3389 | | - | |
3390 | | - | |
3391 | | - | |
3392 | | - | |
3393 | | - | |
3394 | | - | |
3395 | | - | |
3396 | | - | |
3397 | | - | |
3398 | | - | |
3399 | | - | |
3400 | | - | |
3401 | 3329 | | |
3402 | 3330 | | |
3403 | 3331 | | |
| |||
0 commit comments