Repository navigation
Commit f88735f
authored
[chore](build) Add paimon-rust third-party build integration (apache#68562)
### What problem does this PR solve?
Related PR: apache#66227
Prepare the third-party dependency for the Paimon Rust reader on master
independently of the reader implementation. This extracts all
dependency-related changes from apache#66227 after simplifying its Rust
archive installation.
- Download Apache paimon-rust v0.4.0-rc1 with a pinned checksum and
build `paimon-c` with HDFS support.
- Generate the C header with pinned cbindgen 0.29.4 and install the
header and archive using the existing third-party copy/strip
conventions.
- Use Rust 1.94.0 by default for both Lance and Paimon. When changing
the toolchain, rebuild both libraries with the same `RUSTUP_TOOLCHAIN`
to avoid conflicting Rust standard libraries.
- Register the dependency in CMake while preserving libm and glibc
compatibility link ordering.
The changes cover dependency registration, building, and installation
checks, plus build-script regression tests. Existing installations
missing either Rust archive or the generated Paimon header follow the
full dependency rebuild path so Lance and Paimon are rebuilt together.
Header generation inherits the selected Cargo executable and offline
mode. Reader code and its functional tests remain in apache#66227. The Rust
archive identity helper and its dedicated test file are not included.
### Validation
- Bash syntax and `git diff --check` passed.
- `python3 thirdparty/test/paimon-build-test.py`: all 10 tests passed on
both this branch and apache#66227. Eight tests fail before the fix. Coverage
includes old Linux/macOS installs, missing archives/header,
complete-install reuse, clean forwarding, custom Cargo, and offline
metadata settings. External compiler/header-generator commands are test
doubles; the tests execute the actual build-script logic.
- Six temporary mock-toolchain scenarios passed on this branch, covering
installation, strip behavior, offline flags, default toolchain
selection, explicitly newer toolchains, and rejection of older
toolchains. The same scenarios passed on apache#66227.
- CMake fragment checks passed with `GLIBC_COMPATIBILITY` enabled and
disabled; compatibility-library ordering was also checked.
- The official source archive MD5 matches the manifest.
- Full Rust/Doris compilation and final binary ABI validation were not
run locally; compilation is requested through CI.
### Release note
None
### Check List (For Author)
- Test: Build-script smoke checks and dependency/link-order validation
described above.
- Behavior changed: Yes. Third-party builds include paimon-rust and both
Rust libraries default to Rust 1.94.0.
- Does this need documentation: No. This prepares an internal build
dependency.1 parent 2eaaf45 commit f88735f
6 files changed
Lines changed: 378 additions & 9 deletions
File tree
- be
- cmake
- thirdparty
- test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
860 | 860 | | |
861 | 861 | | |
862 | 862 | | |
863 | | - | |
| 863 | + | |
| 864 | + | |
864 | 865 | | |
865 | 866 | | |
866 | 867 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
120 | 120 | | |
121 | 121 | | |
122 | 122 | | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
123 | 130 | | |
124 | 131 | | |
125 | 132 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
486 | 486 | | |
487 | 487 | | |
488 | 488 | | |
489 | | - | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
490 | 495 | | |
491 | 496 | | |
492 | 497 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2372 | 2372 | | |
2373 | 2373 | | |
2374 | 2374 | | |
| 2375 | + | |
| 2376 | + | |
| 2377 | + | |
| 2378 | + | |
| 2379 | + | |
| 2380 | + | |
| 2381 | + | |
| 2382 | + | |
| 2383 | + | |
| 2384 | + | |
| 2385 | + | |
| 2386 | + | |
| 2387 | + | |
| 2388 | + | |
| 2389 | + | |
| 2390 | + | |
| 2391 | + | |
| 2392 | + | |
2375 | 2393 | | |
2376 | 2394 | | |
2377 | 2395 | | |
| |||
2381 | 2399 | | |
2382 | 2400 | | |
2383 | 2401 | | |
2384 | | - | |
| 2402 | + | |
2385 | 2403 | | |
2386 | 2404 | | |
2387 | 2405 | | |
2388 | 2406 | | |
2389 | 2407 | | |
2390 | 2408 | | |
2391 | 2409 | | |
2392 | | - | |
| 2410 | + | |
2393 | 2411 | | |
2394 | 2412 | | |
2395 | 2413 | | |
2396 | 2414 | | |
2397 | 2415 | | |
2398 | 2416 | | |
2399 | | - | |
| 2417 | + | |
| 2418 | + | |
| 2419 | + | |
| 2420 | + | |
| 2421 | + | |
| 2422 | + | |
| 2423 | + | |
2400 | 2424 | | |
2401 | 2425 | | |
2402 | 2426 | | |
| |||
2407 | 2431 | | |
2408 | 2432 | | |
2409 | 2433 | | |
2410 | | - | |
| 2434 | + | |
2411 | 2435 | | |
2412 | 2436 | | |
2413 | 2437 | | |
| |||
2440 | 2464 | | |
2441 | 2465 | | |
2442 | 2466 | | |
2443 | | - | |
| 2467 | + | |
| 2468 | + | |
| 2469 | + | |
| 2470 | + | |
| 2471 | + | |
| 2472 | + | |
| 2473 | + | |
| 2474 | + | |
| 2475 | + | |
| 2476 | + | |
| 2477 | + | |
| 2478 | + | |
| 2479 | + | |
| 2480 | + | |
| 2481 | + | |
| 2482 | + | |
| 2483 | + | |
| 2484 | + | |
| 2485 | + | |
| 2486 | + | |
| 2487 | + | |
| 2488 | + | |
| 2489 | + | |
| 2490 | + | |
| 2491 | + | |
| 2492 | + | |
| 2493 | + | |
| 2494 | + | |
| 2495 | + | |
| 2496 | + | |
| 2497 | + | |
| 2498 | + | |
| 2499 | + | |
2444 | 2500 | | |
2445 | | - | |
2446 | | - | |
| 2501 | + | |
| 2502 | + | |
| 2503 | + | |
| 2504 | + | |
2447 | 2505 | | |
| 2506 | + | |
| 2507 | + | |
| 2508 | + | |
| 2509 | + | |
| 2510 | + | |
| 2511 | + | |
| 2512 | + | |
| 2513 | + | |
| 2514 | + | |
| 2515 | + | |
| 2516 | + | |
| 2517 | + | |
| 2518 | + | |
| 2519 | + | |
| 2520 | + | |
| 2521 | + | |
| 2522 | + | |
| 2523 | + | |
| 2524 | + | |
| 2525 | + | |
| 2526 | + | |
| 2527 | + | |
| 2528 | + | |
| 2529 | + | |
| 2530 | + | |
| 2531 | + | |
| 2532 | + | |
| 2533 | + | |
| 2534 | + | |
| 2535 | + | |
| 2536 | + | |
| 2537 | + | |
| 2538 | + | |
| 2539 | + | |
| 2540 | + | |
| 2541 | + | |
| 2542 | + | |
| 2543 | + | |
| 2544 | + | |
| 2545 | + | |
| 2546 | + | |
| 2547 | + | |
| 2548 | + | |
| 2549 | + | |
| 2550 | + | |
| 2551 | + | |
| 2552 | + | |
| 2553 | + | |
| 2554 | + | |
| 2555 | + | |
| 2556 | + | |
| 2557 | + | |
| 2558 | + | |
| 2559 | + | |
| 2560 | + | |
| 2561 | + | |
| 2562 | + | |
| 2563 | + | |
| 2564 | + | |
| 2565 | + | |
| 2566 | + | |
| 2567 | + | |
| 2568 | + | |
| 2569 | + | |
| 2570 | + | |
| 2571 | + | |
| 2572 | + | |
| 2573 | + | |
| 2574 | + | |
| 2575 | + | |
| 2576 | + | |
| 2577 | + | |
| 2578 | + | |
| 2579 | + | |
| 2580 | + | |
| 2581 | + | |
| 2582 | + | |
| 2583 | + | |
| 2584 | + | |
| 2585 | + | |
| 2586 | + | |
| 2587 | + | |
| 2588 | + | |
| 2589 | + | |
| 2590 | + | |
| 2591 | + | |
| 2592 | + | |
2448 | 2593 | | |
2449 | 2594 | | |
2450 | 2595 | | |
| |||
2526 | 2671 | | |
2527 | 2672 | | |
2528 | 2673 | | |
| 2674 | + | |
2529 | 2675 | | |
2530 | 2676 | | |
2531 | 2677 | | |
| |||
2636 | 2782 | | |
2637 | 2783 | | |
2638 | 2784 | | |
| 2785 | + | |
2639 | 2786 | | |
2640 | 2787 | | |
2641 | 2788 | | |
| |||
0 commit comments