You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 119fad9
Browse filesBrowse the repository at this point in the historyBrowse files
authored
ci: replace MinIO with RustFS for integration tests (#3273)
* ci: replace MinIO with RustFS for integration tests
`quay.io/minio/minio` no longer allows anonymous pulls, so
`make docker-up` fails and the integration tests cannot start.
DataFusion hit the same failure and switched to RustFS in
apache/datafusion#25706. PyIceberg moved to RustFS earlier in
apache/iceberg-python#3928.
Run `rustfs/rustfs:1.0.0` as the shared S3 service instead. It keeps
the `admin`/`password` credentials, the 9000/9001 ports, and
virtual-hosted-style access through `RUSTFS_SERVER_DOMAINS` and the
bucket aliases. The healthcheck uses `/health/ready`, which waits for
storage and IAM. The `mc` bucket setup becomes `curl --aws-sigv4` run
from the RustFS image, and the public bucket policy is dropped because
no test reads anonymously.
Point the REST fixture, HMS, and Spark at `http://rustfs:9000`, and
rename the MinIO-specific test helpers and the
`ICEBERG_TEST_MINIO_ENDPOINT` override to RustFS.
Closes#3272
* ci: use vendor-neutral object-store naming for S3 test service
Rename the RustFS service, hostname, bucket aliases, test helpers and env
var to a generic object-store name (matching PyIceberg), so future
backend swaps only need an image change.
Co-authored-by: Copilot App <[email protected]>
* ci: fail fast and gate readiness on test bucket creation
Treat any non-200/409 response as a failure instead of ignoring it, and
add a healthcheck so `docker compose up --wait` blocks until the buckets
exist.
Co-authored-by: Copilot App <[email protected]>
* ci: remove orphan containers on docker-up
Renaming the minio service leaves stale containers holding port 9000 for
anyone with a previously started stack; clean them up automatically.
Co-authored-by: Copilot App <[email protected]>
* ci: make create-buckets a one-shot job
RustFS returns 200 when re-creating an existing bucket, so plain chained
curl -f calls are idempotent. Replace the status-code parsing, marker-file
healthcheck and tail with a one-shot container, and gate spark-iceberg on
its successful completion.
Co-authored-by: Copilot App <[email protected]>
---------
Co-authored-by: ovoievodin <[email protected]>
Co-authored-by: Kevin Liu <[email protected]>
Co-authored-by: Copilot App <[email protected]>
/usr/bin/mc alias set minio http://minio:9000 admin password;
0 commit comments