Skip to content

feat(storage)(test): Add a test suite for iceberg-storage - #2253

Open
CTTY wants to merge 11 commits into
apache:mainfrom
CTTY:ctty/storage-test-suite
Open

CTTY wants to merge 11 commits into
apache:mainfrom
CTTY:ctty/storage-test-suite

Conversation

@CTTY

@CTTY CTTY commented Mar 18, 2026

Copy link
Copy Markdown
Collaborator

Which issue does this PR close?

What changes are included in this PR?

  • Add iceberg-storage-common
  • Add a new test suite for iceberg-storage

Are these changes tested?

They are tests themselves :D

@CTTY
CTTY marked this pull request as ready for review March 19, 2026 22:36
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Apr 19, 2026
@CTTY CTTY removed the stale label Apr 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

@blackmwk

Copy link
Copy Markdown
Contributor

Don't close it, it's a missing feature.

@CTTY
CTTY force-pushed the ctty/storage-test-suite branch from 9f41c52 to 8149d7d Compare May 26, 2026 22:56
@CTTY CTTY removed the stale label May 28, 2026
cargo test -p iceberg-storage-common \
--test file_io_suite \
--test resolving_suite \
-- --ignored --test-threads=1 _hf hf_

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels kinda hacky and makes me think if we should just leave hf io tests alone. curious to hear what others think

// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this going to be expanded later?

}

async fn load_opendal_s3() -> Option<StorageHarness> {
let minio_endpoint = get_minio_endpoint();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is minio still supporting these test cases?

)
}

async fn load_opendal_hf() -> Option<StorageHarness> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: can a comment clarify that this is for hugging face and not hadoop or something else.

pub const ENV_HF_DATASET: &str = "HF_DATASET";

#[derive(Debug, Clone, Copy)]
pub enum StorageKind {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving the comment for note taking.

It would be cool to see Azure, Glue, & S3Tables.

@Sruhvx-jpg

Copy link
Copy Markdown

so this is the stale pr ur talking about?

@Sruhvx-jpg

Copy link
Copy Markdown

cc @CTTY

@Sruhvx-jpg

Copy link
Copy Markdown

ig ur busy, so am starting to work on this

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a test suite for Storage crates in general

4 participants