Refactor bash tests: storage-compatibility#7299
Conversation
751196b to
e95ac66
Compare
📝 WalkthroughWalkthroughAdds Docker build-arg Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
tests/e2e_tests/test_data_compatibility.py (3)
15-31: Annotate class attributes withClassVarfor clarity.Class-level constants should be annotated with
typing.ClassVarto distinguish them from instance attributes and align with PEP 526 best practices.Apply this diff:
+from typing import ClassVar + class TestStorageCompatibility: """Test storage and snapshot compatibility with defined previous Qdrant versions.""" - PREV_PATCH_VERSION = "v1.14.1" - PREV_MINOR_VERSION = "v1.15.4" + PREV_PATCH_VERSION: ClassVar[str] = "v1.14.1" + PREV_MINOR_VERSION: ClassVar[str] = "v1.15.4" - EXPECTED_COLLECTIONS = [ + EXPECTED_COLLECTIONS: ClassVar[list[str]] = [ "test_collection_vector_memory", "test_collection_vector_on_disk", "test_collection_vector_on_disk_threshold",
110-126: Prefix unused fixture parameters with underscore.The
docker_clientandqdrant_imageparameters are required for the pytest fixture dependency graph but unused directly in the test. Following pytest convention, prefix them with an underscore to signal intentional non-use.Apply this diff:
@pytest.mark.parametrize("version", [PREV_PATCH_VERSION, PREV_MINOR_VERSION]) - def test_storage_compatibility(self, docker_client, qdrant_image, temp_storage_dir, version, qdrant_container_factory): + def test_storage_compatibility(self, _docker_client, _qdrant_image, temp_storage_dir, version, qdrant_container_factory): """Test storage compatibility with previous versions."""
128-150: Prefix unused fixture parameters with underscore.Same as the storage compatibility test -
docker_clientandqdrant_imageare fixture dependencies but unused directly.Apply this diff:
@pytest.mark.parametrize("version", [PREV_PATCH_VERSION, PREV_MINOR_VERSION]) - def test_snapshot_compatibility(self, docker_client, qdrant_image, temp_storage_dir, version, qdrant_container_factory): + def test_snapshot_compatibility(self, _docker_client, _qdrant_image, temp_storage_dir, version, qdrant_container_factory): """Test snapshot recovery compatibility with previous versions."""
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
.github/workflows/integration-tests.yml(1 hunks).github/workflows/storage-compat.yml(0 hunks)tests/e2e_tests/README.md(1 hunks)tests/e2e_tests/test_data_compatibility.py(1 hunks)tests/storage-compat/storage-compatibility.sh(0 hunks)
💤 Files with no reviewable changes (2)
- tests/storage-compat/storage-compatibility.sh
- .github/workflows/storage-compat.yml
🧰 Additional context used
🧬 Code graph analysis (1)
tests/e2e_tests/test_data_compatibility.py (4)
tests/e2e_tests/models.py (1)
QdrantContainerConfig(109-177)tests/e2e_tests/conftest.py (4)
qdrant_container_factory(118-217)docker_client(26-32)qdrant_image(46-114)temp_storage_dir(221-249)tests/e2e_tests/client_utils.py (3)
list_collections_names(54-62)get_collection_info_dict(64-80)wait_for_server(24-38)tests/e2e_tests/utils.py (1)
extract_archive(197-269)
🪛 Ruff (0.13.1)
tests/e2e_tests/test_data_compatibility.py
18-31: Mutable class attributes should be annotated with typing.ClassVar
(RUF012)
86-86: Do not catch blind exception: Exception
(BLE001)
103-103: Do not catch blind exception: Exception
(BLE001)
110-110: Unused method argument: docker_client
(ARG002)
110-110: Unused method argument: qdrant_image
(ARG002)
110-110: Redefinition of unused qdrant_container_factory from line 7
(F811)
129-129: Unused method argument: docker_client
(ARG002)
129-129: Unused method argument: qdrant_image
(ARG002)
129-129: Redefinition of unused qdrant_container_factory from line 7
(F811)
🔇 Additional comments (5)
.github/workflows/integration-tests.yml (1)
259-260: LGTM!The addition of
FEATURES=data-consistency-checkto the Docker build arguments is consistent with similar feature flags used in other jobs (lines 40, 71) and correctly enables the data consistency checks required by the new e2e test suite.tests/e2e_tests/README.md (1)
1-16: LGTM!The documentation accurately reflects the refactoring from storage compatibility to data compatibility testing, with correct hyperlinks and path references to the generation script.
tests/e2e_tests/test_data_compatibility.py (3)
34-54: LGTM!The download implementation correctly streams large archives to disk, uses appropriate timeouts, and handles network errors gracefully by skipping the test. The UUID-based filename prevents conflicts in parallel execution.
56-77: LGTM!The extraction methods correctly handle nested archives and optional snapshot files, with appropriate cleanup to avoid disk space accumulation during test runs.
79-106: LGTM!The collection validation correctly ensures all expected collections are present and report "ok" status. The broad exception handling is acceptable in test code for robustness, as errors are logged and properly propagated via return value.
# Conflicts: # tests/storage-compat/storage-compatibility.sh
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
tests/e2e_tests/test_data_compatibility.py (2)
18-31: Use immutable tuple for collection names.The
EXPECTED_COLLECTIONSattribute is a mutable list used as a constant. Use a tuple for immutability and clarity, or annotate withClassVarif it must remain a list.Apply this diff:
- EXPECTED_COLLECTIONS = [ + EXPECTED_COLLECTIONS = ( "test_collection_vector_memory", "test_collection_vector_on_disk", "test_collection_vector_on_disk_threshold", "test_collection_scalar_int8", "test_collection_product_x64", "test_collection_product_x32", "test_collection_product_x16", "test_collection_product_x8", "test_collection_binary", "test_collection_mmap_field_index", "test_collection_vector_datatype_u8", "test_collection_vector_datatype_f16" - ] + )
84-88: Consider more specific exception handling.The broad
Exceptioncatch may hide unexpected errors. Consider catching more specific exceptions fromClientUtils.list_collections_names()or allowing unexpected errors to propagate for better debugging.For example:
try: collections = client.list_collections_names() - except Exception as e: + except (ConnectionError, TimeoutError) as e: print(f"Error listing collections: {e}") return False
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
tests/e2e_tests/test_data_compatibility.py(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
tests/e2e_tests/test_data_compatibility.py (4)
tests/e2e_tests/models.py (1)
QdrantContainerConfig(109-177)tests/e2e_tests/conftest.py (4)
qdrant_container_factory(118-217)docker_client(26-32)qdrant_image(46-114)temp_storage_dir(221-249)tests/e2e_tests/client_utils.py (4)
ClientUtils(14-359)list_collections_names(54-62)get_collection_info_dict(64-80)wait_for_server(24-38)tests/e2e_tests/utils.py (1)
extract_archive(197-269)
🪛 Ruff (0.13.1)
tests/e2e_tests/test_data_compatibility.py
18-31: Mutable class attributes should be annotated with typing.ClassVar
(RUF012)
86-86: Do not catch blind exception: Exception
(BLE001)
103-103: Do not catch blind exception: Exception
(BLE001)
110-110: Unused method argument: docker_client
(ARG002)
110-110: Unused method argument: qdrant_image
(ARG002)
110-110: Redefinition of unused qdrant_container_factory from line 7
(F811)
129-129: Unused method argument: docker_client
(ARG002)
129-129: Unused method argument: qdrant_image
(ARG002)
129-129: Redefinition of unused qdrant_container_factory from line 7
(F811)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
- GitHub Check: integration-tests-consensus
- GitHub Check: test-consensus-compose
- GitHub Check: integration-tests
- GitHub Check: e2e-tests
- GitHub Check: test-shard-snapshot-api-s3-minio
- GitHub Check: test-consistency
- GitHub Check: lint
- GitHub Check: rust-tests (macos-latest)
- GitHub Check: rust-tests (windows-latest)
- GitHub Check: rust-tests (ubuntu-latest)
- GitHub Check: rust-tests-no-rocksdb (ubuntu-latest)
🔇 Additional comments (7)
tests/e2e_tests/test_data_compatibility.py (7)
33-54: LGTM!The streaming download implementation properly handles large archives, uses specific exception types, and generates unique filenames to support parallel test execution.
56-66: LGTM!The storage extraction logic correctly handles nested archives and cleans up intermediate files.
68-77: LGTM!The snapshot extraction logic correctly handles the optional snapshot file and returns
Nonewhen not present.
97-105: Broad exception handling is acceptable here for robustness.The
Exceptioncatch on line 103 ensures the compatibility check remains resilient to various collection info retrieval failures and provides helpful diagnostics.
109-126: LGTM!The storage compatibility test correctly downloads, extracts, mounts storage data, and validates collections. The
docker_clientandqdrant_imageparameters are required as fixture dependencies (pytest will inject them forqdrant_container_factory), so the unused argument warnings are false positives.
128-150: LGTM!The snapshot compatibility test correctly handles snapshot extraction, mounting, and recovery validation. The read-only mount and snapshot command arguments are appropriate. The
docker_clientandqdrant_imageparameters are required as fixture dependencies.
15-16: All version pins are still valid. Compatibility archives for v1.14.0 and v1.15.4 are released and archived (v1.14.0 – Apr 22 2025(github.com); v1.15.4 – Aug 27 2025(github.com)), and PR #7321 did not modify the test_data_compatibility.py file, so no changes are needed.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
tests/e2e_tests/test_data_compatibility.py (1)
18-31: Consider annotating mutable class attributes withClassVar.For clarity and to satisfy static analysis, class-level mutable collections should be annotated with
typing.ClassVar.Apply this diff:
+from typing import ClassVar + class TestStorageCompatibility: """Test storage and snapshot compatibility with defined previous Qdrant versions.""" PREV_PATCH_VERSION = "v1.14.1" PREV_MINOR_VERSION = "v1.15.4" - EXPECTED_COLLECTIONS = [ + EXPECTED_COLLECTIONS: ClassVar[list[str]] = [ "test_collection_vector_memory", "test_collection_vector_on_disk", "test_collection_vector_on_disk_threshold", "test_collection_scalar_int8", "test_collection_product_x64", "test_collection_product_x32", "test_collection_product_x16", "test_collection_product_x8", "test_collection_binary", "test_collection_mmap_field_index", "test_collection_vector_datatype_u8", "test_collection_vector_datatype_f16" ]
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
tests/e2e_tests/test_data_compatibility.py(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
tests/e2e_tests/test_data_compatibility.py (4)
tests/e2e_tests/models.py (1)
QdrantContainerConfig(109-177)tests/e2e_tests/client_utils.py (4)
ClientUtils(14-359)list_collections_names(54-62)get_collection_info_dict(64-80)wait_for_server(24-38)tests/e2e_tests/utils.py (1)
extract_archive(197-269)tests/e2e_tests/conftest.py (4)
docker_client(26-32)qdrant_image(46-114)temp_storage_dir(221-249)qdrant_container_factory(118-217)
🪛 Ruff (0.13.1)
tests/e2e_tests/test_data_compatibility.py
18-31: Mutable class attributes should be annotated with typing.ClassVar
(RUF012)
86-86: Do not catch blind exception: Exception
(BLE001)
103-103: Do not catch blind exception: Exception
(BLE001)
110-110: Unused method argument: docker_client
(ARG002)
110-110: Unused method argument: qdrant_image
(ARG002)
129-129: Unused method argument: docker_client
(ARG002)
129-129: Unused method argument: qdrant_image
(ARG002)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
- GitHub Check: e2e-tests
- GitHub Check: test-shard-snapshot-api-s3-minio
- GitHub Check: rust-tests (macos-latest)
- GitHub Check: rust-tests (ubuntu-latest)
- GitHub Check: test-consistency
- GitHub Check: rust-tests-no-rocksdb (ubuntu-latest)
- GitHub Check: integration-tests
- GitHub Check: rust-tests (windows-latest)
- GitHub Check: integration-tests-consensus
- GitHub Check: lint
🔇 Additional comments (6)
tests/e2e_tests/test_data_compatibility.py (6)
1-9: LGTM!The imports are correct and well-organized. The
qdrant_container_factoryis properly referenced as a pytest fixture parameter rather than being imported.
33-54: LGTM!The download implementation properly streams large archives to avoid memory issues and handles errors appropriately by skipping the test.
56-77: LGTM!The extraction methods correctly handle nested archives and properly clean up temporary files.
79-106: LGTM!The collection validation logic correctly verifies all expected collections are present and healthy. The broad exception catching on lines 86 and 103 is appropriate here since this is a health-check method in test code that needs to handle various failure modes gracefully.
109-127: LGTM!The storage compatibility test correctly downloads, extracts, mounts storage, and verifies collections. The
docker_clientandqdrant_imageparameters are required by pytest's fixture dependency system (they're dependencies ofqdrant_container_factory) even though not directly referenced in the method body.
128-150: LGTM!The snapshot compatibility test properly verifies snapshot recovery functionality. As with the storage test, the
docker_clientandqdrant_imageparameters are required by pytest's fixture system.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
tests/e2e_tests/test_data_compatibility.py (3)
18-31: Annotate class attributes with ClassVar to satisfy Ruff (RUF012) and intent.These are constants; mark them as ClassVar and add typing import.
@@ -from pathlib import Path +from pathlib import Path +from typing import ClassVar, List @@ class TestStorageCompatibility: @@ - PREV_PATCH_VERSION = "v1.14.0" - PREV_MINOR_VERSION = "v1.15.4" + PREV_PATCH_VERSION: ClassVar[str] = "v1.14.0" + PREV_MINOR_VERSION: ClassVar[str] = "v1.15.4" @@ - EXPECTED_COLLECTIONS = [ + EXPECTED_COLLECTIONS: ClassVar[List[str]] = [ "test_collection_vector_memory", "test_collection_vector_on_disk", "test_collection_vector_on_disk_threshold", "test_collection_scalar_int8", "test_collection_product_x64", "test_collection_product_x32", "test_collection_product_x16", "test_collection_product_x8", "test_collection_binary", "test_collection_mmap_field_index", "test_collection_vector_datatype_u8", "test_collection_vector_datatype_f16" ]
110-110: Remove unused fixture parameters (ARG002).docker_client and qdrant_image are dependencies of qdrant_container_factory; they don’t need to be in the test signature.
-def test_storage_compatibility(self, docker_client, qdrant_image, temp_storage_dir, version, qdrant_container_factory): +def test_storage_compatibility(self, temp_storage_dir, version, qdrant_container_factory): @@ -def test_snapshot_compatibility(self, docker_client, qdrant_image, temp_storage_dir, version, qdrant_container_factory): +def test_snapshot_compatibility(self, temp_storage_dir, version, qdrant_container_factory):Also applies to: 129-129
122-123: Increase server wait to reduce flakiness on heavy datasets/snapshots.30s can be tight; bump to 90–120s.
- if not client.wait_for_server(): + if not client.wait_for_server(timeout=90): pytest.fail(f"Server failed to start for {version}") @@ - if not client.wait_for_server(): + if not client.wait_for_server(timeout=120): pytest.fail(f"Server failed to start from snapshot for {version}")Also applies to: 146-147
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
tests/e2e_tests/test_data_compatibility.py(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
tests/e2e_tests/test_data_compatibility.py (4)
tests/e2e_tests/models.py (1)
QdrantContainerConfig(109-177)tests/e2e_tests/client_utils.py (4)
ClientUtils(14-359)list_collections_names(54-62)get_collection_info_dict(64-80)wait_for_server(24-38)tests/e2e_tests/utils.py (1)
extract_archive(197-269)tests/e2e_tests/conftest.py (4)
docker_client(26-32)qdrant_image(46-114)temp_storage_dir(221-249)qdrant_container_factory(118-217)
🪛 Ruff (0.13.1)
tests/e2e_tests/test_data_compatibility.py
18-31: Mutable class attributes should be annotated with typing.ClassVar
(RUF012)
86-86: Do not catch blind exception: Exception
(BLE001)
103-103: Do not catch blind exception: Exception
(BLE001)
110-110: Unused method argument: docker_client
(ARG002)
110-110: Unused method argument: qdrant_image
(ARG002)
129-129: Unused method argument: docker_client
(ARG002)
129-129: Unused method argument: qdrant_image
(ARG002)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
- GitHub Check: rust-tests (ubuntu-latest)
- GitHub Check: rust-tests-no-rocksdb (ubuntu-latest)
- GitHub Check: test-consistency
- GitHub Check: test-shard-snapshot-api-s3-minio
- GitHub Check: rust-tests (windows-latest)
- GitHub Check: rust-tests (macos-latest)
- GitHub Check: integration-tests
- GitHub Check: lint
- GitHub Check: integration-tests-consensus
🔇 Additional comments (1)
tests/e2e_tests/test_data_compatibility.py (1)
12-31: Nice migration and clear structure.Good replacement of the bash flow with readable pytest, version params, and helpers.
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
tests/e2e_tests/utils.py(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
- GitHub Check: rust-tests (ubuntu-latest)
- GitHub Check: rust-tests-no-rocksdb (ubuntu-latest)
- GitHub Check: rust-tests (macos-latest)
- GitHub Check: rust-tests (windows-latest)
- GitHub Check: integration-tests
- GitHub Check: test-shard-snapshot-api-s3-minio
- GitHub Check: test-consistency
- GitHub Check: e2e-tests
- GitHub Check: test-consensus-compose
- GitHub Check: integration-tests-consensus
- GitHub Check: lint
* Add e2e test for compatibility
This is a next PR in a series of PRs to replace some of the old bash tests. The goal is to reduce the number of jobs run in Integration tests workflow and unify running of the tests.
This replaces tests/storage-compat/storage-compatibility.sh