Fixing Bugs in Benchmarking ClickHouse with vectordbbench #523
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
/assign @XuanYang-cn |
alwayslove2013
approved these changes
May 13, 2025
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: alwayslove2013, yuyuankang The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Collaborator
|
@yuyuankang Fantastic~ I believe it will greatly help the community better test |
shaharuk-yb
added a commit
to yugabyte/VectorDBBench
that referenced
this pull request
May 15, 2025
* fix: Unable to run vebbench and cli fix: remove comma of logging str fix cli unable to run zilliztech#444 Signed-off-by: yangxuan <[email protected]> * enhance: Unify optimize and remove ready_to_load PyMilvus used to be the only client that uses ready_to_load. Not it'll load the collection when creating it, so this PR removes `ready_to_load` from the client.API Also this PR enhance optimize and remove the optimize_with_size Signed-off-by: yangxuan <[email protected]> * add mongodb client Signed-off-by: zhuwenxing <[email protected]> * add mongodb client in readme Signed-off-by: zhuwenxing <[email protected]> * add some risk warnings for custom dataset - limit the number of test query vectors. Signed-off-by: min.tian <[email protected]> * Bump grpcio from 1.53.0 to 1.53.2 in /install Bumps [grpcio](https://github.com/grpc/grpc) from 1.53.0 to 1.53.2. - [Release notes](https://github.com/grpc/grpc/releases) - [Changelog](https://github.com/grpc/grpc/blob/master/doc/grpc_release_schedule.md) - [Commits](grpc/grpc@v1.53.0...v1.53.2) --- updated-dependencies: - dependency-name: grpcio dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> * add mongodb config Signed-off-by: zhuwenxing <[email protected]> * Opensearch interal configuration parameters (zilliztech#463) * Added the configuration parameters to create Opensearch dynamically with right replicas, shards and other opensearch related configurations. Added the feature to create OS index with 0 replica and once the data is loaded update the replicas according to the parameter. * Updated the readme for config parameters --------- Co-authored-by: xavrathi <[email protected]> * ui control num of concurrencies Signed-off-by: siqi.an <[email protected]> * Update README.md * environs version should <14.1.0 Signed-off-by: min.tian <[email protected]> * Support GPU_BRUTE_FORCE index for Milvus (zilliztech#476) Signed-off-by: Rachit Chaudhary <[email protected]> Co-authored-by: Signed-off-by: Rachit Chaudhary - r0c0axe <[email protected]> * Add table quantization type * Support MariaDB database (zilliztech#375) MariaDB introduced vector support in version 11.7, enabling MariaDB Server to function as a relational vector database. https://mariadb.com/kb/en/vectors/ Now add support for MariaDB server, verified against MariaDB server of version 11.7.1: - Support MariaDB vector search with HNSW algorithm, support filter search. - Support index and search parameters: - storage_engine: InnoDB or MyISAM - M: M parameter in MHNSW vector indexing - ef_search: minimal number of result candidates to look for in the vector index for ORDER BY ... LIMIT N queries. - max_cache_size: Upper limit for one MHNSW vector index cache - Support CLI of `vectordbbench mariadbhnsw`. * Add TiDB backend (zilliztech#484) * Add TiDB backend Signed-off-by: Wish <[email protected]> * Fix Signed-off-by: Wish <[email protected]> * Fix Signed-off-by: Wish <[email protected]> * Improve error handling Signed-off-by: Wish <[email protected]> --------- Signed-off-by: Wish <[email protected]> * CLI fix for GPU index (zilliztech#485) * Support GPU_BRUTE_FORCE index for Milvus Signed-off-by: Rachit Chaudhary <[email protected]> * MilvusGPUBruteForceTypedDict addition Signed-off-by: Rachit Chaudhary <[email protected]> --------- Signed-off-by: Rachit Chaudhary <[email protected]> Co-authored-by: Signed-off-by: Rachit Chaudhary - r0c0axe <[email protected]> * remove duplicated code * feat: initial commit * Add vespa integration * remove redundant empty_field config check for qdrant and tidb Signed-off-by: min.tian <[email protected]> * reformat all Signed-off-by: min.tian <[email protected]> * fix cli crush Signed-off-by: min.tian <[email protected]> * downgrade streamlit version * add more milvus index types: hnsw sq/pq/prq; ivf rabitq Signed-off-by: min.tian <[email protected]> * add more milvus index types: ivf_pq Signed-off-by: min.tian <[email protected]> * Add HNSW support for Clickhouse client (zilliztech#500) * feat: add hnsw support * refactor: minor fixes * feat: reformat code * fix: remove sql injections, reformat code * fix bugs when use custom_dataset without groundtruth file Signed-off-by: min.tian <[email protected]> * fix: prevent the frontend from crashing on invalid indexes in results * fix ruff warnings * Fix formatting * Add lancedb * Add --task-label option for cli (zilliztech#517) * Add --task-label option for cli * Fix lint issues * Add qdrant cli * Update README.md * Fixing Bugs in Benchmarking ClickHouse with vectordbbench (zilliztech#523) * Update cli.py * Update clickhouse.py * Update clickhouse.py * Update cli.py * Update config.py * remove space * Add --concurrency-timeout option to avoid long time waiting (zilliztech#521) * Add --concurrency-timeout option to avoid long time waiting, by default, it's 3600s. * Fix lint error * Update README.md, add --concurrency-timeout option --------- Signed-off-by: yangxuan <[email protected]> Signed-off-by: zhuwenxing <[email protected]> Signed-off-by: min.tian <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: siqi.an <[email protected]> Signed-off-by: Rachit Chaudhary <[email protected]> Signed-off-by: Wish <[email protected]> Co-authored-by: yangxuan <[email protected]> Co-authored-by: zhuwenxing <[email protected]> Co-authored-by: min.tian <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Xavierantony1982 <[email protected]> Co-authored-by: xavrathi <[email protected]> Co-authored-by: siqi.an <[email protected]> Co-authored-by: Xiaofan <[email protected]> Co-authored-by: Rachit Chaudhary <[email protected]> Co-authored-by: Signed-off-by: Rachit Chaudhary - r0c0axe <[email protected]> Co-authored-by: Luca Giacchino <[email protected]> Co-authored-by: Hugo Wen <[email protected]> Co-authored-by: Wenxuan <[email protected]> Co-authored-by: yuyuankang <[email protected]> Co-authored-by: Arseniy Ahtaryanov <[email protected]> Co-authored-by: nuvotex-tk <[email protected]> Co-authored-by: Polo Vezia <[email protected]> Co-authored-by: MansorY <[email protected]> Co-authored-by: Andreas Opferkuch <[email protected]> Co-authored-by: LoveYou3000 <[email protected]> Co-authored-by: Yuyuan Kang <[email protected]>
euphoria0-0
pushed a commit
to CryptoLabInc/VectorDBBench
that referenced
this pull request
Nov 21, 2025
…#523) * Update cli.py * Update clickhouse.py * Update clickhouse.py * Update cli.py * Update config.py * remove space
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, I found three issues when running
vectordbbenchto benchmark a local ClickHouse instance:--user <username>argument invectordbbench clickhouseis not passed to the ClickHouse client, so it always defaults to the username "clickhouse".vector_similaritysyntax is incorrect — the third parameter should specify the dimension.AttributeError: 'NoneType' object has no attribute 'get'.