Change log

Improvements

• #9282 - Log slow operations during shard WAL recovery
• #9137 - Clear cache of ID tracker after building a segment
• #9138 - Don't rebuild payload index if changing on_disk flag

Bug Fixes

• #9285 - Fix potential infinite loop in optimizer when using multi vectors with prevent_unoptimized
• #9217 - Clean up unfinished segment optimizations on cancellation, don't load them on restart
• #9260 - Fix MatchAny with an empty list being rejected on integer index
• #9239 - Add timeout to shard snapshot streaming endpoint, drop connections not actively read from
• #9215 - Fix abort transfer with resharding not being idempotent
• #9237 - Fix reporting old progress in new snapshot transfers
• #9226 - Fix WAL lock error on Android platforms

Security

• #9254 - Fix REST auth whitelist bypass on specially crafted paths, resolve route before authorizing
• #9268 - Fix out of bound heap read with malicious snapshot by rejecting incorrect length

Change log

Improvements

• #8988 - Refactor quantized multi-vector scorers for io_uring support
• #9058 - Validate vector dimensions before WAL write for async upserts

Bug Fixes

• #8990 - Notify pending consensus ops on snapshot apply
• #9054 - Fix indexed integer range filter with float values
• #9055 - Fix {match: {except: []}} returning zero results with payload index
• #9070 - Fix empty vector panic
• #9099 - Fix TurboQuant heap memory under-reporting
• #9107 - Do not use unnecessary sequential mmaps
• #9014 - Fix resharding cleanup data race with update queue
• #9120 - Use cluster default shard transfer method for fallback
• #9121 - Recreate workers/optimizers async to not block consensus
• #9110 - Restart optimizer on conflicting vector schema
• #9122 - Do not remove shard for snapshot transfer receiver
• #9125 - Fix snapshot transfer cancellation on external malformed requests
• #9116 - Fix deleted mask race on optimization and snapshot creation

Security

• #9031 - Authorize request before we accept snapshot file upload

Change log

Features 🏎️

• milestone!48 - Add TurboQuant quantization variant, 8x vector compression without the recall tax
• #8605 - Add API to create and delete named vectors in existing collection
• #8606 - Deep memory reporting, show memory usage breakdown for storage components
• #8714 - Add low memory mode, force open everything on disk to minimize out of memory crash on startup
• #8715 - Add strict mode parameter to reject updates when memory usage is high (max_resident_memory_percent)

Improvements 📏

• #8790, #8769 - Use dynamic CPU pool for search workers, improve search performance when there's high IO wait
• #8526, #8656 - Use operation size based batching in shard transfers to achieve higher throughput
• #8673 - Reduce memory usage of immutable geo index by 7x
• #8529 - Fully remove RocksDB support, simplifying storage handling
• #8768 - Don't stall update queue when waiting for deferred points
• #8944 - Use default timeout of 60 seconds to update requests, clean up long running update requests
• #8767 - When collection update triggers optimizer recreation, clear any optimizer errors to allow recovery without restart
• #8784 - Use snapshot based shard transfers for much faster recovery if WAL delta cannot be resolved
• #8782 - On snapshot transfer, first clear shard data on receiver to prevent out of disk crash
• #8689 - Clean temporary directories before loading collections, may prevent out of disk crash on recovery
• #8830 - Don't hold shard holder lock during search to prevent blocking various other operations
• #8626, #8635 - Record API method path in audit log events
• #8402 - Report tracing ID in audit log events
• #8636 - Report audit log configuration and data size in telemetry and metrics
• #8640 - Optimize cosine normalization on ARM CPUs (SIMD)
• #8571 - Skip audit logging on telemetry endpoints
• #8693 - Make HNSW sub graph building deterministic
• #8712, #8708, #8706 - Report various structures in deterministic order
• #8932 - Link to agentic skills in startup message

Bug Fixes 🪺

• #8728 - Fix stop words always being lowercased over gRPC even if disabled
• #8719 - Fix datetime parsing for YYYY-MM-DDTHH:MM format
• #8734 - Fix IsEmpty condition on freshly rebuilt null index
• #8939 - Fix nested MatchTextAny not using full-text index
• #8646 - Fix geo circle hash bound checks
• #8938 - Fix phrase matching crossing string-array element boundaries
• #8762 - Fix hard panic when there are multiple validation errors
• #8194 - Disallow sparse vectors with an empty name
• #8832 - Fix GPU not being used for HNSW without main graph
• #8691 - Fix infinite loop in HNSW sub graph building
• #8577 - Fix panic when snapshot path contains non-UTF-8 characters
• #8293 - Fix hard panic on corrupt collection alias file, return error instead
• #8574 - Fix disk IO not always being measured in map index
• #8645 - Fix chunk capacity reservation to allocate more efficiently
• #8651 - Fix payload index flushers absorbing cancellations as errors
• #8474 - Fix snapshot files not cleaned up on snapshot restore failure
• #8654 - Fix clearing page cache not being reliable, add missing candidates
• #8542 - Fix map index listing modified file as immutable
• #8847 - Fix data race causing disk watcher not to be woken up causing timeouts
• #8789, #8917 - Make resharding operations idempotent, fix cluster state corruption on crash
• #8831 - Fix slow request logging for read operations with shard level errors
• #8756 - Don't allocate async search workers we don't use reducing thread count
• #8596 - Don't write to old shards on resharding down after write hashring commit, fix missing shard errors
• #8543 - Fix edge sparse vector search panic on score post processing
• #8588 - Fix various Raft consensus correctness bugs upstream

Security 🕵️

• #8676 - Enforce API key/JWT authentication on internal gRPC endpoints
• #8628 - Add config option to disable snapshot restore from URL
• #8619 - Bump TLS dependencies to resolve security vulnerabilities

Web UI 🍱

• qdrant/qdrant-web-ui#372 - Add memory and disk inspector, view usage breakdown per storage component
• qdrant/qdrant-web-ui#370 - Add high contrast theme
• qdrant/qdrant-web-ui#369 - Render inline documentation in collection info panel
• qdrant/qdrant-web-ui#362 - Add refresh button to some collection views
• qdrant/qdrant-web-ui#373, qdrant/qdrant-web-ui#377 - Fix search quality latency measurement and use better naming

Change log

Improvements

• milestone#46 - Defer point updates, efficiently apply and optimize points with prevent_unoptimized=true
• #8188 - Make Gridstore flushes non-blocking to reduce search tail latencies
• #8235 - Improve performance of filtered search in case of singular payload value
• #8402 - Add request tracing ID into audit log
• #8460 - Propagate WAL errors instead of panicking during loading of shards
• #8301 - Allow peer to bootstrap with used URI if empty

Bug fixes

• #8177 - GPU: fix raw Vulkan name pointer type
• #8193 - Fix creation of uninitialized shard key with replication factor > 1, fixing tiered multi-tenancy workflow
• #8217 - Prevent min_should panic on large amount of conditions
• #8179 - Fix for restore of cluster snapshots which creates unnecessary replicas in Partial state
• #8220 - Fix Server disconnected without sending a response error while performing concurrent ingestion using
• #8341 - Security patch to force snapshot recovery from snapshot directory only
• #8373 - Fix lock for collection-level operations during shard transfer
• #8438 - Do not fail on collection-level operations with dummy shard
• #8455 - Fix WAL reading issue introduced in 1.17.0
• #8454 - Fix another panic in WAL replay
• #8475 - Fix for WAL delta transfer: reject truncated recovery points before equal-pruning
• #8514 - Fix incorrect warning in geo-index
• #8374 - Prevent locking shard holder for a long time in slow stream records transfers
• #8480 - Fix panic in optimizer logging
• #8449 - Fix panic in chunked vector storage

Preview

This features are not officially released yet, but available in the build:

• #8214 - Per-collection metrics in Prometheus
• #8498 - API for reading audit log entries
• #8469 - Maximum batch size config in strict mode

Edge

• Rust Crate for Qdrant Edge v0.6.0 - https://crates.io/crates/qdrant-edge
• Python Package for Qdrant Edge v0.6.0 - https://pypi.org/project/qdrant-edge-py/

Change log

Features 🏋️

• milestone#38 - Relevance Feedback (docs)
• milestone#44 - API for detailed report on optimization progress and stages (docs)
• milestone#40 - API for aggregated telemetry of the whole cluster (docs)
• milestone#43 - Unlimited update queue to gracefully smooth update spikes (docs)
• #8071 - Add Audit Access Logging (docs)
• #8063 - Add Weighted RRF (docs)
• #7643 - Add config option to control update throughput and prevent unoptimized searches (docs)
• #7929 - Add configurable read fan-out delay for dealing with tail latency in distributed clusters (docs)
• #7963 - For upserts, add update_mode parameter to either upsert, update or insert (docs)
• #7835 - Add secondary API key configuration for zero downtime key rotation in distributed clusters
• #7838 - Add dedicated HTTP port for /metrics endpoint for internal monitoring
• #7615 - Add API to list shard keys (docs)

Improvements 🤸

• #7802 - Improve timeout handling on read operations
• #7750 - Improve timeout handling in update operations, prevent shard failures in case of timed out updates after WAL
• #8025 - Recover snapshot without creating intermediate files, greatly improves recovery time and disk usage
• #8059 - Recover snapshots directly into target file system to avoid expensive file movements
• #7883 - Flush after snapshot unpack with syncfs to persist a large number of files much more efficiently
• #8072 - Don't lock shard holder structure during creation of a snapshot, previously blocking shard level operations
• #8166 - Add timeout to snapshot downloads, abort if connection gets stuck for more than a minute
• #8007, #8056 - Improve segments locking approach to minimize lock contention
• #8105 - Limit number of parallel updates on a shard to 64 to prevent order tracking overhead
• #8169 - Reduce locking in Gridstore to lower search tail latencies
• #8164 - Actively free cache memory for closed WAL segments to reduce memory pressure
• #7952 - Disable in-place payload updates on unindexed fields, improve immutability guarantees of indexed segments improving partial snapshots
• #7887 - Add ability to disable extra HNSW links construction for specific payload indices (docs)
• #7971 - Enable missing option for vector storage to populate single-file mmap
• #7928 - Enable io_uring when reading batch of vectors
• #7919 - Improve error message for datetime parse failures
• #8053 - Allow to configure load concurrency for collections, shards and segments
• #7809 - Add more convenient way to provide API-keys for external inference providers (docs)
• #8093 - Don't lock WAL during serialization of new updates, which was costly for large operations
• #7834 - Extend WAL retention when replicas are dead, prevent full shard transfers in case of peer failures
• #7565 - Disable old shard key format deprecated in 1.15.0
• #8125 - Skip building extra HNSW links for deleted vectors
• #8163 - Improve search result processing to use less CPU with a high search limit
• #8175 - Use less allocations for HNSW plain filtered search

Bug Fixes 🤹

• #7850 - Fix flush ordering to follow segment dependencies, prevents dataloss by CoW on flush interruption
• #8103 - Fix data race in stream records transfer potentially missing ongoing updates
• #7983 - Fix interlocking problem on creation of payload index
• #7999 - Fix interlocking problem on collection-level update operations
• #8131 - Fix deadlock during snapshot with concurrent updates
• #8128 - Fix gRPC/HTTP2 too_many_internal_resets error due to how we internally cancel ongoing requests
• #8019 - Improve handling of HTTP2 channels closing in connection pool
• #8104 - Fix data race in WAL and shard clocks snapshot, ensure they remain consistent
• #7961 - Fix using incorrect versions in partial snapshot manifest construction
• #8095 - Fix incorrect internal protocol usage for shard snapshot transfers
• #7950 - Fix integer overflow in query batch when using high limits
• #7972 - Fix search aggregator panic with limit 0
• #8100 - Fix round floats not used in integer index, JSON doesn't distinguish between integers and floats
• #8097 - Fix score_threshold not being used in score boosting queries
• #7877 - Fix Corrupted ID tracker mapping storage bug when disk is full
• #7944 - Fix gRPC API response status counting in telemetry and metrics
• #7857 - Fix total count in progress tracker for replicate points with filter
• #7856 - Fix creation of payload index in empty collection using user-defined sharding
• #8099 - Fix ignoring CA certs for internal requests if configured
• #8176 - Add missing timeout parameter to some endpoints

Web UI 🍱

• qdrant/qdrant-web-ui#345 - Detailed visualization of optimization progress
• qdrant/qdrant-web-ui#334 - Create collection dialog now previews the exact command
• qdrant/qdrant-web-ui#341 - Buttons for resharding control
• qdrant/qdrant-web-ui#344 - Points filter and search bar restored and improved

Qdrant Edge 🔪

Qdrant Edge is an in-process version of Qdrant, which shares the same internals, storage format, and points API as the server version, but designed to work locally. Qdrant Edge is compatible with server version and it can read shard snapshots created by server version of Qdrant. More documentation available here.

Deprecations ⚠️

• Starting from v1.17.0 Qdrant changes response format for vector fields in gRPC interface. All official Qdrant clients should be already adopted to this change, so please make sure you upgrade your client libraries and check that you are not using deprecated fields. More info: #7183

• Upcoming deprecations:

  • In Qdrant v1.18.x all deprecated search methods will be completely removed and won't be available even from old client libraries.
  • In Qdrant v1.17.x we will completely remove RocksDB support in favor of gridstore, that means that direct upgrade from v1.15.x into v1.17.x won't be possible. Please follow upgrade instructions and upgrade one minor version at a time to avoid unsupported storage errors. Note that Qdrant Cloud infrastructure automatically generates a proper upgrade steps, so you don't have to worry about that.

Change log

Improvements

• #7755, #7588 - Respect search and point retrieve timeout when trying to access segments
• #7685 - Respect telemetry timeout when fetching shard statistics
• #7715 - Log snapshot download duration and speed

Bug fixes

• #7787, #7791 - Fix WAL delta transfer corrupting replica after a previous full transfer was aborted
• #7801, #7805 - Fix flush losing changes on transient disk IO errors, potentially corrupting data
• #7792 - Fix incorrectly aborting shard transfers when dropping unrelated shard
• #7741 - Fix flush error in Gridstore, potentially corrupting data when quickly alternating inserts/deletes
• #7702 - Fix flush data race in Gridstore, potentially corrupting data when storage is cleared in parallel
• #7759, #7782 - Fix handling of collection names with weird characters, breaking snapshot transfers for example
• #7788 - Fix snapshot metrics not always reporting when zero (snapshot_{creation,recovery}_running, snapshot_created_total)
• #7783 - Fix incorrectly reporting optimization errors in telemetry on panic
• #7765 - Fix slow shutdown on SIGINT when optimizations are running
• #7690 - Fix Qdrant not building on Windows ARM64
• #7683 - Keep RocksDB support until 1.18.0 in development builds

Change log

Improvements

• #7607 - Improve request timeout handling for telemetry and metrics
• #7623 - Add user agent to HTTP requests sent by Qdrant server

Bug fixes

• #7674 - Fix critical WAL bug that could break consensus or cause data corruption on restart
• #7684 - Fix consensus crash when applying consensus snapshot with non-replicated collection
• #7620 - Fix panic during search on segments with empty HNSW graph
• #7629, #7640, #7673 - Fix shard resource cleanup when shard is replaced, prevent deadlock on small CPUs
• #7621, #7626 - Fix payload index storage still flushing after removal, fixing data corruption and IO errors on Windows
• #7624, #7627 - Fix Gridstore storage still flushing after wipe, fixing data corruption and IO errors
• #7614, #7618 - Fix Docker/WSL on Windows with bind mount corrupting storage
• #7678 - Fix collections_vector_total metric reporting -0.0 if there are no vectors
• #7649 - Also report collection_indexed_only_excluded_points metric if zero

Change log

Improvements

• #7514, #7572 - Make batch queries up to 3 times faster on full scans by reading each point only once
• #7551 - Actively migrate vector, payload and payload index storage from RocksDB into Gridstore on startup for better and more predictable performance
• #7579 - Add 60s internal timeout for telemetry/metrics endpoints to prevent long hanging tasks
• #7557 - Add validation to restart shard transfer operation
• #7446 - Defer Gridstore flushing to make flushing behavior consistent with all other storage components
• #7580 - Improve consensus WAL compaction logging to aid debugging
• #7598 - Make timeout for inference requests user configurable

Bug fixes

• #7564 - Fix panic at startup on old clusters with user defined sharding, if not updated to Qdrant 1.15.5 first
• #7577 - Fix breaking Raft by killing node at specific time during consensus snapshot, preventing potential crash loop
• #7587 - Fix corrupting WAL with broken flush edge case after WAL is cleared or truncated
• #7570 - Fix incorrect rescoring default on mutable segments when using binary quantization
• #7569, #7575 - Spawn search/update tasks on the correct runtime, significantly reduce number of general/actix threads
• #7558 - Fix data race in shard transfers, wait on transfer to start before initializing shards
• #7556 - Fix incorrect log message when failing to read lock segment for some time

Change log

Features 🌰

• https://github.com/qdrant/qdrant/milestone/33?closed=1 - Inline Storage: Add option to inline vectors in HNSW graph for efficient IO usage (docs)
• https://github.com/qdrant/qdrant/milestone/37?closed=1 - Tenant promotion mechanism for tiered multitenancy: (docs)
  • Add ReplicatePoints action to promote payload based tenant into dedicated shard key (docs)
  • Add fallback shard key for intelligent routing to tenants that are or are not promoted to a dedicated shard (docs)
  • On shard key creation, allow to specify initial state of new replicas
• #7414 - Add ACORN-1 search method, accurate search over many filtered points at the cost of performance (docs)
• #7408 - Add ASCII folding (normalization) to full text indices, fold diacritics into ASCII characters (docs)
• #7006 - Add conditional update functionality, only apply update on points matching filter (docs)
• #7100 - Add text_any full text filter to match any query term (docs)
• #7065 - Add option to customize RRF k parameter (docs)
• #7222 - In strict mode, specify maximum number of payload indices per collection
• #7123 - Add custom key-value metadata to collections (docs)
• #7291 - Add profiler to log slow point update and read requests

Improvements 🏎️

• #7385 - When loading Gridstore, populate tracker data into memory for faster first access
• #7407, #7405 - Spawn updates and flush workers task on updates runtime, use two system threads less for each local shard
• #7413 - Use system thread on demand in WAL, use one less system thread per local shard by default
• #7468 - Use atomic bit flags on HNSW construction to significantly increase indexing performance
• #7052, #7471 - Implement AVX512 SIMD optimizations for binary quantization on modern x86_64 CPUs
• #7433 - Switch new mutable payload indices and storage from RocksDB to Gridstore for better performance
• #7508 - Enable quantization in appendable segments by default, improving search performance
• #7347 - Change default score of query-less prefetch to 1.0 to ease score boosting
• #7369 - Don't explicitly disable strict mode by default
• #7345 - Simplify internal handling of copy-on-write segments, now write incoming updates to dedicated segments directly
• #7293, #7523 - Add warnings field to collection info, report misconfiguration
• #7319, #7401 - Report more helpful error messages on file IO errors
• #7377 - When using inference, propagate rate limit responses
• #7434 - Rate limit slow request warning to prevent spamming logs
• #7373 - Log shard transfers as a result of the consensus recovery procedure
• #7337 - Tone down optimizer logging to make it less verbose
• #7370 - Add TARGET_CPU and JEMALLOC_SYS_WITH_LG_PAGE build parameters to Docker image

Metrics 📈

• #7302, #7441 - In metrics, report point and vector counts per collection and vector name (collection_{points,vectors})
• #7307 - In metrics, report number of points skipped in indexed_only search (collection_indexed_only_excluded_points)
• #7301 - In metrics, report global effective minimum and maximum shard replication count (collection_active_replicas_{min,max})
• #7310, #7516 - In metrics, report total number of non-active replicas (collection_dead_replicas)
• #7316, #7480 - In metrics, expose collection_running_optimizations with number of optimizers running per collection
• #7497 - In metrics, report running, created and recovered number of snapshots (snapshot_{creation,recovery}_running, snapshot_created_total)
• #7484 - In metrics, report active thread count (process_threads)
• #7451 - In metrics, report number of open file descriptors and the limit (process_{open,max}_fds)
• #7487 - In metrics, report open mmaps and system limit (process_open_mmaps, system_max_mmaps)
• #7482 - In metrics, report total number of major and minor page faults (process_{minor,major}_page_faults_total)
• #7438 - In metrics, add configuration option to prefix all metrics with qdrant_ or something else

Bug fixes 🕵️

• #7527 - Fix logger API allowing arbitrary file writes, now this can only be configured through configuration
• #7530 - Abort pending search tasks when search is cancelled, fixing optimizer instability under huge load
• #7533 - Abort other blocking tasks such as retrieve and snapshot prematurely if the caller gets cancelled
• #7531 - Cancel ongoing searches more aggressively if the search is cancelled
• #7517 - Fix resharding down panic if no shard key is provided on collection with custom sharding
• #7469 - Fix panic on certain queries with unknown vector name
• #7375 - Forbid peer to join cluster with URI that is already used, which could break a cluster
• #7400 - Fix corrupt segments on load if segment was partially flushed, prevent payload index corruption
• #7342, #7404, #7427 - Force flush all segments when taking snapshot to prevent data corruption
• #7381 - Fix flush ordering on segments currently being snapshotted, fixing data consistency on crash
• #7388, #7416 - Fix flush ordering with concurrent flushes to ensure data consistency
• #7512, #7521 - Fix dummy shards not allowing snapshot recovery

Web UI 🫂

• https://github.com/qdrant/qdrant-web-ui/releases/tag/v0.2.0 - New web UI design to match Qdrant Cloud

Deprecations 🚧

• #7454 - Remove init_from collection API, deprecated since Qdrant 1.15
• #7449 - Remove lock API, deprecated since Qdrant 1.15
• #7047 - Remove old internal shard key format, deprecated and migrated away from in Qdrant 1.15
• #7450 - Remove payload filter from RBAC/JWT, deprecated since Qdrant 1.15, API keys using it are rejected
• #7183 - Deprecate old variant of vector output

Change log

Improvements

• #7157 - Acknowledge update/delete by filter operations on flush, preventing very slow restart
• #7217, #7218, #7219, #7220, #7221 - Add API validation to min_should, filters, point update batch and others
• #7235 - Add timeout parameter to remove peer operation
• #7320 - Decrease internal update batch sizes to minimize search latency spikes on large user batches
• #7233 - Limit number of segments loaded in parallel, preventing potential OOM on large clusters
• #7222 - Add strict mode configuration to specify max number of payload indices
• #7240 - Improve error reporting on mutable ID tracker load errors
• #7151 - Improve error reporting on flush problems
• #7244 - Remove vector count field from collection info
• #7177 - Do not anonymize peer ID in telemetry data

Bug fixes

• #7263 - Fix not flushing mutable ID tracker files after creation, potentially causing segment corruption
• #7248 - Fix data race at the end of snapshot creation causing missing points
• #7298, #7306 - Fix data race during snapshots corrupting point data if a point is moved
• #7241, #7265, #7267, #7269, #7277 - Fix potential deadlock in REST runtime while streaming shard snapshot
• #7015 - Fix potential deadlock on REST server runtime
• #7172 - Fix potential recursive deadlock when fetching all vectors
• #7203 - Fix potential deadlock in scroll API
• #7303 - Fix incorrectly deleting old point versions from segments
• #7194 - Fix upsert operations with duplicate point IDs not being applied properly
• #7252 - Fix phrase matching ignoring unknown tokens
• #7264 - Fix strict mode validation on nested filters