{"@attributes":{"version":"2.0"},"channel":{"title":"Manage Data on Qdrant - Vector Search Engine","link":"https:\/\/qdrant.tech\/documentation\/manage-data\/","description":"Recent content in Manage Data on Qdrant - Vector Search Engine","generator":"Hugo","language":"en-us","managingEditor":"info@qdrant.tech (Andrey Vasnetsov)","webMaster":"info@qdrant.tech (Andrey Vasnetsov)","item":[{"title":"Points","link":"https:\/\/qdrant.tech\/documentation\/manage-data\/points\/","pubDate":"Mon, 01 Jan 0001 00:00:00 +0000","author":"info@qdrant.tech (Andrey Vasnetsov)","guid":"https:\/\/qdrant.tech\/documentation\/manage-data\/points\/","description":"<h1 id=\"points\">Points<\/h1>\n<p>The points are the central entity that Qdrant operates with.\nA point is a record consisting of a <a href=\"https:\/\/qdrant.tech\/documentation\/manage-data\/vectors\/\">vector<\/a> and an optional <a href=\"https:\/\/qdrant.tech\/documentation\/manage-data\/payload\/\">payload<\/a>.<\/p>\n<p>It looks like this:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" class=\"chroma\"><code class=\"language-json\" data-lang=\"json\"><span class=\"line\"><span class=\"cl\"><span class=\"c1\">\/\/ This is a simple point\n<\/span><\/span><\/span><span class=\"line\"><span class=\"cl\"><span class=\"c1\"><\/span><span class=\"p\">{<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"nt\">&#34;id&#34;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">129<\/span><span class=\"p\">,<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"nt\">&#34;vector&#34;<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span><span class=\"mf\">0.1<\/span><span class=\"p\">,<\/span> <span class=\"mf\">0.2<\/span><span class=\"p\">,<\/span> <span class=\"mf\">0.3<\/span><span class=\"p\">,<\/span> <span class=\"mf\">0.4<\/span><span class=\"p\">],<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"nt\">&#34;payload&#34;<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span><span class=\"nt\">&#34;color&#34;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&#34;red&#34;<\/span><span class=\"p\">},<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"><span class=\"p\">}<\/span>\n<\/span><\/span><\/code><\/pre><\/div><p>You can search among the points grouped in one <a href=\"https:\/\/qdrant.tech\/documentation\/manage-data\/collections\/\">collection<\/a> based on vector similarity.\nThis procedure is described in more detail in the <a href=\"https:\/\/qdrant.tech\/documentation\/search\/search\/\">search<\/a> and <a href=\"https:\/\/qdrant.tech\/documentation\/search\/filtering\/\">filtering<\/a> sections.<\/p>\n<p>This section explains how to create and manage vectors.<\/p>"},{"title":"Vectors","link":"https:\/\/qdrant.tech\/documentation\/manage-data\/vectors\/","pubDate":"Mon, 01 Jan 0001 00:00:00 +0000","author":"info@qdrant.tech (Andrey Vasnetsov)","guid":"https:\/\/qdrant.tech\/documentation\/manage-data\/vectors\/","description":"<h1 id=\"vectors\">Vectors<\/h1>\n<p>Vectors (or embeddings) are the core concept of the Qdrant Vector Search engine.\nVectors define the similarity between objects in the vector space.<\/p>\n<p>If a pair of vectors are similar in vector space, it means that the objects they represent are similar in some way.<\/p>\n<p>For example, if you have a collection of images, you can represent each image as a vector.\nIf two images are similar, their vectors will be close to each other in the vector space.<\/p>"},{"title":"Payload","link":"https:\/\/qdrant.tech\/documentation\/manage-data\/payload\/","pubDate":"Mon, 01 Jan 0001 00:00:00 +0000","author":"info@qdrant.tech (Andrey Vasnetsov)","guid":"https:\/\/qdrant.tech\/documentation\/manage-data\/payload\/","description":"<h1 id=\"payload\">Payload<\/h1>\n<p>One of the significant features of Qdrant is the ability to store additional information along with vectors.\nThis information is called <code>payload<\/code> in Qdrant terminology.<\/p>\n<p>Qdrant allows you to store any information that can be represented using JSON.<\/p>\n<p>Here is an example of a typical payload:<\/p>\n<div class=\"highlight\"><pre tabindex=\"0\" class=\"chroma\"><code class=\"language-json\" data-lang=\"json\"><span class=\"line\"><span class=\"cl\"><span class=\"p\">{<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"nt\">&#34;name&#34;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&#34;jacket&#34;<\/span><span class=\"p\">,<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"nt\">&#34;colors&#34;<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span><span class=\"s2\">&#34;red&#34;<\/span><span class=\"p\">,<\/span> <span class=\"s2\">&#34;blue&#34;<\/span><span class=\"p\">],<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"nt\">&#34;count&#34;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">10<\/span><span class=\"p\">,<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"nt\">&#34;price&#34;<\/span><span class=\"p\">:<\/span> <span class=\"mf\">11.99<\/span><span class=\"p\">,<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"nt\">&#34;locations&#34;<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"p\">{<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"nt\">&#34;lon&#34;<\/span><span class=\"p\">:<\/span> <span class=\"mf\">52.5200<\/span><span class=\"p\">,<\/span> \n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"nt\">&#34;lat&#34;<\/span><span class=\"p\">:<\/span> <span class=\"mf\">13.4050<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"p\">}<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"p\">],<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"nt\">&#34;reviews&#34;<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"p\">{<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"nt\">&#34;user&#34;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&#34;alice&#34;<\/span><span class=\"p\">,<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"nt\">&#34;score&#34;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">4<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"p\">},<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"p\">{<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"nt\">&#34;user&#34;<\/span><span class=\"p\">:<\/span> <span class=\"s2\">&#34;bob&#34;<\/span><span class=\"p\">,<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"nt\">&#34;score&#34;<\/span><span class=\"p\">:<\/span> <span class=\"mi\">5<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"p\">}<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"> <span class=\"p\">]<\/span>\n<\/span><\/span><span class=\"line\"><span class=\"cl\"><span class=\"p\">}<\/span>\n<\/span><\/span><\/code><\/pre><\/div><h2 id=\"payload-types\">Payload types<\/h2>\n<p>In addition to storing payloads, Qdrant also allows you search based on certain kinds of values.\nThis feature is implemented as additional filters during the search and will enable you to incorporate custom logic on top of semantic similarity.<\/p>"},{"title":"Collections","link":"https:\/\/qdrant.tech\/documentation\/manage-data\/collections\/","pubDate":"Mon, 01 Jan 0001 00:00:00 +0000","author":"info@qdrant.tech (Andrey Vasnetsov)","guid":"https:\/\/qdrant.tech\/documentation\/manage-data\/collections\/","description":"<h1 id=\"collections\">Collections<\/h1>\n<p>A collection is a named set of points (vectors with a payload) among which you can search. The vector of each point within the same collection must have the same dimensionality and be compared by a single metric. <a href=\"#collection-with-multiple-vectors\">Named vectors<\/a> can be used to have multiple vectors in a single point, each of which can have their own dimensionality and metric requirements.<\/p>\n<p>Distance metrics are used to measure similarities among vectors.\nThe choice of metric depends on the way vectors obtaining and, in particular, on the method of neural network encoder training.<\/p>"},{"title":"Storage","link":"https:\/\/qdrant.tech\/documentation\/manage-data\/storage\/","pubDate":"Mon, 01 Jan 0001 00:00:00 +0000","author":"info@qdrant.tech (Andrey Vasnetsov)","guid":"https:\/\/qdrant.tech\/documentation\/manage-data\/storage\/","description":"<h1 id=\"storage\">Storage<\/h1>\n<p>All data within one collection is divided into segments.\nEach segment has its independent vector and payload storage as well as indexes.<\/p>\n<p>Data stored in segments usually do not overlap.\nHowever, storing the same point in different segments will not cause problems since the search contains a deduplication mechanism.<\/p>\n<p>The segments consist of vector and payload storages, vector and payload <a href=\"https:\/\/qdrant.tech\/documentation\/manage-data\/indexing\/\">indexes<\/a>, and id mapper, which stores the relationship between internal and external ids.<\/p>"},{"title":"Indexing","link":"https:\/\/qdrant.tech\/documentation\/manage-data\/indexing\/","pubDate":"Mon, 01 Jan 0001 00:00:00 +0000","author":"info@qdrant.tech (Andrey Vasnetsov)","guid":"https:\/\/qdrant.tech\/documentation\/manage-data\/indexing\/","description":"<h1 id=\"indexing\">Indexing<\/h1>\n<p>A key feature of Qdrant is the effective combination of vector and traditional indexes. It is essential to have this because for vector search to work effectively with filters, having a vector index only is not enough. In simpler terms, a vector index speeds up vector search, and payload indexes speed up filtering.<\/p>\n<p>The indexes in the segments exist independently, but the parameters of the indexes themselves are configured for the whole collection.<\/p>"},{"title":"Quantization","link":"https:\/\/qdrant.tech\/documentation\/manage-data\/quantization\/","pubDate":"Mon, 01 Jan 0001 00:00:00 +0000","author":"info@qdrant.tech (Andrey Vasnetsov)","guid":"https:\/\/qdrant.tech\/documentation\/manage-data\/quantization\/","description":"<h1 id=\"quantization\">Quantization<\/h1>\n<p>Quantization is an optional feature in Qdrant that enables efficient storage and search of high-dimensional vectors.\nBy transforming original vectors into a new representations, quantization compresses data while preserving close to original relative distances between vectors.\nDifferent quantization methods have different mechanics and tradeoffs. We will cover them in this section.<\/p>\n<p>Quantization is primarily used to reduce the memory footprint and accelerate the search process in high-dimensional vector spaces.\nIn the context of the Qdrant, quantization allows you to optimize the search engine for specific use cases, striking a balance between accuracy, storage efficiency, and search speed.<\/p>"},{"title":"Multitenancy","link":"https:\/\/qdrant.tech\/documentation\/manage-data\/multitenancy\/","pubDate":"Mon, 01 Jan 0001 00:00:00 +0000","author":"info@qdrant.tech (Andrey Vasnetsov)","guid":"https:\/\/qdrant.tech\/documentation\/manage-data\/multitenancy\/","description":"<h1 id=\"configure-multitenancy\">Configure Multitenancy<\/h1>\n<aside role=\"alert\">\nIt is not recommended to create hundreds and thousands of collections per cluster as it increases resource overhead unsustainably. Eventually this will lead to increased costs and at some point performance degradation and cluster instability. In Qdrant Cloud, we limit the amount of collections per cluster to 1000.\n<\/aside>\n<p><strong>How many collections should you create?<\/strong> In most cases, a single collection per embedding model with payload-based partitioning for different tenants and use cases. This approach is called multitenancy. It is efficient for most users, but requires additional configuration. This document will show you how to set it up.<\/p>"}]}}