Efficiently Monitoring Top-k Pairs over Sliding Windows

ACM Transactions on Database Systems, 2015

A sliding window top-k ( top-k/w ) query monitors incoming data stream objects within a sliding window of size w to identify the k highest-ranked objects with respect to a given scoring function over time. Processing of such queries is challenging because, even when an object is not a top-k/w object at the time when it enters the processing system, it might become one in the future. Thus a set of potential top-k/w objects has to be stored in memory while its size should be minimized to efficiently cope with high data streaming rates. Existing approaches typically store top-k/w and candidate sliding window objects in a k-skyband over a two-dimensional score-time space. However, due to continuous changes of the k-skyband, its maintenance is quite costly. Probabilistic k-skyband is a novel data structure storing data stream objects from a sliding window with significant probability to become top-k/w objects in future. Continuous probabilistic k-skyband maintenance offers considerably i...

IEEE Transactions on Knowledge and Data Engineering, 2000

Top-k dominating queries use an intuitive scoring function which ranks multidimensional points with respect to their dominance power, i.e., the number of points that a point dominates. The k points with the best (e.g., highest) scores are returned to the user. Both top-k and skyline queries have been studied in a streaming environment, where changes to the data set are very frequent. In such an environment, continuous query processing techniques are required towards efficient monitoring of query results, since periodic query re-execution is computationally intensive, and therefore, prohibitive. This work contains the first study of continuous top-k dominating queries over data streams. In comparison to continuous top-k and skyline queries, continuous top-k dominating queries pose additional challenges. Three exact algorithms (BFA, EVA, ADA) are studied, and among them ADA, which is enhanced with additional optimization techniques, shows the best overall performance. In some cases, we are willing to trade accuracy for speed. Towards this direction, two approximate algorithms are proposed (AHBA and AMSA). AHBA offers probabilistic guarantees regarding the accuracy of the result based on the Hoeffding bound, whereas AMSA performs a more aggressive computation resulting in more efficient processing. Evaluation results, based on real-life and synthetic data sets, show the efficiency and scalability of our techniques.

IOSR Journal of Computer Engineering, 2013

Efficient execution of top-k queries is increasingly becoming a major challenge for relational database technology. Top-k query has emerged as a key requirement in modern application. In these applications, efficient and adaptive evaluation of top-k queries is an integral part of application semantics. In this paper we discuss a query optimization framework of top-k queries that fully integrates rank-join operators. Ranking queries produce results that are ordered on some computed score. A key property of top-k queries is that, users are interested only in the first k results and not in total ranking of all query results. This property directly impacts the optimization of top-k queries by optimizing for the first k results. Traditionally, most real world database systems offer the feature of first n row optimization.

2015

Top-k dominating queries combine the advantages of top-k queries and skyline queries, and eliminate their disadvantages. They return k objects with the highest domination score, which is defined as the number of dominated objects. As a top-k query, the user can bound the number of returned results through the parameter k, and like a skyline query a user-selected scoring function is not required. Top-k dominating queries have been studied by different perspectives, such as in indexed and nonindexed multi-dimensional data using efficient exact computation algorithms, in uncertain data using randomized algorithms with accuracy guarantees, and in data streams. In addition, topk dominating queries have been studied over distance-based dynamic attribute vectors, defined over a metric space, using efficient progressive algorithms. Top-k dominating queries have become an important tool for decision support, data mining, web search, and multi-criteria retrieval applications.

2011

Top-k pairs queries have many real applications. k closest pairs queries, k furthest pairs queries and their bichromatic variants are some of the examples of the top-k pairs queries that rank the pairs on distance functions. While these queries have received significant research attention, there does not exist a unified approach that can efficiently answer all these queries. Moreover, there is no existing work that supports top-k pairs queries based on generic scoring functions. In this paper, we present a unified approach that supports a broad class of top-k pairs queries including the queries mentioned above. Our proposed approach allows the users to define a local scoring function for each attribute involved in the query and a global scoring function that computes the final score of each pair by combining its scores on different attributes. We propose efficient internal and external memory algorithms and our theoretical analysis shows that the expected performance of the algorithms is optimal when two or less attributes are involved. Our approach does not require any pre-built indexes, is easy to implement and has low memory requirement. We conduct extensive experiments to demonstrate the efficiency of our proposed approach.

Top-k queries are useful in retrieving top-k records from a given set of records depending on the value of a function F on their attributes. Many techniques have been proposed in database literature for answering top-k queries.These are mainly categorized into three:Sorted-list based,layer based and View based. In first category, records are sorted along each dimension and then assigned a rank to each of the records using parallel scanning method.Threshold Algorithm(TA) and Fagin's Algorithm(FA) are the examples of sorted-list based category. Second category is layer based category,in which all the records are organized into layers such as in onion technique and robust indexing technique.Third category includes methods such as PREFER and LPTA(Linear Programming Adaptation of Threshold Algorithm) and processing is based on the materialized views.

2011

The way that we interact with databases has changed. The ubiquitous textbook example of finding employees with salaries over 60K and ages under 30 no longer reflects a typical use case in modern applications; instead, users are interested in books, movies, music, documents, services, and users that are similar to another object that they have in mind. This sort of similarity search is often implemented by means of a top-k query, wherein each tuple is assigned a score and the tuples with the k highest scores are reported back to the user.

The way that we interact with databases has changed. The ubiquitous textbook example of finding employees with salaries over 60K and ages under 30 no longer reflects a typical use case in modern applications; instead, users are interested in books, movies, music, documents, services, and users that are similar to another object that they have in mind. This sort of similarity search is often implemented by means of a top-k query, wherein each tuple is assigned a score and the tuples with the k highest scores are reported back to the user.

Proceedings of the VLDB Endowment, 2010

We consider the problem of efficiently finding the top-k answers for join queries over web-accessible databases. Classical algorithms for finding top-k answers use branch-and-bound techniques to avoid computing scores of all candidates in identifying the top-k answers. To be able to apply such techniques, it is critical to efficiently compute (lower and upper) bounds and expected scores of candidate answers in an incremental fashion during the evaluation. In this paper, we describe novel techniques for these problems. The first contribution of this paper is a method to efficiently compute bounds for the score of a query result when tuples in tables from the "FROM" clause are discovered incrementally, through either sorted or random access. Our second contribution is an algorithm that, given a set of partially evaluated candidate answers, determines a good order in which to access the tables to minimize wasted efforts in the computation of top-k answers. We evaluate our algorithms on a variety of queries and data sets and demonstrate the significant benefits they provide.

IEEE Transactions on Knowledge and Data Engineering, 2014

Given a scoring function that computes the score of a pair of objects, a top-k pairs query returns k pairs with the smallest scores. In this paper, we present a unified framework for answering generic top-k pairs queries including k-closest pairs queries, kfurthest pairs queries and their variants. Note that k-closest pairs query is a special case of top-k pairs queries where the scoring function is the distance between the two objects in a pair. We are the first to present a unified framework to efficiently answer a broad class of top-k queries including the queries mentioned above. We present efficient algorithms and provide a detailed theoretical analysis that demonstrates that the expected performance of our proposed algorithms is optimal for two dimensional data sets. Furthermore, our framework does not require pre-built indexes, uses limited main memory and is easy to implement. We also extend our techniques to support top-k pairs queries on multi-valued (or uncertain) objects. We also demonstrate that our framework can handle exclusive top-k pairs queries. Our extensive experimental study demonstrates effectiveness and efficiency of our proposed techniques. Index Terms-Closest pairs queries, furthest pairs queries, top-k queries, multi-valued objects, uncertain objects !