A Near-Optimal Parallel Algorithm for Joining Binary Relations

Lecture Notes in Computer Science, 2018

Nowadays parallel DBMSs compete with graph Hadoop Big Data systems to analyze large graphs. In this paper, we study the processing and optimization of relational queries to solve fundamental graph problems, giving a theoretical foundation on time complexity and parallel processing. Specifically, we consider reachability, shortest paths from a single vertex, weakly connected components, PageRank, transitive closure and all pairs shortest paths. We explain how graphs can be stored on a relational database and then we show relational queries can be used to efficiently analyze such graphs. We identify two complementary families of algorithms: iteration of matrix-vector multiplication and iteration of matrix-matrix multiplication. We show all problems can be solved with a unified algorithm with an iteration of matrix multiplications. We present intuitive theory results on cardinality estimation and time complexity considering graph size, shape and density. Finally, we characterize parallel computational complexity and speedup per iteration, focusing on joins and aggregations.

2021

We study theta-joins in general and join predicates with conjunctions and disjunctions of inequalities in particular, focusing on ranked enumeration where the answers are returned incrementally in an order dictated by a given ranking function. Our approach achieves strong time and space complexity properties: with n denoting the number of tuples in the database, we guarantee for acyclic full join queries with inequality conditions that for every value of k, the k top-ranked answers are returned in O(n polylog n + k log k) time. This is within a polylogarithmic factor of the best known complexity for equi-joins and even of 𝒪(n+k), the time it takes to look at the input and return k answers in any order. Our guarantees extend to join queries with selections and many types of projections, such as the so-called free-connex queries. Remarkably, they hold even when the entire output is of size n^ℓ for a join of ℓ relations. The key ingredient is a novel 𝒪(n polylog n)-size factorized repr...

Fundamenta Informaticae, 2016

We consider a new graph operation c 2-join which generalizes join and co-join. We show that odd hole-free graphs (odd antihole-free graphs) are closed under c 2-join and describe a polynomial time algorithm to recognize graphs that admit a c 2-join. The time complexity of the (a) recognition problem, (b) maximum weight independent set (MWIS) problem, and (c) minimum coloring (MC) problem for odd hole-free graphs are still unknown. Let H be an odd hole-free graph that contains an odd antihole as an induced subgraph and G H be the class of all graphs generated from the induced subgraphs of H by using c 2-join recursively. Then G H is odd hole-free, contains all P 4-free graphs, complement of all bipartite graphs, and some imperfect graphs. We show that the MWIS problem, maximum weight clique (MWC) problem, MC problem, and minimum clique cover (MCC) problem can be solved efficiently for G H .

SIAM Journal on Computing, 1984

In this paper, we present efficient parallel algorithms for the following graph problems: finding the lowest common ancestors for vertex pairs of a directed tree; finding all fundamental cycles, a directed spanning forest, all bridges, all bridge-connected components, all separation vertices, all biconnected components, and testing the biconnectivity of an undirected graph. All these algorithms achieve the O(lg n) time bound, with the first two algorithms using n[n/lg n] processors and the remaining algorithms using n[n/lg n] processors. In all cases, our algorithms are better than the previously known algorithms and in most cases reduce the number of processors used by a factor of n lg n. Moreover, our algorithms are optimal with respect to the time-processor product for dense graphs, with the exception of the first two algorithms. The machine model we use is the PRAM which is a SIMD model allowing simultaneous reads but not simultaneous writes to the same memory location.

Proceedings of the VLDB Endowment, 2019

We study the subgraph enumeration problem under distributed settings. Existing solutions either suffer from severe memory crisis or rely on large indexes, which makes them impractical for very large graphs. Most of them follow a synchronous model where the performance is often bottlenecked by the machine with the worst performance. Motivated by this, in this paper, we propose RADS, a Robust Asynchronous Distributed Subgraph enumeration system. RADS first identifies results that can be found using single-machine algorithms. This strategy not only improves the overall performance but also reduces network communication and memory cost. Moreover, RADS employs a novel region-grouped multi-round expand verify & filter framework which does not need to shuffle and exchange the intermediate results, nor does it need to replicate a large part of the data graph in each machine. This feature not only reduces network communication cost and memory usage, but also allows us to adopt simple strateg...

2016

We study the problem of distributing the tuples of a relation to a number of processors organized in an r-dimensional hypercube, which is an important task for parallel join processing. In contrast to previous work, which proposed randomized algorithms for the task, we ask here the question of how to construct efficient deterministic distribution strategies that can optimally load balance the input relation. We first present some general lower bounds on the load for any dimension; these bounds depend not only on the size of the relation, but also on the maximum frequency of each value in the relation. We then construct an algorithm for the case of 1 dimension that is optimal within a constant factor, and an algorithm for the case of 2 dimensions that is optimal within a polylogarithmic factor. Our 2-dimensional algorithm is based on an interesting connection with the vector load balancing problem, a well-studied problem that generalizes classic load balancing.

The 9th International Conference on Smart Media and Applications, 2020

Using Regular Path Queries (RPQs) is a common way to explore patterns in graph databases. Traditional automata-based approaches for evaluating RPQs on large graphs are restricted in the graph size and/or highly complex queries, which causes a high evaluation cost. Recently, the threshold rare label based approach applied on large graphs has been proved to be effective. Nevertheless, using rare labels in a graph provides only coarse information which could not always guarantee the minimum searching cost. Hence, the Unit-Subquery Cost Matrix (USCM) based approach has been proposed to reduce the parallel evaluation cost by estimating the searching cost of RPQs. However, the previous approach does not take the joining cost among subqueries into account. In this paper, the method of estimating joining cost of subqueries is proposed in order to accelerate the USCM based parallel evaluation of RPQs. Specifically, the proposed method is realized by estimating the result size of the subqueries. Through our experiments upon real-world datasets, it is depicted that estimating joining cost enhances USCM based approach up to around 20% in terms of response time. CCS CONCEPTS • Computing methodologies → Search methodologies; Parallel computing methodologies.

Mathematics

Using three supercomputers, we broke a record set in 2011, in the enumeration of non-isomorphic regular graphs by expanding the sequence of A006820 in the Online Encyclopedia of Integer Sequences (OEIS), to achieve the number for 4-regular graphs of order 23 as 429,668,180,677,439, while discovering several regular graphs with minimum average shortest path lengths (ASPL) that can be used as interconnection networks for parallel computers. The enumeration of 4-regular graphs and the discovery of minimal-ASPL graphs are extremely time consuming. We accomplish them by adapting GENREG, a classical regular graph generator, to three supercomputers with thousands of processor cores.

2013

Plenty of structural patterns in real world have been represented as graph like molecules, chemical compounds, social network, road network etc. Mining this graph for extracting some useful information is of special interest and has many applications. The application includes drug discovery, compound synthesis, anomaly detection in network, social network analysis for finding groups etc. One of the most interesting problems in graph mining is graph containment problem. In graph containment problem ,given a query graph q ,it is asked to find all graph in given graph dataset containing this query (query graph as subgraph).This means finding all graph which is isomorphic to query graph. As in real world there is vast number of graph in graph dataset so this task of subgraph isomorphism test become tedious, complex, time and space consuming. So it is necessary to create an index of graphs present in dataset for cost efficient query processing. In this paper we proposed a time efficient ...

2010

We consider the problem of finding an unknown graph by using queries with an additive property. This problem was partially motivated by DNA shotgun sequencing and linkage discovery problems of artificial intelligence. Given a graph, an additive query asks the number of edges in a set of vertices while a cross-additive query asks the number of edges crossing between two disjoint sets of vertices. The queries ask the sum of weights for weighted graphs. For a graph G with n vertices and at most m edges, we prove that there exists an algorithm to find the edges of G using O(m log n 2 m log(m+1)) queries of both types for all m. The bound is best possible up to a constant factor. For a weighted graph with a mild condition on weights, it is shown that O(m log n log m) queries are enough provided m (log n) α for a sufficiently large constant α, which is best possible up to a constant factor if m n 2−ε for any constant ε > 0.

Discrete Applied Mathematics, 2005

In this paper, we establish structural properties for the class of complement reducible graphs or cographs, which enable us to describe efficient parallel algorithms for recognizing cographs and for constructing the cotree of a graph if it is a cograph; if the input graph is not a cograph, both algorithms return an induced P 4 . For a graph on n vertices and m edges, both our cograph recognition and cotree construction algorithms run in O(log 2 n) time and require O((n+m)/ log n) processors on the EREW PRAM model of computation. Our algorithms are motivated by the work of Dahlhaus (Discrete Appl. Math. 57 (1995) 29-44) and take advantage of the optimal O(log n)-time computation of the coconnected components of a general graph (Theory Comput. Systems 37 (2004) 527-546) and of an optimal O(log n)-time parallel algorithm for computing the connected components of a cograph, which we present. Our results improve upon the previously known linear-processor parallel algorithms for the problems (Discrete Appl. Math. 57 (1995) 29-44; J. Algorithms 15 (1993) 284-313): we achieve a better time-processor product using a weaker model of computation and we provide a certificate (an induced P 4 ) whenever our algorithms decide that the input graphs are not cographs.

We consider the problem of computing a relational query q on a large input database of size n, using a large number p of servers. The computation is performed in rounds, and each server can receive only O(n/p 1−ε) bits of data, where ε ∈ [0, 1] is a parameter that controls replication. We examine how many global communication steps are needed to compute q. We establish both lower and upper bounds, in two settings. For a single round of communication, we give lower bounds in the strongest possible model, where arbitrary bits may be exchanged; we show that any algorithm requires ε ≥ 1−1/τ * , where τ * is the fractional vertex cover of the hypergraph of q. We also give an algorithm that matches the lower bound for a specific class of databases. For multiple rounds of communication, we present lower bounds in a model where routing decisions for a tuple are tuple-based. We show that for the class of tree-like queries there exists a tradeoff between the number of rounds and the space exponent ε. The lower bounds for multiple rounds are the first of their kind. Our results also imply that transitive closure cannot be computed in O(1) rounds of communication.

Proceedings of the twenty-fifth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, 2006

Given that most elementary problems in database design are NP-hard, the currently used database design algorithms produce suboptimal results. For example, the current 3NF decomposition algorithms may continue further decomposing a relation even though it is already in 3NF. In this paper we study database design problems whose sets of functional dependencies have bounded treewidth. For such sets, which frequently occur in practice, we develop polynomialtime and highly parallelizable algorithms for a number of central database design problems such as: • primality of an attribute • 3NF-test for a relational schema or subschema • BCNF-test for a subschema. For establishing these results, we propose a new characterization for keys and for the primality of a single attribute. In order to define the treewidth of a relational schema, we shall associate a hypergraph with it. Note that there are two main possibilities of defining the treewidth of a hypergraph H: One is via the primal graph of H and one is via the incidence graph of H. Our algorithms apply to the case where the primal graph is considered. However, we also show that the tractability results still hold when the incidence graph is considered instead.

Proceedings of the 2019 ACM Symposium on Principles of Distributed Computing, 2019

The Massively Parallel Computation (MPC) model serves as a common abstraction of many modern large-scale parallel computation frameworks and has recently gained a lot of importance, especially in the context of classic graph problems. In this work, we mainly consider maximal matching and maximal independent set problems in the MPC model. These problems are known to admit efficient MPC algorithms if the space available per machine is near-linear in the number n of nodes. This is not only often significantly more than what we can afford, but also allows for easy if not trivial solutions for sparse graphs-which are common in real-world large-scale graphs. We are, therefore, interested in the low-memory MPC model, where the space per machine is restricted to be strongly sublinear, that is, n δ for any constant 0 < δ < 1.

International Journal of Foundations of Computer Science, 1993

Let k be a positive integer, a subset Q of the set of vertices of a graph G is k-dependent in G if each vertex of Q has no more than k neighbours in Q. We present a parallel algorithm which computes a maximal k-dependent set in a graph on n nodes in time O( log 4 n) on an EREW PRAM with O(n2) processors. In this way, we establish the membership of the problem of constructing a maximal k-dependent set in the class NC. Our algorithm can be easily adapted to compute a maximal k-dependent set in a graph of bounded valence in time O( log * n) using only O(n) EREW PRAM processors. Let f be a positive integer function defined on the set V of vertices of a graph G. A subset F of the set of edges of G is said to be an f-matching if every vertex vɛV is adjacent to at most f(v) edges in-F. We present the first NC algorithm for constructing a maximal f-matching. For a graph on n nodes and m edges the algorithm runs in time O( log 4 n) and uses O(n+m) EREW PRAM processors. For graphs of constant...