A Case for Unstructured Distributed Hash Tables
Krishna Puttaswamy2007, 2007 IEEE Global Internet Symposium
Sign up for access to the world's latest research
checkGet notified about relevant papers
checkSave papers to use in your research
checkJoin the discussion with peers
checkTrack your impact
Abstract
Structured peer-to-peer overlays support compelling applications such as large-scale file systems and distributed backup using the distributed hash table (DHT) interface. While unstructured file-sharing systems continue to flourish, wide adoption of structured applications has been elusive. We explore an alternative path to deployment of these applications by asking the question, can structured applications be run on top of unstructured overlays? We build an unstructured distributed hash table (UDHT) on top of state of the art search and topology management mechanisms, and evaluate whether it can sufficiently emulate properties of DHTs to support structured applications.
Related papers
Storing and locating mutable data in structured peer-to-peer overlay networks
Advances in Informatics, 2005
Structured peer-to-peer overlay networks or Distributed Hash Tables (DHTs) are distributed systems optimized for storage and retrieval of read-only data. In this paper we elaborate on a method that allows them to manage mutable data. We argue that by altering the retrieval algorithm of DHTs, we can enable them to cope with data updates, without sacrificing their fundamental properties of scalability and fault-tolerance. We describe in detail and analyze an implementation of a Kademlia network capable of handling mutable data. Nevertheless, the corresponding protocol additions can easily be applied to any DHT design. Experimental results show that although the process of managing and propagating data updates throughout the network adds up to the total cost of the lookup operation, the extra network utilization can be exploited in favor of overlay resilience to random node joins and failures.
Enhanced Distributed Hash Tables for Complex Queries
2006 1st International Conference on Communication Systems Software & Middleware, 2006
Peer-to-peer file sharing systems have become a very popular way of sharing large number of files over a distributed environment. One of the principal ingredients of such systems is a lookup service which maps a key denoting a file to a location storing the file. Dynamic hash tables (DHT's) were recently proposed as a means of supporting such a lookup service in a completely distributed manner. They have many desirable properties, but suffer from one serious drawback -in order to locate a file, we must have a precise knowledge of the key representing it. In this paper, we propose a lookup service which supports complex queries and has all the advantages of DHT's. We also compare our proposed method with PIER [8], another recently proposed peer-to-peer system for answering complex queries. Our experiments show that our method results in better utilization of the network than PIER.
A Distributed Hash Table-based Approach for Providing a File System Neutral, Robust and Resilient Storage Infrastructure
2009 Next Generation Internet Networks, 2009
During the past years, anywhere, anytime access to large and reliable storage has become increasingly important in both enterprise and home computer environments. In this paper we present the design and implementation of a distributed storage infrastructure, which combines the benefits emanating by the usage of a distributed hash table as the core storage layer with the transparency provided by a low level operating system storage interface. The results of the experimental evaluation are very promising.
Performance Management of Peer-to-Peer Distributed Hash Tables
IFIP International Federation for Information Processing, 2006
P2P networking is a distributed model where entities play both the client and server role. One major problem addressed in this model is the discove~y, searching and routing in a dynamic distributed environment. Among the different envisaged solutions, Distributed Hash Tables (DHT) are very promising. They allow the build of robust content addressable networks. Despite good theoretical performance properties, infrastructures which implement the model need a performance management framework able to monitor them in case of a concrete deployment. In this article we propose a generic performance management information model for DHTs. Our contribution uses a standard management approach based on the Common Information Model (CIM) Metric model.
PDomus: A prototype for cluster-oriented distributed hash tables
Proceedings - 15th EUROMICRO International Conference on Parallel, Distributed and Network-Based Processing, PDP 2007, 2007
The Domus architecture for Distributed Hash Tables (DHTs) is specially designed to support the concurrent deployment of multiple and heterogeneous DHTs, in a dynamic shared-all cluster environment. The execution model is compatible with the simultaneous access of several distributed/parallel client applications to the same or different running DHTs. Support to distributed routing and storage is dynamically configurable per node, as a function of applications requirements, node base resources and the overall cluster communication, memory and storage usage. pDomus is a prototype of Domus that creates an environment where to evaluate the model embedded concepts and planned features. In this paper, we present a series of experiments conduced to obtain figures of merit i) for the performance of basic dictionary operations, and ii) for the storage overhead resulting from several storage technologies. We also formulate a ranking formula that takes into account access patterns of clients to DHTs, to objectively select the most adequate storage technology, as a valuable metric for a wide range of application scenarios. Finally, we also evaluate client applications and services scalability, for a select dictionary operation. Results of the overall evaluation are promising and a motivation for further work.
Scalable Peer-to-Peer File Sharing with Efficient Complex Query Support
2007
A good P2P file sharing system is usually expected to achieve the following design goals: scalability, routing efficiency and complex query support. In this paper, we propose such a system, called PSON, which can satisfy all the three requirements. PSON is essentially a semantic overlay network of logical nodes, in which queries are routed on the basis of semantics. A logical node is formed by a cluster of peers that are close to each other in the physical network. Each cluster selects a powerful peer as super peer to support routing in the overlay network. To facilitate routing, all the super peers (or logical nodes) are organized in the form of a balanced tree. By exploiting the concepts of hierarchy and semantics, PSON can support complex queries in a scalable and efficient way. In this paper, we will describe the system architecture, and examine the key component of PSON design, i.e., semantic overlay construction and routing. We also conduct simulations, and show that the query routing in PSON is very efficient (O(log(n)) in the case of exact query) .
PERFORMANCE STUDY OF DISTRIBUTED HASH TABLE MECHANISMS ON P2P OVERLAY NETWORK UNDER EXTREME CONDITIONS
2007
The recent emergence of peertopeer (P2P) applications (i.e. Napster, Gnutella, etc) has boosted a large number of researches on peertopeer overlay networks due to their scalability, flexibility and efficiency in using resources, which are ideal for large applications such as data sharing, content distribution, multimedia streaming. An overlay network is a computer network which is built on top of another network. Nodes in the overlay can be thought of as being connected by virtual or logical links, each of which corresponds to a path, perhaps through many physical links, in the underlying network. Together with the fast grow of computing capability of mobile devices; P2P network no longer contains only desktop/laptop computers but also mobile and handheld devices connected through wireless connections such as GPRS, WiFi and WiMax. This imposes new challenges on P2P mechanisms, such as intermittent data connections and high frequency of node join/leave. DHT) is known as the most popular mechanism for the third generation of P2P network, featuring efficient data storage and retrieval. In an effort to implement P2P mechanisms on Telecommunication mobile network, the authors have investigated the performance of several new DHT algorithms such as Kelips, Tapestry and Chord under extreme conditions of mobile computing environments. Using simulation method, performance of Kelips, Tapestry and Chord under high churn rate was deeply analyzed and compared. We also identify their most important parameters to be tuned for optimized performance.
A Semantic Layer for a Peer-to-Peer Based on a Distributed Hash Table
Informatics Engineering and …, 2011
A tree-based DHT Approach to scalable weakly consistent peer-to-peer data management
Proceedings. 15th International Workshop on Database and Expert Systems Applications, 2004., 2004
Second generation Peer-to-Peer systems employ a Distributed Hashtable (DHT) to locate data efficiently in their network. But when peers fail, the DHT algorithm needs to route around them, which, in many cases, reduces its efficiency. We propose to separate the data lookup from the data storing concerns. Thus we can guarantee a search cost proportional to the logarithm of the number of peers in the network despite a high change rate, i.e., many peers join or leave the network concurrently. To demonstrate our approach, we use a simple DHT based on a binary tree whose leaves are associated with collections of peers that are responsible for a range of data values associated with search keys. Peers of a group collaborate to form redundant data storage and thus ensure high availability of their data. This means, however, only weak data consistency can be guaranteed when an insertion or update takes place.
Related papers
LDHT: locality-aware distributed hash tables
2008
As the substrate of structured peer-to-peer systems, Distributed Hash DHT) plays a key role in P2P routing infrastructures. Traditional DHT does not consider the location of the nodes for the assignment of identifiers, which will result in high end-to-end latency on DHT-based overlay networks. In this paper, we propose a design of locality-aware DHT called LDHT, which exploits network locality on DHT-based systems. Instead of assigning uniform random node identifiers in traditional DHT, nodes in LDHT are assigned locality-aware identifiers according to their Autonomous System Numbers (ASNs). As a result, each node will have more nearby neighbors than faraway neighbors in the overlay. We evaluate the performance of LDHT on different kinds of typical DHT protocols and on various topologies. The results show that LDHT improves the traditional DHT protocols a lot in terms of end-to-end latency, without introducing additional overhead. It is indicated that LDHT is fit for different kinds of DHT protocols and can work effectively on all structured P2P systems including Chord, Symphony and Kademlia.
Survey on Distributed Hash Tables
2006
Abstract: In this paper we review distributed hash tables (DHTs) that work as overlay networks on top of the IP network. We compare these DHTs under a number of key aspects that concern performance, scalability and self-configurability. Additionally, since these networks are not suitable for wireless ad hoc networks, we shortly review techniques that can be used to take distributed hash tables to wireless environments.
Enabling dynamic querying over distributed hash tables
2010
Dynamic querying (DQ) is a search technique used in unstructured peer-to-peer (P2P) networks to minimize the number of nodes that is necessary to visit to reach the desired number of results. In this paper, we introduce the use of the DQ technique in structured P2P networks. In particular, we present a P2P search algorithm, named DQ-DHT (Dynamic Querying over a Distributed Hash Table), to perform DQ-like searches over DHT-based overlays.
Exploring distributed hash tables in HighEnd computing
Sigmetrics Performance Evaluation Review, 2011
Over the last decade, storage systems have experienced a 10-fold increase between their capacity and bandwidth. This gap is predicted to grow faster with exponentially growing concurrency levels, with future exascales delivering millions of nodes and billions of threads of execution. A critical component of future file systems for high-end computing is metadata management. This extended abstract presents ZHT, a zero-hop distributed hashtable, which has been tuned for the specific requirements of highend computing. The primary goal of ZHT is excellent availability, fault tolerance, high throughput, and low latencies.
Exploiting Network Proximity in Distributed Hash Tables
2002
Self-organizing peer-to-peer (p2p) overlay networks like CAN, Chord, Pastry and Tapestry (also called distributed hash tables or DHTs) offer a novel platform for a variety of scalable and decentralized distributed applications. These systems provide efficient and fault-tolerant routing, object location, and load balancing within a self-organizing overlay network. One important aspect of these systems is how they exploit network proximity in the underlying Internet.
HASH-BASED OVERLAY PARTITIONING IN UNSTRUCTURED PEER-TO-PEER SYSTEMS
2009
Unstructured peer-to-peer (P2P) networks suffer from the increased volume of traffic produced by flooding. Methods such as random walks or dynamic querying managed to limit the traffic at the cost of reduced network coverage. In this paper, we propose a partitioning method of the unstructured overlay network into a relative small number of distinct subnetworks. The partitioning is driven by the categorization of keywords based on a uniform hash function.
DISTRIBUTED HASH TABLE STORAGE MODEL FOR DISTRIBUTED NETWORK SYSTEM
In this research paper, we bring scalable, available, and consistent data management capabilities to distributed platforms using distributed hash table (DHT) capabilities in analysing and designing a reusable, distributed network based storage system for implementation in distributed environment, called Unified Distributed Storage System (UniDSS), specifically designed for the needs of Internet services. A UniDSS presents an in-memory data structure interface to applications implementing a modified chord DHT algorithm, and durably manages the data behind this interface by distributing and replicating it across distributed nodes. This paper describes the design, architecture, and implementation requirements of distributed data structure (a distributed hash table built in Java Hadoop platforms). We evaluate its performance, scalability and availability, and its ability to simplify service construction.
Domus - An architecture for cluster-oriented Distributed Hash Tables
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2006
This paper presents a high level description of Domus, an architecture for cluster-oriented Distributed Hash Tables. As a data management layer, Domus supports the concurrent execution of multiple and heterogeneous DHTs, that may be simultaneously accessed by different distributed/parallel client applications. At system level, a load balancement mechanism allows for the (re)distribution of each DHT over cluster nodes, based on the monitoring of their resources, including CPUs, memory, storage and network. Two basic units of balancement are supported: vnodes, a coarse-grain unit, and partitions, a fine-grain unit. The design also takes advantage of the strict separation of object lookup and storage, at each cluster node, and for each DHT. Lookup follows a distributed strategy that benefits from the joint analysis of multiple partition-specific routing information, to shorten routing paths. Storage is accomplished through different kinds of data repositories, according to the specificity and requirements of each DHT.
Brushwood: Distributed Trees in Peer-to-Peer Systems
2005
There is an increasing demand for locality-preserving distribution of complex data structures in peer-to-peer systems. Current systems either do not preserve object locality or suffer from imbalances in data distribution, routing state, and/or query processing costs. In this position paper, we take a systematic approach that enables the deployment of searchable tree structures in p2p environments. We achieve distributed tree traversal with efficient routing distance and routing state. We show how to implement several p2p applications using distributed tree structures.
A random structure for optimum cache size distributed hash table (DHT) peer-to-peer design
2002
We propose a new and easily-realizable distributed hash table (DHT) peer-to-peer structure, incorporating a random caching strategy that allows for polylogarithmic search time while having only a constant cache size. We also show that a very large class of deterministic caching strategies, which covers almost all previously proposed DHT systems, can not achieve polylog search time with constant cache size. In general, the new scheme is the first known DHT structure with the following highly-desired properties: (a) Random caching strategy with constant cache size; (b) Average search time of O(log 2 (N )); (c) Guaranteed search time of O(log 3 (N )); (d) Truly local cache dynamics with constant overhead for node deletions and additions; (e) Self-organization from any initial network state towards the desired structure; and (f) Allows a seamless means for various trade-offs, e.g., search speed or anonymity at the expense of larger cache size.
Loading Preview
Sorry, preview is currently unavailable. You can download the paper by clicking the button above.