A Novel Load-Balancing Algorithm for Distributed Systems

Cloud computing is an upcoming era in software industry. It's a very vast and developing technology. Distributed file systems play an important role in cloud computing applications based on map reduce techniques. While making use of distributed file systems for cloud computing, nodes serves computing and storage functions at the same time. Given file is divided into small parts to use map reduce algorithms in parallel. But the problem lies here since in cloud computing nodes may be added, deleted or modified any time and also operations on files may be done dynamically. This causes the unequal load distribution of load among the nodes which leads to load imbalance problem in distributed file system. Newly developed distributed file system mostly depends upon central node for load distribution but this method is not helpful in large-scale and where chances of failure are more. Use of central node for load distribution creates a problem of single point dependency and chances of performance of bottleneck are more. As well as issues like movement cost and network traffic caused due to migration of nodes and file chunks need to be resolved. So we are proposing algorithm which will overcome all these problems and helps to achieve uniform load distribution efficiently.

Cloud computing is an upcoming era in software industry. It's a very vast and developing technology. Distributed file systems play an important role in cloud computing applications based on map reduce techniques. While making use of distributed file systems for cloud computing, nodes serves computing and storage functions at the same time. Given file is divided into small parts to use map reduce algorithms in parallel. But the problem lies here since in cloud computing nodes may be added, deleted or modified any time and also operations on files may be done dynamically. This causes the unequal load distribution of load among the nodes which leads to load imbalance problem in distributed file system. Newly developed distributed file system mostly depends upon central node for load distribution but this method is not helpful in large-scale and where chances of failure are more. Use of central node for load distribution creates a problem of single point dependency and chances of performance of bottleneck are more. As well as issues like movement cost and network traffic caused due to migration of nodes and file chunks need to be resolved. So we are proposing algorithm which will overcome all these problems and helps to achieve uniform load distribution efficiently. To verify the feasibility and efficiency of our algorithm we will be using simulation setup and compare our algorithm with existing techniques for the factors like load imbalance factor, movement cost and network traffic.

2016

Distributed file systems are key building blocks for cloud computing applications based on the MapReduce programming paradigm. In such file systems, the nodes are simultaneously serve this computing and storage functions; a file is partitioned into a number of chunks allocated in distinct nodes so that this MapReduce tasks can be performed in parallel over the nodes. However, in a cloud computing environment, failure is the norm, and nodes may be upgraded, replaced, and added in the system. Files can also be dynamically created, deleted, and appended. This results in load imbalance in a distributed file system; that is, the file chunks are not distributed as uniformly as possible among the nodes. Emerging distributed file systems in production systems strongly depend on a central node for chunk reallocation. In this paper, a fully distributed load rebalancing algorithm is presented to solve with the load imbalance problem. This algorithm is compared against a centralized approach in...

Distributed file systems are key building blocks for cloud computing applications based on the MapReduce programming paradigm. In such file systems, nodes simultaneously serve computing and storage functions; a file is partitioned into a number of chunks allocated in distinct nodes so that MapReduce tasks can be performed in parallel over the nodes. However, in a cloud computing environment, failure is the norm, and nodes may be upgraded, replaced, and added in the system. Files can also be dynamically created, deleted, and appended. This results in load imbalance in a distributed file system; that is, the file chunks are not distributed as uniformly as possible among the nodes. Emerging distributed file systems in production systems strongly depend on a central node for chunk reallocation. This dependence is clearly inadequate in a large-scale, failure-prone environment because the central load balancer is put under considerable workload that is linearly scaled with the system size, and may thus become the performance bottleneck and the single point of failure. In this paper, a fully distributed load rebalancing algorithm is presented to cope with the load imbalance problem. Our algorithm is compared against a centralized approach in a production system and a competing distributed solution presented in the literature. The simulation results indicate that our proposal is comparable with the existing centralized approach and considerably outperforms the prior distributed algorithm in terms of load imbalance factor, movement cost, and algorithmic overhead. The performance of our proposal implemented in the Hadoop distributed file system is further investigated in a cluster environment.

IOSR Journal of Computer Engineering, 2014

Cloud Computing is an emerging technology, it is based on demand service in which shared resources, information, software and other devices are provided according to the clients to the requirements at specific time with the availability of internet. Load balancing is one of the challenging issue in cloud computing. An efficient load balancing makes cloud computing more efficient and improves user satisfaction. It includes fault tolerance, high availability, scalability, flexibility, reduced overhead for users, reduced cost of ownership, on demand services etc. Distributed file systems are key building blocks for cloud computing applications based on the Map Reduce programming paradigm. In such file systems, nodes at the same time serve computing and storage functions. Files can be created, deleted, and appended dynamically. This results in load imbalance in a distributed file system; that is, the file chunks are not distributed uniformly as possible among the nodes.

For cloud computing applications the Distributed file system is used as a key building block which is simply a classical model. In such file system a file is partitioned into a number of chunks allocated in distinct nodes .Each chunk allocates to separate node to perform MapReduce function parallel over each node. In cloud, the central node (master in MapReduce) becomes bottleneck if number of storage nodes, number of files and assesses to that file increases. In this survey paper to overcome the above load imbalance problem the fully distributed load rebalancing algorithm is used to exclude the load on central node and also the movement cost is reduced. In this paper the load misbalancing problem is overcome.

For cloud computing applications the Distributed file system is used as a key building block which is simply a classical model. In such file system a file is partitioned into a number of chunks allocated in distinct nodes .Each chunk allocates to separate node to perform MapReduce function parallel over each node. In cloud, the central node (master in MapReduce) becomes bottleneck if number of storage nodes, number of files and assesses to that file increases. In this survey paper to overcome the above load imbalance problem the fully distributed load rebalancing algorithm is used to exclude the load on central node and also the movement cost is reduced. In this paper the load misbalancing problem is overcome.

Contemporary Engineering Sciences, 2015

File storage load can be balanced in the storage nodes avail in the cloud system by using totally distributed load rebalancing algorithm. Large level distributed systems such as cloud applications come with rising challenges on how to transfer and where to store compute data. Cloud computing is a distributed computing over a network. Node concurrently serves as a computing and storage task. In cloud computing environment, files can also be dynamically created, deleted and append. The file chunks are not distributed as equally among the nodes. Lead to load inequity in a distributed file system. The existing distributed file system depends on single node to manage almost all operations such as chunk reallocation of every data block in the file system. As a result it can be bottleneck resource and a single point of failure. A new technique Random Linear Network Coding (RLNC) is employed in the proposed system. RLNC is performed at the opening when the file is stored in the cloud. Using this strategy, a file will be split into different parts and send distinct parts to each chunk server. RSA algorithm applied to calculate the response time file size and deadlock detection. RSA algorithm used to detect the anomalies of the system. Dynamic scheduling algorithm present in the proposed system to overcome to load inequity problem. This algorithm is compared against a centralized approach in a production system. The results indicate that our proposal is considerably outperforms the prior distributed in terms of load inequity factor, movement cost, and algorithmic overhead.

International Journal of Engineering Sciences & Research Technology, 2013

Distributed file systems are key building blocks for cloud computing applications based on the Map Reduce programming paradigm. In such file systems, nodes all at once serve computing and storage functions; a file is partitioned into a number of chunks allocated in distinct nodes so that Map Reduce tasks can be performed in parallel on the nodes. However, in a cloud computing environment, failure is the norm, and nodes may be improved, replaced, and added in the system. This dependence is clearly incompetent in a large-scale, failure-prone environment on account of the central load balancer is put under considerable workload that is linearly scaled with the system size, and may thus become the attainment bottleneck and the single point of failure. Here, a fully distributed load rebalancing algorithm is presented to cope with the load imbalance problem.

2014

Map Reduce programming paradigm plays a vital role in the development of cloud computing application using the Distributed file system where nodes concurrently provide computing as well as storage functions. Initially a file is partitioned into number of chunks allocated into different nodes so that Map Reduce technique can be performed in the nodes. Since cloud computing is a dynamic environment upgrading, replacing and adding new nodes to the environment is a frequent concern. This confidence is obviously insufficient in a large-scale, failure-prone atmosphere since the central load balancer is put under significant workload that is linearly scaled with the structure of the system range, and may lead to a performance bottleneck the single point of failure. To overcome the failure in this paper, a fully distributed load rebalancing algorithm is presented to handle the load imbalance problem. The proposed algorithm is compared alongside a centralized approach in a production system ...