The Average Diffusion Method for the Load Balancing Problem

Theory of Computing Systems, 2002

Several different diffusion schemes have previously been developed for load balancing on homogeneous processor networks. We generalize existing schemes, in order to deal with heterogeneous networks. Generalized schemes may operate efficiently on networks where each processor can have arbitrary computing power, i.e., the load will be balanced proportionally to these powers. The balancing flow that is calculated by schemes for homogeneous networks is minimal with regard to the l 2-norm and we prove this to hold true for generalized schemes, too. We demonstrate the usability of generalized schemes by a number of experiments on several heterogeneous networks.

International Journal of Information Technology and Computer Science, 2014

In distributed computing system some nodes are very fast and some are slow and during the computation many fast nodes become idle or under loaded while the slow nodes become over loaded due to the uneven distribution of load in the system. In distributed system, the most common important factor is the information collection about loads on different nodes. The success of load balancing algorithm depends on how quickly the information about the load in the system is collected by a node willing to transfer or accept load. In this paper we have shown that the number of communication overheads depends on the number of overloaded nodes present in the domain of an under loaded nodes and vice-versa. We have also shown that communication overhead for load balancing is always fairly less than KN but in worst case our algorithm's complexity becomes equal to KN.

The grid and cluster computing uses interconnected nodes to solve a problem in parallel in order to improve the response time of the system. Diffusive load balancing algorithms works well when the nodes in the system have the same processing capacity. But little attention was paid in diffusion load balancing algorithms in the literature for distributing the workload in the nodes with different processing capabilities when the load of the nodes is treated as integers. When the loads are distributed to the nodes without considering their processing capacities it would affect the response time of the system. Effective load balancing in heterogeneity can be achieved by considering the processing capacities of the nodes. This paper propose a diffusive load balancing algorithm which distributes a proportion of excessive workload of heavily loaded node to lightly loaded node by considering the nodes processing capabilities.

Theoretical Computer Science, 2008

This paper studies the Diffusion method for the load balancing problem in case of weighted torus graphs. Closed form formulae for the optimum values of the edge weights are determined using local Fourier analysis. It is shown that an extrapolated version of Diffusion can become twice as fast for the stretched torus graphs.

7th International Symposium on Parallel Architectures, Algorithms and Networks, 2004. Proceedings., 2004

Efficient load balancing algorithms are the key to many efficient parallel applications. Until now, research in this area mainly focused on static networks. However, observations show that diffusive algorithms, originally designed for these networks, can also be applied in non static scenarios. In this paper we prove that the general diffusion scheme can be deployed on dynamic networks and show that its convergence rate depends on the average value of the quotient of the second smallest eigenvalue and the maximum vertex degree of the networks occurring during the iterations. In the presented experiments we illustrate that even if communication links of static networks fail with high probability, load can still be balanced quite efficiently. Simulating diffusion on ad-hoc networks we demonstrate that diffusive schemes provide a reliable and efficient load balancing strategy also in mobile environments.

We consider the load balancing problem for a synchronous distributed processor network. The processor network is modeled by an undirected, connected graph Î µ in which node Ú ¾ Î possesses a computational load Ù . We want to determine a schedule in order to move load across edges so that the weight on each node is approximately equal. This problem models load balancing when we associate each node with a processor and each edge with a communication link of unbounded capacity between two processors. The schedules for the load balancing problem are iterative in nature and their theory closely resembles the theory of iterative methods for solving large, sparse, linear systems [Mar75, Var62, You71]. There are mainly two iterative load balancing algorithms : diffusion [Cyb89] and dimension exchange [Cyb89, XL97]. Diffusion algorithms assume that a processor exchanges load between neighbour processors simultaneously, whereas dimension exchange assumes that a processor exchanges load with a neighbour processor in each dimension at the time. In this paper we consider the application of accelerated techniques in order to increase the rate of convergence of the diffusive iterative load balancing algorithms. In particular, we compare the application of Semi-Iterative, Second Degree and Variable Extrapolation techniques on the basic diffusion method for various types of network graphs.

2011

Load balancing is a well known problem, which has been extensively addressed in parallel algorithmic. However, there subsist some contexts in which the existing algorithms cannot be used. One of these contexts is the case of dynamic networks where the links between the different elements are intermittent. We propose in this paper an efficient algorithm, based on asynchronous diffusion, to perform load balancing in such a context. A convergence theorem is proposed and proved. Finally, experimental results performed in the SimGrid environment confirm the efficiency of our algorithm.

Parallel Computing, 1996

Many applications require some form of load balancing in order to obtain good performance on parallel computing systems. A number of load balancing algorithms have been proposed, and implemented on real systems. However, application developers are in need of a performance tool that compares and evaluates various balancing algorithms, and helps select the best algorithm for a given application and computing platform. In this paper, we propose a matrix iterative model that can represent a wide range of dynamic load balancing algorithms. The model captures the change in task distribution over the processors due to both the application imbalance and load balancing algorithm. We also define performance measures that capture the costs of executing the parallel application and load balancing algorithm on a given computing platform. The model and the performance measures are used to compare and evaluate various load balancing algorithms. In this paper, the model is parameter&d to represent three load balancing algorithms -the random, diffusion and redistribution algorithms. Model predictions and experimental measurements are compared for the parallel N-body simulation application. This application is representative of a wide class of scientific applications where time-varying workload distributions are present. The model predictions are within 20% of measured values. The performance of the three algorithms are compared, and optimal algorithm parameters are derived showing how the model can be used to both select an algorithm and set its parameters. 0167-8191/%/$15.00 8 1996 Elsevier Science B.V. All rights reserved P/I SOl67-8191(96)00026-9 970 M.A. Franklin, V. Gouindan/ Parallel Computing 22 (1996) 969-989

1989

In this paper we study diffusion schemes for dynamic load balancing on message passing multiprocessor networks. One of the main results concerns conditions under which these dynamic schemes converge and their rates of convergence for arbitrary topologies. These results use the eigenstructure of the iteration matrices that arise in dynamic load balancing. We completely analyze the hypercube network by explicitly computing the eigenstructure of its node adjacency matrix. Using a realistic model of interprocessor communications, we show that a diffusion approach to load balancing on a hypercube multiprocessor is inferior to another approach which we call the dimension exchange method. For a d-dimensional hypercube, we compute the rate of convergence to a uniform work distribution and show that after d + 1 iterations of a diffusion type approach, we can guarantee that the work distribution is approximately within e-* of the uniform distribution independent of the hypercube dimension d. Both static and dynamic random models of work distribution are studied. o 1989 Academic Press, Inc.

An efficient load balancing algorithm may reduce the communication overheads among the nodes in a network .In this paper we propose an improvement over the dynamic load balancing in the 16 processor 2D mess. In decentralized approach both the dimension exchange method and the diffusion method are widely applied for the load balancing. But along with the dimension exchange method we use four nodes as leader nodes to reduce the communication over heads. We made an analytical model to show the improvement.

Proceedings of the sixth annual ACM symposium on Parallel algorithms and architectures - SPAA '94, 1994

Diusion is a well-known algorithm for load-balancing in which tasks move from heavily-loaded processors to lightly-loaded neighbors. This paper presents a rigorous analysis of the performance of the diusion algorithm on arbitrary networks.

Pollack Periodica, 2014

The grid and cluster computing uses interconnected nodes to solve a problem in parallel in order to improve the response time of the job. Diffusive load balancing algorithms work well when the nodes in the system have the same processing capacity. However, little attention paid in diffusion load balancing techniques, for the nodes with different processing capabilities. In this paper, a load-balancing algorithm using diffusion technique proposed for distributing the load between the nodes by treating the loads as an integer quantity. The proposed load balancing algorithm distributes apportion of excessive workload of a heavily loaded node to a lightly loaded node by considering the node's processing capacities.

Proceedings 16th International Parallel and Distributed Processing Symposium, 2002

Efficient load balancing algorithms are the key to many efficient parallel applications. Until now, research in this area has mainly been focusing on homogeneous schemes. However, observations show that the convergence rate of diffusion algorithms can be improved using edge weighted graphs without deteriorating the flows quality. In this paper we consider common interconnection topologies and demonstrate, how optimal edge weights can be calculated for the First and Second Order Diffusion Schemes. Using theoretical analysis and practical experiments we show, what improvements can be archived on selected networks.

Journal of Parallel and Distributed Computing, 1995

Lecture Notes in Computer Science, 2001

In this paper, we present a time dependent diffusion model for load balancing on synchronous networks with dynamically changing topology. This situation can arise for example from link failures. Our model is general and include Cybenko's diffusion models for fixed topology networks. We will show that under some assumptions, the time dependent load balancing algorithm converges and balances the total work load among the processors in the distributed network.