Optimal probabilistic routing in distributed parallel queues

Queueing Systems, 1995

We consider the problem of allocating a single server to a system of queues with Poisson arrivals. Each queue represents a class of jobs and possesses a holding cost rate, general service distribution, and a set-up cost. The objective is to minimize the expected cost due to the waiting of jobs and the switching of the server. A set-up cost is required to effect an instantaneous switch from one queue to another. We partially characterize an optimal policy and provide a simple heuristic scheduling policy. The heuristic's performance is evaluated in the cases of two and three queues by comparison with a numerically obtained optimal policy. Simulation results are provided to demonstrate the effectiveness of our heuristic over a wide range of problem instances with four queues.

Probability in the Engineering and …, 1992

Journal of Applied Probability, 1999

2004

The optimality of shortest remaining processing time (SRPT) and its variants with respect to minimizing mean sojourn times are well known. Some recent studies have further argued that SRPT does not unfairly penalize large customers in order to benefit small customers, and thus have proposed the use of SRPT to improve performance in computer systems under various applications such as Web sites and databases. On the other hand, the variance of customer sojourn times is another important property of performance in these systems. We therefore consider alternative approaches to scheduling customers in queueing systems with the goal of providing mean sojourn times relatively close to those obtained under SRPT while also providing better variance properties. Our analysis includes deriving expressions for the mean and variance of customer sojourn times in these queueing systems, as well as for the parameters of the alternative scheduling policies. These results illustrate and quantify a fun...

1989

In this paper we derive an approximation for the mean response time of a multiple queue system in which shortest queue routing is used. We assume there are K identical queues with infinite capacity and service times that are exponentially distributed. Arrivals of jobs to this system are Poisson and are routed to a queue of minimal length. We develop an approximation which is based on both theoretical and experimental considerations and, for K 5 8, has an relative error of less than one half of one percent when compared to simulation. For K = 16, the relative error is still acceptable, being less than 2 percent. An application to a model of parallel processing and a comparison of static and dynamic load balancing schemes are presented.

2010

We investigate an optimal scheduling problem for a discrete-time system of two parallel queues with infinite capacity, sharing two symmetrical servers. This model can be used to study a variety of scheduling problems in wireless networks. At any time slot, a queue can be served by one or two connected servers; the queue-server connectivity is assumed to be random and modeled by a two-state Markov chain. The arrivals to each queue are assumed to be independent and identically distributed. A scheduled packet completes service successfully with a given probability. Otherwise, it has to be retransmitted in a later time slot. The optimal scheduling policy is defined as the server allocation policy that minimizes, in a stochastic ordering sense, the total number of packets in the system. We prove, using a dynamic coupling method, that a "Most Balancing" policy, a policy that attempts to balance the lengths of the two queues, is optimal. We also compare the performance of the optimal policy to that of a few other policies via simulations.

Probability in the Engineering and Informational Sciences, 2009

We consider a network of parallel service stations each modelled as a single server queue. Each station serves its own dedicated customers as well as generic customers who are routed from a central controller. We suppose that the cost incurred by a customer is an increasing function of her time spent in system. In a significant advance on most previous work, we do not require waiting costs to be convex, still less linear. With the objective of minimizing the long-run average cost, we develop heuristic routing policies and demonstrate their superior performance in an extensive numerical study.

Proceedings of the 11th EAI International Conference on Performance Evaluation Methodologies and Tools

Routing jobs to parallel servers is a common and important task in today's computer systems. Join-the-shortest-queue (JSQ) routing minimizes the mean response time under rather general settings as long as the servers are identical and service times are independent and exponentially distributed. Apart from this, surprisingly few optimality results exist, mainly due to the complexities arising from the infinite state spaces. Indeed, it is difficult to analyze the performance of any given routing policy. In this paper, we consider an elementary job routing problem with heterogeneous servers and general cost structures. By a novel approximation, we reduce the state space to finite size, which enables us to estimate the mean performance, and to determine (practically) optimal routing policies, for a large class of cost structures. We demonstrate the approximation and its application to job routing policy optimization in numerical examples. CCS CONCEPTS • Mathematics of computing → Queueing theory; • Theory of computation → Markov decision processes; Routing and network design problems; • Information systems → Data centers;

Queueing Systems, 2011

Consider a number of parallel queues, each with an arbitrary capacity and multiple identical exponential servers. The service discipline in each queue is firstcome-first-served (FCFS). Customers arrive according to a state-dependent Poisson process. Upon arrival, a customer joins a queue according to a state-dependent policy or leaves the system immediately if it is full. No jockeying among queues is allowed. An incoming customer to a parallel queue has a general patience time dependent on that queue after which he/she must depart from the system immediately. Parallel queues are of two types: type 1, wherein the impatience mechanism acts on the waiting time; or type 2, a single server queue wherein the impatience acts on the sojourn time. We prove a key result, namely, that the state process of the system in the long run converges in distribution to a well-defined Markov process. Closed-form solutions for the probability density function of the virtual waiting time of a queue of type 1 or the offered sojourn time of a queue of type 2 in a given state are derived which are, interestingly, found to depend only on the local state of the queue. The efficacy of the approach is illustrated by some numerical examples.

Performance Evaluation, 1993

Nelson, R.D. and T.K. Philips, An approximation for the mean response time for shortest queue routing with general interarrival and service times, Performance Evaluation 17 (1993) 123-139. In this paper we derive an approximation for the mean respone time of a multiple queue system in which shortest queue routing is used. We assume there are K identical queues with infinite capacity. Interarrival and service times are generally distributed, and an arriving job is routed to a queue of minimal length. Our approximation is a simple closed form equation that requires only the mean and coefficient of variation of job's interarrival and service times. The approximation is extensively compared to simulated values for values of K ~< 8, and has small relative errors, typically less than 5%, for systems where the coefficient of variation of interarrival and serivce times are both ~< 1. For the system consisting of Poisson arrivals and exponential service times, we extend the approximation so that the error is less than one half of one percent for K~<8.