BEST RESPONSE DYNAMICS FOR CONTINUOUS ZERO-SUM GAMES
sylvain sorin2006
Sign up for access to the world's latest research
checkGet notified about relevant papers
checkSave papers to use in your research
checkJoin the discussion with peers
checkTrack your impact
Abstract
We study best response dynamics in continuous time for continuous concave-convex zero-sum games and prove convergence of its trajectories to the set of saddle points, thus providing a dynamical proof of the minmax theorem. Consequences for the corresponding discrete time process with small or diminishing step-sizes are established, including convergence of the fictitious play procedure.
Related papers
Two general properties of the saddle-point solutions of dynamic games
IEEE Transactions on Automatic Control, 1977
In deterministic team problems every closed-loop representation of an optimal open-loop solution is also optimal. This property, however, no longer holds true when the optimization problem is a zero-sum or a nonzero-sum game. In zero-sum games, two weaker (but still general enough) versions of this statement are valid, which still fail to hold in the case of nonzero-sum games. In this correspondence we state and prove these two general properties of the saddle-point solution in dynamic games.
On asymptotic value for dynamic games with saddle point
The paper is concerned with two-person games with saddle point. We investigate the limits of value functions for long-time-average payoff, discounted average payoff, and the payoff that follows a probability density on R ≥0 .
Advances in Zero-Sum Dynamic Games
Handbook of Game Theory with Economic Applications, 2015
The survey presents recent results in the theory of two-person zero-sum repeated games and their connections with differential and continuous-time games. The emphasis is made on the following points: 1) A general model allows to deal simultaneously with stochastic and informational aspects. 2) All evaluations of the stage payoffs can be covered in the same framework (and not only the usual Cesàro and Abel means). 3) The model in discrete time can be seen and analyzed as a discretization of a continuous time game. Moreover, tools and ideas from repeated games are very fruitful for continuous time games and vice versa. 4) Numerous important conjectures have been answered (some in the negative). 5) New tools and original models have been proposed. As a consequence, the field (discrete versus continuous time, stochastic versus incomplete information models) has a much more unified structure, and research is extremely active.
Zero-sum Stochastic Games: Limit Optimal Trajectories
arXiv: Optimization and Control, 2018
We consider zero sum stochastic games. For every discount factor $\lambda$, a time normalization allows to represent the game as being played on the interval [0, 1]. We introduce the trajectories of cumulated expected payoff and of cumulated occupation measure up to time t $\in$ [0, 1], under $\epsilon$-optimal strategies. A limit optimal trajectory is defined as an accumulation point as the discount factor tends to 0. We study existence, uniqueness and characterization of these limit optimal trajectories for absorbing games.
Zero-Sum Games: The General Case
Universitext, 2019
This chapter deals with the general case of a zero-sum game, where the strategy sets of the players may be infinite. A zero-sum game will always be a triple (S, T, g) where S and T are non-empty sets and g is a mapping from S × T to R. By definition, the game has a value if sup s∈S inf t∈T g(s, t) = inf t∈T sup s∈S g(s, t), and minmax theorems, such as von Neumann minmax theorem in the previous Chap. 2, refer to results providing sufficient conditions on the triple (S, T, g) for the existence of a value. Recall that if the value exists, a strategy s in S achieving the supremum in sup s∈S inf t∈T g(s, t) is called an optimal strategy of player 1 in the game. Similarly, t ∈ T achieving the infimum in inf t∈T sup s∈S g(s, t) is called optimal for player 2. We prove here various minmax theorems. We start with Sion's theorem for convex compact action sets and payoff functions which are quasi-concave upper semicontinuous in the first variable and quasi-convex lower semi-continuous in the second variable. Then we prove the standard minmax theorem in mixed strategies, extending von Neumann's theorem to compact Hausdorff action sets and measurable bounded payoff functions which are u.s.c. in the first variable and l.s.c. in the second variable. Finally, we consider the value operator (strategy sets are fixed and the payoff function varies) and its directional derivatives, and introduce the derived game. 3.2 Minmax Theorems in Pure Strategies The following result is known as the intersection lemma (see [22, p. 172]), and will be useful later. Lemma 3.2.1 Let C 1 ,. .. , C n be non-empty convex compact subsets of a Euclidean space. Assume that the union n i=1 C i is convex and that for each j = 1,. .. , n, the intersection i = j C i is non-empty. Then the full intersection n i=1 C i is also non-empty.
Zero-sum continuous-time Markov games with unbounded transition and discounted payoff rates
Bernoulli, 2005
This paper is concerned with two-person zero-sum games for continuous-time Markov chains, with possibly unbounded payoff and transition rate functions, under the discounted payoff criterion. We give conditions under which the existence of the value of the game and a pair of optimal stationary strategies is ensured by using the optimality (or Shapley) equation. We prove the convergence of the value iteration scheme to the game's value and to a pair of optimal stationary strategies. Moreover, when the transition rates are bounded we further show that the convergence of value iteration is exponential. Our results are illustrated with a controlled queueing system with unbounded transition and reward rates.
A New Existence and Uniqueness Theorem for Continuous Games
The B.E. Journal of Theoretical Economics, 2000
This paper derives a general sufficient condition for existence and uniqueness in continuous games using a variant of the contraction mapping theorem applied to mappings from a subset of the real line on to itself. We first prove this contraction mapping variant, and then show how the existence of a unique equilibrium in the general game can be shown by proving the existence of a unique equilibrium in an iterative sequence of games involving such mappings. Finally, we show how a general condition for this to occur is that a matrix derived from the Jacobian matrix of best-response functions has positive leading principal minors, and how this condition generalises some existing uniqueness theorems for particular games. In particular, we show how the same conditions used in those theorems to show uniqueness, also guarantee existence in games with unbounded strategy spaces.
Policy Optimization Provably Converges to Nash Equilibria in Zero-Sum Linear Quadratic Games
2019
We study the global convergence of policy optimization for finding the Nash equilibria (NE) in zero-sum linear quadratic (LQ) games. To this end, we first investigate the landscape of LQ games, viewing it as a nonconvex-nonconcave saddle-point problem in the policy space. Specifically, we show that despite its nonconvexity and nonconcavity, zero-sum LQ games have the property that the stationary point of the objective function with respect to the linear feedback control policies constitutes the NE of the game. Building upon this, we develop three projected nested-gradient methods that are guaranteed to converge to the NE of the game. Moreover, we show that all of these algorithms enjoy both globally sublinear and locally linear convergence rates. Simulation results are also provided to illustrate the satisfactory convergence properties of the algorithms. To the best of our knowledge, this work appears to be the first one to investigate the optimization landscape of LQ games, and pro...
Advances in Dynamic Games, Applications to Economics, Finance, Optimization, and Stochastic Control
Automatica, 2006
His main research interests include: stability of nonlinear systems, in particular Input-to-State stability and Passivity, constrained and model predictive control and, more recently, systems biology and nonlinear analysis inspired by biological applications.
Constant payoff in zero-sum stochastic games
Annales de l'Institut Henri Poincaré, Probabilités et Statistiques
In a zero-sum stochastic game, at each stage, two adversary players take decisions and receive a stage payoff determined by them and by a random variable representing the state of nature. The total payoff is the discounted sum of the stage payoffs. Assume that the players are very patient and use optimal strategies. We then prove that, at any point in the game, players get essentially the same expected payoff: the payoff is constant. This solves a conjecture by Sorin, Venel and Vigeral (2010). The proof relies on the semi-algebraic approach for discounted stochastic games introduced by Bewley and Kohlberg (1976), on the theory of Markov chains with rare transitions, initiated by Friedlin and Wentzell (1984), and on some variational inequalities for value functions inspired by the recent work of Davini, Fathi, Iturriaga and Zavidovique (2016).
Related topics
Related papers
Better-reply dynamics and global convergence to Nash equilibrium in aggregative games
Games and Economic Behavior, 2006
We consider n-person, generic, quasi-concave games with continuous action spaces and in which the payo¤ of a player depends on her own action and the sum of the actions of opponents. We study a discrete-time, stochastic adjustment process (the better-reply dynamics) in which players move towards better replies. Our main result is a su¢cient condition for this process to converge globally to a Nash equilibrium of the game. This condition requires that actions be either locally strategic substitutes or locally strategic complements for all players at each Nash equilibrium that is locally asymptotically stable under an associated deterministic, adjusted best-reply dynamics. We provide an example of a 2-person game with a unique Nash equilibrium at which the derivatives of the best-reply functions have di¤erent signs and in which the better-reply dynamics does not converge to the equilibrium.
Some thoughts on saddle-point conditions and information structures in zero-sum differential games
Journal of Optimization Theory and Applications, 1976
For a very simple two-stage, linear-quadratic, zero-sum difference game with dynamic information structure, we show that (i) there exist nonlinear saddle-point strategies which require the same existence conditions as the well-known linear, closed-loop, no-memory solution and (ii) there exist both linear and nonlinear saddle-point strategies which require more stringent conditions than the unique open-loop solution. We then discuss the implication of this result with respect to the existence of saddle points in zero-sum differential games for different information patterns.
Continuity of equilibria for two-person zero-sum games with noncompact action sets and unbounded payoffs
Annals of Operations Research, 2017
This paper extends Berge's maximum theorem for possibly noncompact action sets and unbounded cost functions to minimax problems and studies applications of these extensions to two-player zero-sum games with possibly noncompact action sets and unbounded payoffs. For games with perfect information, also known under the name of turn-based games, this paper establishes continuity properties of value functions and solution multifunctions. For games with simultaneous moves, it provides results on the existence of lopsided values (the values in the asymmetric form) and solutions. This paper also establishes continuity properties of the lopsided values and solution multifunctions.
Limit Value of Dynamic Zero-Sum Games with Vanishing Stage Duration
Mathematics of Operations Research
We consider two person zero-sum games where the players control, at discrete times {tn} induced by a partition Π of R + , a continuous time Markov state process. We prove that the limit of the values vΠ exist as the mesh of Π goes to 0. The analysis covers the cases of : 1) stochastic games (where both players know the state) 2) symmetric no information. The proof is by reduction to a deterministic differential game.
On saddle-point optimality in differential games
Journal of Optimization Theory and Applications, 1976
A family of two-person, zero-sum differential games in which the admissible strategies are Borel measurable is defined, and two types of saddle-point conditions are introduced as optimality criteria. In one, saddle-point candidates are compared at each point of the state space with all playable pairs at that point; and, in the other, they are compared only with strategy pairs playable on the entire state space. As a theorem, these two types of optimality are shown to be equivalent for the defined family of games. Also, it is shown that a certain closure property-is sufficient for this equivalence. A game having admissible strategies everywhere constant, in which the two types of saddle-point candidates are not equivalent, is discussed.
Optimality principles of dynamic programming in differential games
Journal of Mathematical Analysis and Applications, 1989
Convergence in games with continua of equilibria
Journal of Mathematical Economics, 2020
In game theory, the question of convergence of dynamical systems to the set of Nash equilibria has often been tackled. When the game admits a continuum of Nash equilibria, however, a natural and challenging question is whether convergence to the set of Nash equilibria implies convergence to a Nash equilibrium. In this paper we introduce a technique developed in as a useful way to answer this question. We illustrate it with the best-response dynamics in the local public good game played on a network, where continua of Nash equilibria often appear.
A strategy-based proof of the existence of the value in zero-sum differential games
arXiv: Optimization and Control, 2013
The value of a zero-sum differential games is known to exist, under Isaacs' condition, as the unique viscosity solution of a Hamilton-Jacobi-Bellman equation. In this note we provide a self-contained proof based on the construction of $\ep$-optimal strategies, which is inspired by the "extremal aiming" method from Krasovskii and Subbotin.
Zero-Sum Games for Continuous-time Markov Decision Processes with Risk-Sensitive Average Cost Criterion
2021
Abstract. We consider zero-sum stochastic games for continuous time Markov decision processes with risk-sensitive average cost criterion. Here the transition and cost rates may be unbounded. We prove the existence of the value of the game and a saddle-point equilibrium in the class of all stationary strategies under a Lyapunov stability condition. This is accomplished by establishing the existence of a principal eigenpair for the corresponding Hamilton-Jacobi-Isaacs (HJI) equation. This in turn is established by using the nonlinear version of Krein-Rutman theorem. We then obtain a characterization of the saddle-point equilibrium in terms of the corresponding HJI equation. Finally, we use a controlled population system to illustrate results.
Differential games with maximum cost
Nonlinear Analysis-theory Methods & Applications, 1990
GIVEN THE dynamics on the interval [t, T] $ = f(r9 r(r), C(r), q(r)) OrtcrsT, (0.1) and the payoff r(t) = x E IR", (0.2)
Loading Preview
Sorry, preview is currently unavailable. You can download the paper by clicking the button above.