Stochastic Approximations and Differential Inclusions

Mathematics of Operations Research, 2006

We apply the theoretical results on "stochastic approximations and differential inclusions" developed in Benaïm, to several adaptive processes used in game theory including: classical and generalized approachability, no-regret potential procedures (Hart and Mas-Colell), smooth fictitious play (Fudenberg and Levine).

Mathematics of Operations Research, 2010

A successful method to describe the asymptotic behavior of a discrete time stochastic process governed by some recursive formula is to relate it to the limit sets of a well chosen mean differential equation. Under an attainability condition, convergence to a given attractor of the flow induced by this dynamical system was proved to occur with positive probability (Benaïm, 1999) for a class of Robbins Monro algorithms. generalised this approach for stochastic approximation algorithms whose average behavior is related to a differential inclusion instead. We pursue the analogy by extending to this setting the result of convergence with positive probability to an attractor.

Dynamic Games and Applications, 2012

We present upper-semicontinuity results for attractors and the chain-recurrent set of differential inclusions, in particular w.r.t. discretizations, and applications to game dynamics. Keywords Differential inclusion • Attractor • Chain-recurrence • Discretization • Game dynamics 1 Set-Valued Dynamical Systems: Notations and Basic Properties Let F be an upper semi-continuous set-valued map from R k to itself with compact convex values and X be a given compact convex subset of R k. Σ denotes the set of solutions of the differential inclusioṅ x ∈ F (x) (1) for which X is forward invariant. More precisely, x ∈ Σ iff x is an absolutely continuous map from an interval I x to R k , where [0, +∞) ⊂ I x , that satisfies (1) a.e., such that x(t) ∈ X for some t ∈ I x and for any s, t ∈ I x , s < t and x(s) ∈ X implies x(t) ∈ X.

1999

The Annals of Applied Probability, 2012

This paper considers a stochastic approximation algorithm, with decreasing step size and martingale difference noise. Under very mild assumptions, we prove the non convergence of this process toward a certain class of repulsive sets for the associated ordinary differential equation (ODE). We then use this result to derive the convergence of the process when the ODE is cooperative in the sense of . In particular, this allows us to extend significantly the main result of on the convergence of stochastic fictitious play in supermodular games.

Journal of Mathematical Analysis and Applications, 1989

Mathematics of Control, Signals, and Systems, 1996

We consider duality relations between risk-sensitive stochastic control problems and dynamic games. They are derived from two basic duality results, the first involving free energy and relative entropy and resulting from a Legendre-type transformation, the second involving power functions. Our approach allows us to treat, in essentially the same way, continuous-and discrete-time problems, with complete and partial state observation, and leads to a very natural formal justification of the structure of the cost functional of the dual. It also allows us to obtain the solution of a stochastic game problem by solving a risk-sensitive control problem.

IEEE Transactions on Automatic Control, 2020

In this paper, we analyze the behavior of stochastic approximation schemes with set-valued maps in the absence of a stability guarantee. We prove that after a large number of iterations if the stochastic approximation process enters the domain of attraction of an attracting set it gets locked into the attracting set with high probability. We demonstrate that the above result is an effective instrument for analyzing stochastic approximation schemes in the absence of a stability guarantee, by using it obtain an alternate criteria for convergence in the presence of a locally attracting set for the mean field and by using it to show that a feedback mechanism, which involves resetting the iterates at regular time intervals, stabilizes the scheme when the mean field possesses a globally attracting set, thereby guaranteeing convergence. The results in this paper build on the works of V.S. Borkar, C. Andrieu and H. F. Chen , by allowing for the presence of set-valued drift functions.

Automatica, 2006

His main research interests include: stability of nonlinear systems, in particular Input-to-State stability and Passivity, constrained and model predictive control and, more recently, systems biology and nonlinear analysis inspired by biological applications.

2000

This paper provides deterministic approximation results for stochastic processes that arise when finite populations recurrently play finite games. The deterministic approximation is defined in continuous time as a system of ordinary differential equations of the type studied in evolutionary game theory. We establish precise connections between the long-run behavior of the stochastic process, for large populations, and its deterministic approximation.