Nash equilibria in risk-sensitive dynamic games

Stochastic Processes and their Applications, 2016

In this paper we consider two-person zero-sum risk-sensitive stochastic dynamic games with Borel state and action spaces and bounded reward. The term risk-sensitive refers to the fact that instead of the usual risk neutral optimization criterion we consider the exponential certainty equivalent. The discounted reward case on a finite and an infinite time horizon is considered, as well as the ergodic reward case. Under continuity and compactness conditions we prove that the value of the game exists and solves the Shapley equation and we show the existence of optimal (non-stationary) strategies. In the ergodic reward case we work with a local minorization property and a Lyapunov condition and show that the value of the game solves the Poisson equation. Moreover, we prove the existence of optimal stationary strategies. A simple example highlights the influence of the risk-sensitivity parameter. Our results generalize findings in [1] and answer an open question posed there.

2021

Abstract. We study nonzero-sum stochastic games for continuous time Markov decision processes on a denumerable state space with risk-sensitive ergodic cost criterion. Transition rates and cost rates are allowed to be unbounded. Under a Lyapunov type stability assumption, we show that the corresponding system of coupled HJB equations admits a solution which leads to the existence of a Nash equilibrium in stationary strategies. We establish this using an approach involving principal eigenvalues associated with the HJB equations. Furthermore, exploiting appropriate stochastic representation of principal eigenfunctions, we completely characterize Nash equilibria in the space of stationary Markov strategies.

arXiv (Cornell University), 2016

We study nonzero-sum stochastic games for continuous time Markov chains on a denumerable state space with risk sensitive discounted and ergodic cost criteria. For the discounted cost criterion we first show that the corresponding system of coupled HJB equations has an appropriate solution. Then under an additional additive structure on the transition rate matrix and payoff functions, we establish the existence of a Nash equilibrium in Markov strategies. For the ergodic cost criterion we assume a Lyapunov type stability assumption and a small cost condition. Under these assumptions we show that the corresponding system of coupled HJB equations admits a solution which leads to the existence of Nash equilibrium in stationary strategies.

2021

Abstract. We consider zero-sum stochastic games for continuous time Markov decision processes with risk-sensitive average cost criterion. Here the transition and cost rates may be unbounded. We prove the existence of the value of the game and a saddle-point equilibrium in the class of all stationary strategies under a Lyapunov stability condition. This is accomplished by establishing the existence of a principal eigenpair for the corresponding Hamilton-Jacobi-Isaacs (HJI) equation. This in turn is established by using the nonlinear version of Krein-Rutman theorem. We then obtain a characterization of the saddle-point equilibrium in terms of the corresponding HJI equation. Finally, we use a controlled population system to illustrate results.

Stochastic Analysis and Applications, 2016

We study infinite horizon discounted-cost and ergodic-cost risk-sensitive zero-sum stochastic games for controlled continuous time Markov chains on a countable state space. For the discounted-cost game we prove the existence of value and saddle-point equilibrium in the class of Markov strategies under nominal conditions. For the ergodic-cost game we prove the existence of values and saddle point equilibrium by studying the corresponding Hamilton-Jacobi-Isaacs equation under a certain Lyapunov condition.

3C Empresa. Investigación y pensamiento crítico

In this manuscript, we study continuous-time risk-sensitive finite-horizon time-homogeneous zero-sum dynamic games for controlled Markov decision processes (MDP) on a Borel space. Here, the transition and payoff functions are extended real-valued functions. We prove the existence of the game’s value and the uniqueness of the solution of Shapley equation under some reasonable assumptions. Moreover, all possible saddle-point equilibria are completely characterized in the class of all admissible feedback multi-strategies. We also provide an example to support our assumptions.

Stochastic Analysis and Applications, 2020

We study nonzero-sum stochastic differential games with risk-sensitive discounted cost criteria. Under fairly general conditions on drift term and diffusion coefficients, we establish a Nash equilibrium in Markov strategies for the discounted cost criterion. We achieve our results by studying relevant systems of coupled HJB equations.

— We investigate the use of pricing mechanisms as a means to achieve a desired feedback control strategy among selfish agents. We study a hierarchical linear-quadratic game with many dynamically coupled Nash followers and an uncoupled leader. The leader influences the game by choosing the quadratic dependence on control actions for each follower's cost function. We show that determining whether the leader can establish the desired feedback control as a Nash equilibrium among the followers is a convex feasibility problem for the continuous-time infinite horizon, discrete-time infinite horizon, and discrete-time finite horizon settings, and we present several extensions to this main result. In particular, we discuss methods for ensuring that the total cost incurred due to the leader's pricing is as close as possible to a specified nominal cost, as well as methods for minimizing the explicit dependence of a player's cost on other players' control inputs. Finally, we apply the proposed method to the problem of ensuring the security of a multi-network and to the problem of pricing for controlled diffusion in a general network.

IEEE Transactions on Automatic Control, 1997

This paper employs logarithmic transformations to establish relations between continuous-time nonlinear partially observable risk-sensitive control problems and analogous output feedback dynamic games. The first logarithmic transformation is introduced to relate the stochastic information state to a deterministic information state. The second logarithmic transformation is applied to the risk-sensitive cost function using the Laplace-Varadhan lemma. In the small noise limit, this cost function is shown to be logarithmically equivalent to the cost function of an analogous dynamic game.

Mathematics of Control, Signals, and Systems, 1996

We consider duality relations between risk-sensitive stochastic control problems and dynamic games. They are derived from two basic duality results, the first involving free energy and relative entropy and resulting from a Legendre-type transformation, the second involving power functions. Our approach allows us to treat, in essentially the same way, continuous-and discrete-time problems, with complete and partial state observation, and leads to a very natural formal justification of the structure of the cost functional of the dual. It also allows us to obtain the solution of a stochastic game problem by solving a risk-sensitive control problem.