The Complexity of Stochastic Rabin and Streett Games

2016 IEEE 55th Conference on Decision and Control (CDC), 2016

We introduce a new class of 2 1 /2-player games, the 2 1 /2-player GR(1) games, that allows for solving problems of stochastic nature by adding a probabilistic component to simple 2-player GR(1) games. Further, we present an efficient approach for solving qualitative 2 1 /2-player GR(1) games with polynomial-time complexity. Our approach is based on a reduction from 2 1 /2-player GR(1) games to 2-player GR(1) games that allows for solving the game and constructing, from a sure winning strategy for player 2 (resp. 3) in a 2-player GR(1) game, an almost-sure (resp. positively) winning strategy for its corresponding 2 1 /2-player GR(1) game. Key to the effectiveness of the proposed approach is the fact that the reduction generates a 2-player game that is linearly larger than the original 2 1 /2player game, more precisely, it is linear with respect to the number of probabilistic states in the 2 1 /2-player GR(1) game.

Mathematical Methods of Operations Research, 2004

We introduce the concept of locally acting distributed players into sequential stochastic games with general compact state and action spaces. The state transition function for the system is of local structure as well and this results in Markov properties in space and time for the describing processes. We prove that we can reduce optimality problems for local strategies to only considering Markov strategies. We further prove the existence of optimal strategies and of a value for the game with respect to the asymptotic average reward criterion.

Theory and Decision Library, 1991

Stochastic games were first formulated by Shapley in 1953. In his fundamental paper Shapley [13] established the existence of value and optimal stationary strategies for zero-sum ,a-discounted stochastic games with finitely many states and actions for the two players. A positive stochastic game with countable state space and finite action spaces consists of the following objects: 1. State space Sthe set of nonnegative integers. 2. Finite action spaces A(B) for players 1(11). 3. The space of mixed strategies P(A)(P(B)) on the action spaces A(B) for players 1(11). 4. Nonnegative (immediate) reward function r(s, a, b). 5. Markovian transition q(tls, a, b) where q(tls, a, b) is the chance of moving from state s to state t when actions a, b are chosen by players I and II in the current state s. When playing a stochastic game, the players, in selecting their actions for the k th day, could use all the available information to them till that day, namely on the partial history up to the kth day given by (sllallbllsz,az,bz, ...sk-lIak-l,bk-l,Sk). Thus a strategy P for player I is a sequence (PlIPZ, ..) where Pk selects a mixed strategy (an element of P(A), for the k th day. We can classify stochastic games with additional structure on the immediate rewards and transition probabilities. The law of motion is said to be controlled by one player (say player II) if q(tls,i,i) = q(tls,i) for all i. We call a stochastic

Proceedings of the 7th and 8th Asian Logic Conferences, 2003

McNaughton in his known paper [7], motivated by the work of Gurevich and Harrington [4], introduced a class of games played on finite graphs. In his paper McNaughton proves that winners in his games have winning strategies that can be implemented by finite state automata. McNaughton games have attracted attention of many experts in the area, partly because the games have close relationship with automata theory, the study of reactive systems, and logic (see, for instance, [12] and [11]). McNaughton games can also be used to develop game-theoretical approach for many important concepts in computer science such as models for concurrency, communication networks, and update networks, and provide natural examples of computational problems. For example, Nerode, Remmel and Yakhnis in a series of papers (e.g., [8], [9]) developed foundations of concurrent programming in which finite state strategies of McNaughton games are identified with distributed concurrent programs.

Lecture Notes in Computer Science, 2014

We consider two-player partial-observation stochastic games on finitestate graphs where player 1 has partial observation and player 2 has perfect observation. The winning condition we study are ω-regular conditions specified as parity objectives. The qualitative-analysis problem given a partial-observation stochastic game and a parity objective asks whether there is a strategy to ensure that the objective is satisfied with probability 1 (resp. positive probability). These qualitative-analysis problems are known to be undecidable. However in many applications the relevant question is the existence of finite-memory strategies, and the qualitative-analysis problems under finite-memory strategies was recently shown to be decidable in 2EXPTIME. We improve the complexity and show that the qualitative-analysis problems for partial-observation stochastic parity games under finite-memory strategies are EXPTIME-complete; and also establish optimal (exponential) memory bounds for finite-memory strategies required for qualitative analysis.

Proceedings of the 43rd annual ACM symposium on Theory of computing - STOC '11, 2011

Shapley's discounted stochastic games, Everett's recursive games and Gillette's undiscounted stochastic games are classical models of game theory describing two-player zero-sum games of potentially infinite duration. We describe algorithms for exactly solving these games. When the number of positions of the game is constant, our algorithms run in polynomial time.

2006

We consider two-player infinite games played on graphs. The games are concurrent, in that at each state the players choose their moves simultaneously and independently, and stochastic, in that the moves determine a probability distribution for the successor state. The value of a game is the maximal probability with which a player can guarantee the satisfaction of her objective. We show that the values of concurrent games with ω-regular objectives expressed as parity conditions can be computed in NP ∩ coNP. This result substantially improves the best known previous bound of 3EXPTIME. It also shows that the full class of concurrent parity games is no harder than the special cases of turnbased deterministic parity games (Emerson-Jutla) and of turn-based stochastic reachability games (Condon), for both of which NP ∩ coNP is the best known bound. While the previous, more restricted NP ∩ coNP results for graph games relied on the existence of particularly simple (pure memoryless) optimal strategies, in concurrent games with parity objectives optimal strategies may not exist, and ε-optimal strategies (which achieve the value of the game within a parameter ε > 0) require in general both randomization and infinite memory. Hence our proof must rely on a more detailed analysis of strategies and, in addition to the main result, yields two results that are interesting on their own.

Proceedings of the Twentieth Annual ACM-SIAM Symposium on Discrete Algorithms, 2009

We consider the problem of computing Nash Equilibria of action-graph games (AGGs). AGGs, introduced by Bhat and Leyton-Brown, is a succinct representation of games that encapsulates both 'local'dependencies as in graphical games, and partial indifference to other agents' identities as in anonymous games, which occur in many natural settings. This is achieved by specifying a graph on the set of actions, so that the payoff of an agent for selecting a strategy depends only on the number of agents playing each of the neighboring strategies in the action graph. We present a Polynomial Time Approximation Scheme for computing mixed Nash equilibria of AGGs with constant treewidth and a constant number of agent types (and an arbitrary number of strategies), together with hardness results for the cases when either the treewidth or the number of agent types is unconstrained. In particular, we show that even if the action graph is a tree, but the number of agent-types is unconstrained, it is NP-complete to decide the existence of a pure-strategy Nash equilibrium and PPAD-complete to compute a mixed Nash equilibrium (even an approximate one); similarly for symmetric AGGs (all agents belong to a single type), if we allow arbitrary treewidth. These hardness results suggest that, in some sense, our PTAS is as strong of a positive result as one can expect.

Journal of Computer and System Sciences, 2013

We consider concurrent games played on graphs. At every round of a game, each player simultaneously and independently selects a move; the moves jointly determine the transition to a successor state. Two basic objectives are the safety objective to stay forever in a given set of states, and its dual, the reachability objective to reach a given set of states. First, we present a simple proof of the fact that in concurrent reachability games, for all ε > 0, memoryless ε-optimal strategies exist. A memoryless strategy is independent of the history of plays, and an ε-optimal strategy achieves the objective with probability within ε of the value of the game. In contrast to previous proofs of this fact, our proof is more elementary and more combinatorial. Second, we present a strategy-improvement (a.k.a. policy-iteration) algorithm for concurrent games with reachability objectives. Finally, we present a strategy-improvement algorithm for turn-based stochastic games (where each player selects moves in turns) with safety objectives. Our algorithms yield sequences of player-1 strategies which ensure probabilities of winning that converge monotonically (from below) to the value of the game.

Information and Computation, 1997

We show that Safra's determinization of |-automata with Streett (strong fairness) acceptance condition also gives memoryless winning strategies in infinite games, for the player whose acceptance condition is the complement of the Streett condition. Both determinization and memorylessness are essential parts of known proofs of Rabin's tree automata complementation lemma. Also, from Safra's determinization construction, along with its memoryless winning strategy extension, a single exponential complementation of Streett tree automata follows. A different single exponential construction and proof first appeared in [N. Klarlund (1992), Progress measures, immediate determinacy, and a subset construction for tree automata, in``Proceedings, 7th IEEE Symposium on Logics in Computer Science''].