A Strategic Analysis of

Multi-agent Protocols

Sieuwert van Otterloo

A Strategic Analysis of
Multi-agent Protocols
 ILLC Dissertation Series DS-2005-05

For further information about ILLC-publications, please contact

Institute for Logic, Language and Computation

Universiteit van Amsterdam
Plantage Muidergracht 24
1018 TV Amsterdam
phone: +31-20-525 6051
fax: +31-20-525 5206
e-mail: [email protected]
homepage: http://www.illc.uva.nl/
 A Strategic Analysis of
Multi-agent Protocols

Thesis submitted in accordance with the requirements

of the University of Liverpool for the degree of Doctor in
Philosophy

by

Sieuwert Maarten van Otterloo

Defended November 25 2005

Promotors: Prof. Dr. Wiebe van der Hoek
Prof. Dr. Michael Wooldridge
University of Liverpool
Department of Computer Science
Liverpool L69 7ZF
United Kingdom

Co-promotor: Prof. Dr. Johan van Benthem

Institute for Logic Language and Computation
Universiteit van Amsterdam
Plantage Muidergracht 24
1018 TV Amsterdam
The Netherlands

Copyright c 2005-2006 by Sieuwert van Otterloo

Cover design by Ernst van Rheenen. The cover shows Achilles and Ajax playing
a game during the Trojan war
Printed and bound by Febodruk BV

ISBN: 90–5776–153–x
 Contents

Acknowledgments ix

1 Introduction 1
1.1 Multi-agent Protocols . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Protocol Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Outline of this Dissertation . . . . . . . . . . . . . . . . . . . . . 6
1.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2 Logic 9
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Propositional Logic . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.1 Minimal Propositional Logic . . . . . . . . . . . . . . . . . 11
2.2.2 Full Propositional Logic . . . . . . . . . . . . . . . . . . . 13
2.3 Modal Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.3.1 Epistemic Logic . . . . . . . . . . . . . . . . . . . . . . . . 19
2.3.2 Common Knowledge . . . . . . . . . . . . . . . . . . . . . 21
2.4 Theorem Proving, Satisfiability and Model Checking . . . . . . . 22
2.5 Computational Complexity . . . . . . . . . . . . . . . . . . . . . . 27

3 Game Theory 33
3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.2 Strategic Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.3 Extensive Games . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.3.1 Perfect Information . . . . . . . . . . . . . . . . . . . . . . 42
3.3.2 Imperfect Information . . . . . . . . . . . . . . . . . . . . 46
3.4 Existing Work on Logic and Games . . . . . . . . . . . . . . . . . 48
3.4.1 Coalition Logics . . . . . . . . . . . . . . . . . . . . . . . . 49
3.4.2 Power Level Logic and Bisimulation . . . . . . . . . . . . . 52
3.4.3 Alternating-time Temporal Logic . . . . . . . . . . . . . . 52

v
 3.4.4 Dynamic Epistemic Logic . . . . . . . . . . . . . . . . . . 54

4 Logics for Protocols 57

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.2 Defining Effectivity Logic . . . . . . . . . . . . . . . . . . . . . . 58
4.3 Model Checking for EFL . . . . . . . . . . . . . . . . . . . . . . . 60
4.4 Bisimulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.5 Completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.6 Linear Representations . . . . . . . . . . . . . . . . . . . . . . . . 70
4.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

5 Politeness and Side Effects 79

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
5.2 Defining EFLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
5.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.3.1 Alice and Bob eat Cake . . . . . . . . . . . . . . . . . . . 83
5.3.2 Joint Decision Problem . . . . . . . . . . . . . . . . . . . . 85
5.4 Model Checking EFLS . . . . . . . . . . . . . . . . . . . . . . . . 86
5.5 Extensions of EFL . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.5.1 Model Checking efln . . . . . . . . . . . . . . . . . . . . 89
5.5.2 Model Checking eflns . . . . . . . . . . . . . . . . . . . . 92
5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

6 Preference Logics in Extensive Games 99

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
6.2 Preference Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
6.2.1 Bisimulation . . . . . . . . . . . . . . . . . . . . . . . . . . 103
6.2.2 Proof System . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.3 An Alternative Preference Logic . . . . . . . . . . . . . . . . . . . 111
6.3.1 Proof System . . . . . . . . . . . . . . . . . . . . . . . . . 113
6.4 Finite Tree Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
6.5 Backward Induction: An Application . . . . . . . . . . . . . . . . 119
6.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

7 Knowledge Condition Games 127

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
7.2 Defining Knowledge Condition Games . . . . . . . . . . . . . . . . 128
7.2.1 Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
7.2.2 Strategic Games . . . . . . . . . . . . . . . . . . . . . . . . 131
7.2.3 Knowledge Condition Games . . . . . . . . . . . . . . . . . 131
7.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
7.3.1 Anonymous Voting . . . . . . . . . . . . . . . . . . . . . . 133
7.3.2 Fifty-Fifty Problem . . . . . . . . . . . . . . . . . . . . . . 133

vi
 7.3.3 Russian Cards Problem . . . . . . . . . . . . . . . . . . . . 136
7.3.4 Communication Example . . . . . . . . . . . . . . . . . . . 139
7.4 Computational Complexity . . . . . . . . . . . . . . . . . . . . . . 141
7.4.1 Tractable Variants . . . . . . . . . . . . . . . . . . . . . . 146
7.5 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
7.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

8 Entropy and Privacy 153

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
8.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
8.3 Information Theory . . . . . . . . . . . . . . . . . . . . . . . . . . 157
8.4 Minimal Information Games . . . . . . . . . . . . . . . . . . . . . 160
8.5 Most Normal Games . . . . . . . . . . . . . . . . . . . . . . . . . 164
8.6 Equilibrium Refinements . . . . . . . . . . . . . . . . . . . . . . . 166
8.7 Telecom Network Example . . . . . . . . . . . . . . . . . . . . . . 167
8.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

9 Conclusion 173
9.1 Perfect Information Protocols . . . . . . . . . . . . . . . . . . . . 174
9.2 Imperfect Information Protocols . . . . . . . . . . . . . . . . . . . 174
9.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

Bibliography 179

Index 189

List of Symbols 193

Samenvatting 197

Abstract 199

Curriculum Vitae 201

vii
 Acknowledgments

This dissertation has benefited enormously from comments, suggestions and dis-
cussions with other people, and on these pages I would like to mention as many
of these people as I can remember, in a more or less arbitrary order.
Since I started my PhD by moving to Liverpool, it seems appropriate to start
with the people from this city. I am very grateful to my supervisors, Wiebe van
der Hoek and Mike Wooldridge. A special mention should go to Peter McBurney,
who was not only my advisor but has also encouraged and stimulated me contin-
uously. I would like to thank many colleagues who have helped me to find my
way in Liverpool and in research: Hanno Hildmann, Steve Phelps, Dirk Walther,
Ian Blacoe, Laera Loredana, Rafael Bordini, Carmen Pardavilla and of course
my office mates Benjamin Hirsch, Justin Wang, Qin Xin, Claudia Nalon, Mari
Carmen Fernandez-Gago, Mark Roberts and Nivea de Carvalho Ferreira. I feel
greatly indebted to the many people that contributed to the great atmosphere in
the department of Computer Science, including Thelma Williams, Ken Chan and
the other members of the technical staff, Katie Atkinson, Trevor Bench-Capon,
Aiman Badri, Celia Casado-Castilla, Alison Chorley, Andrea Kam, Christian
Guensel, Catherine Atherton, Shaheen Fatima, Tim Miller and Adele Maggs.
I would like to thank Frank Wolter and Mark Ryan for acting as examiners in
my PhD defense.
In Amsterdam I have benefited from support and encouragement from many
sources, and from a great research atmosphere. First of all I would like to give a
special mention to Johan van Benthem for his critical and stimulating comments,
and for giving me the opportunity to work in the ILLC, and my office mates Aline
Honingh, Merlijn Sevenster and Yoav Seginer. I would like to thank Olivier Roy,
Fenrong Liu, Clemens Kupke, Nick Bezhanishvili, Jelle Zuidema, Reut Tsarfaty,
Maricarmen Martinez Baldares, Balder ten Cate, Ulle Endriss, Joost Joosten,
Chiaki Ohkura, Scott Grimm and Eric Pacuit for many pleasant conversations,
not only during lunch time. I wish to thank Peter van Emde Boas, Marjan
Veldhuizen, Igrid van Loon, Rens Bod, Benedikt Löwe and Robert van Rooij for

ix
their support and comments.
I have always enjoyed presenting my work, so I would like to thank the people
who have given me the opportunity to present my work at their institute: Mathijs
de Weerdt, Vincent van Oostrom, Wouter Teepe, Michael Fischer and Boudewijn
de Bruin.
I am grateful for the financial support that I have received from various exter-
nal organisations, which allowed me to visit summer schools, workshops and con-
ferences. First of all I want to thank AgentLink, the European research network
that has initiated many extremely fruitful meetings and initiatives. Without the
support of the International Joint Conferences on Artificial Intelligence, I could
not have attended the ESSLLI summer school in Nancy. The Association for
Computing Machinery has supported my attendance to the AAMAS conference
in the great city of New York.
The discussions in the logic and games reading group in Amsterdam have been
very helpful, and I would like to thank the members I have not already mentioned
above for their input: Paul Harrenstein, Tijmen Daniëls, Floris Roelofsen, Michael
Franke and Yanjing Wang. I hope this reading group will have a long and fruit-
ful existence. I also feel indebted to Valentin Goranko, Marc Pauly, Mark Jago,
Karl Tuyls, Ronald Cramer, Iyad Rahwan, Tero Tulenheimo, Renze Steenhuisen,
Ernst van Rheenen, Hans van Ditmarsch, Barteld Kooi, Alessio Lomuscio, Fran-
cien Deschesne, John-Jules Meyer and the other people from Utrecht, for the
encouragement and criticisms they have given. Geert Jonker and Stefan Leijnen
should be thanked for their contagious enthusiasm. Finally I would like to give a
very special thanks to Marthélise Vervest for reading my work and all the time
we ave spent together in both Liverpool and Utrecht, and to Rutger van Otterloo
and my parents.

Amsterdam Sieuwert van Otterloo

September, 2005.

x
Chapter 1
Introduction

1.1 Multi-agent Protocols

Multi-agent protocols are sets of rules that specify how agents can interact with
each other. For example, an auction has strict bidding rules and is thus an
example of a protocol. Elections form another example. These activities have in
common that they can be done in real life, without use of computers or networks.
However, one can also imagine auctions and elections in which computer programs
participate, perhaps even in competition with humans. These examples of multi-
agent protocols are already widely used in all kinds of settings. The next examples
illustrate situations in which multi-agent protocols are useful.

• Everyone is familiar with the problem of dividing a cake fairly among several
people. Assume that a round birthday cake is to be shared fairly among
a certain number of guests. If ‘fair’ means into equally sized and shaped
parts and one can use ruler and compass, then this becomes a mathematical
problem. One can also see this situation as a social science problem, by using
another notion of fairness. One can require envy-freeness, which means that
nobody should be envious of somebody else. Everyone should judge his or
her own piece at least as good as the other pieces. This is achievable for
two agents by using so-called ‘cut and choose’ protocols: one agent splits
the cake into two parts, the other agent chooses who gets which part. The
fact that both agents play an active role in this protocol makes it easier
for agents to accept this protocol. What makes the protocol fair is the fact
that the cutter has an incentive to cut as fair as possible. In case of larger
sets of agents, more elaborate procedures exist [17].

• Suppose you want to raise money for a good cause, and a sponsor has given
you a car in order to help you to do so. Should you auction this car, or start
a lottery? A lottery is the traditional way to raise money, at least in The
Netherlands. However, auctions of different types have become increasingly

1
2 Chapter 1. Introduction

popular in other domains (consider eBay, or the distribution of mobile phone

network licenses), and a lot is known about the good theoretical properties
of auctions [63]. Recent work by Goeree and others [39] indicates that
a lottery, despite being random, is a better way to raise money than an
auction. The authors do not only give mathematical arguments. In one
example where parents are asked to donate money to their children’s school,
they note that social arguments also play a role:

“Some parents may be offended when told they contributed noth-

ing because they lost the auction, or, in other words, because their
contributions were not high enough.” [39, p.3]

• On auction websites such as eBay, many buyers prefer to place their bids at
the very last moments. This is called sniping. These snipers choose to bid
in the last minute even when the auction lasts a week [76]. Possibly they
do this in order to avoid bidding wars with other bidders, who in the face of
competition want to spend more money than they originally planned. An-
other explanation is that the snipers somehow enjoy to surprisingly outbid
other people. Either way, some bidders refuse to behave as prescribed by
auction-theory textbooks. To some bidders and sellers sniping seems unfair,
but others feel it is justified by the amount of pleasure they derive from it.

“A lot of people who do not snipe feel it is unfair, but it hap-

pens to be my absolute favorite way to win at auctions.” Marcia
Collier [38]

The need to understand multi-agent protocols is growing, because these proto-

cols are used in new and possibly surprising environments. What worked well
in real life may work differently on the Internet or with computer programs as
participants. The stakes are also getting higher. The Internet is not only used
for buying low cost commodities such as books, but also for flights, cars and
houses. A new research field focused on understanding and designing protocols,
sometimes called ‘social software’ [82], is therefore emerging.

Agents
It is difficult to come up with a universally accepted definition of the word
agent [124]. Nevertheless, every researcher should know the meaning of the words
he or she uses, or risk talking nonsense. I use the word agent to mean a decision
making entity, and thus I accept humans, computer programs or even organisa-
tions as agents. The word has been used in this way within the English language
for centuries, for instance in the following quotation.
1.1. Multi-agent Protocols 3

“Nor are we to be meer instruments moved by the will of those in

authority..but are morall Agents.” Samuel Bolton, 1646 [80]
(original spelling)

In a more recent tradition, agents are seen as software programs with artificial
intelligence-like abilities and properties, such as reflexes, mobility, intelligence
and emotions. The English science-fiction writer Douglas Adams for instance
describes how agents might function when a spaceship tries to recover from a
meteorite hit.

Small modules of software – agents - surged through the logical path-

ways, grouping, consulting, re-grouping. They quickly established
that the ship’s memory, all the way back to its central mission mod-
ule, was in tatters. [2]

I am not concerned with designing or dissecting actual agents, but with proto-
cols for agents. Thus, where other researchers see the construction of agents, or
even ‘intelligent agents’, as a long term, yet unattained research goal, in my view
there are already agents and protocols. The focus is not on the internal work-
ings of these individual agents, but on the way protocols function when used by
agents. Since protocols are formal objects, they can be studied formally, without
experiments or empirical investigations.

Protocols
A protocol is different from an algorithm because it gives agents a choice of
actions. Agents have the freedom to bid whatever they think an item is worth, or
to vote for whatever they think is best. All agents together determine what the
outcome of the protocol is. In an ideal world, the protocol allows all participants
to reach an outcome that is ‘optimal’ in some sense: the protocol should be fair,
efficient, democratic, or otherwise meet some expectation. There are often many
different protocols for a certain problem. One can for instance sell a house by
asking all interested parties to submit a bid in a closed envelop, open all envelops
at the same time, and sell the house to the highest bidder. Alternatively, one
could have an open-cry auction, in which bidding continues until no agent wants
to bid higher than the current bid. Another option would be to have a lottery,
or an essay contest. It is often not immediately obvious which protocol is best.
Selecting the best protocol for a certain task is thus a relevant and sometimes
difficult problem.

Traditional versus Computational Approaches

What we call multi-agent protocols has been already studied under different
names by other disciplines than computer science. For example, economists and
4 Chapter 1. Introduction

game theorists have studied the properties of auctions [63]. Social choice scien-
tists have been working on voting protocols and fair division protocols [17]. These
related fields also have their own traditional applications. Fair division protocols
are can be used for cake cutting, but also for divorce settlements [17]. Economists’
examples are often more trade-oriented or money-oriented. Thus, an opportunity
exists here to take results and insights from computer science, and use them to
get improvements in applications outside the traditional scope of computer sci-
ence. We hope that knowledge about multi-agent protocols can be used to design
better solutions for the example scenarios described in this chapter.
One can distinguish ‘traditional’ approaches from computer science (and AI)
approaches by the fact that the computer scientists emphasise computational
properties. In the traditional approach one determines whether some solution
to a protocol problem exists. The computer scientists are also interested in the
question whether something can be computed efficiently. This emphasis on com-
putation can lead to interesting insights. Even though it has been proven that
no voting scheme is completely immune to manipulation [36], one can show that
in certain schemes it is very hard to compute how one should manipulate the
voting in order to get a required outcome [24]. In these schemes, it is unlikely
that someone can manipulate an election.
In this dissertation computational arguments are also used, but in a different
way. By comparing the complexity of different protocol verification problems,
one can determine what kind of goals are hard for agents to achieve, and which
properties are hard to verify. This leads to more insight into the causes of the
difficulty of protocol verification of design. For instance it is often assumed that
the interaction between agents makes games and protocols difficult. Sometimes
however also models with only one agent can have interesting computational
properties, which is a surprising result.

Logical Approach
In the examples of cake-cutting and charity auctions, it has become clear that the
properties that one wants protocols to have can be very diverse. In the second
example, parents actually wanted to contribute, while in the first example fairness
meant that agents would not choose to swap pieces. In some cases, for example
money-based auctions, one can reduce the quest for the right protocol to a nu-
merical problem: the problem is reduced to computing the optimal parameters,
or finding the right side payments. If this is not possible, for instance because
money is not available in the protocol, one must capture these properties in some
other precise way, before one can test protocols for these properties. Logical lan-
guages are very suitable for this task: one can describe complicated properties
in short logical formulas. Logics are also very expressive: one can state both the
presence and the absence of certain properties. Different logics are thus used as
specification languages: the formulas express what one wants or does not want
1.2. Protocol Problems 5

from a protocol.

1.2 Protocol Problems

Throughout the dissertation different example problems are used to illustrate the
issues at hand. For instance chapter 4 contains several protocols that can be used
in the following situation.
Three agents Alice, Bob and Caroline (or A, B and C) have to
select one of the alternatives x, y and z. They are looking for a
suitable voting protocol to select exactly one of these three alterna-
tives as the outcome. The protocol should be democratic, so that
any majority can enforce any outcome.
In chapter 4, the focus is on what outcomes can be guaranteed by agents and
coalitions of agents. Thus, the issue of effectivity is studied. In this chapter, we
find many solutions for this problem. The logical approach of chapter 4 cannot
be used for distinguishing these many solutions. Therefore, in chapter 5 and 6
more expressive logics are studied that can find subtle differences between the
different solution to this problem.
The following two problems are more basic than the voting problem stated
above, and using the logic from chapter 4 one can again find protocols that solve
these problems. In chapters 4 and 5 it is shown how solutions for these problems
differ from each other.
joint decision problem A decision p is taken if either Alice or Bob think that
p should be the case. If both agents do not want p, it should be rejected.
independent decision problem Alice can decide whether a should hold or
not, and Bob can decide whether b should hold or not.
Chapter 7 is concerned with the knowledge that agents have at the end of a
protocol, and how this knowledge depends on the strategies that are used. One
example problem that can be analysed using the techniques from this chapter is
the following problem.
In a TV quiz show the quiz master asks a candidate the following
question: Which day of the week comes directly after Tuesday? Is it
a) Monday, b) Wednesday, c) Friday or d) Saturday. The candidate
has no clue whatsoever about the days of the week, and replies: ‘I am
not sure. Can I do fifty-fifty?’. The quiz master has to remove two
options that are not the answer, so he says: ‘The answer is not Monday
and neither Friday’. Does the candidate now know the answer?
In chapter 8, agents are concerned about their privacy, and use random strate-
gies in order to hide their preferences.
6 Chapter 1. Introduction

Alice needs to buy one box of breakfast cereals every week. Each week
she can either buy Allgrain (A), Barley (B) or Cornflakes (C). She
likes A better than B and B better than C. However, Alice knows
that the shop is watching her shopping behaviour closely, and she
does not want the shop to know for sure what her preferences are.
Therefore, she buys a different brand every day.

The techniques presented in this chapter allow one to calculate the optimal (ran-
dom) strategies that agents should use if they are concerned about keeping their
preferences private.

1.3 Outline of this Dissertation

This first chapter is a formula-free introduction. The last chapter is also written
in plain English, and presents some conclusions. The chapters in the middle have
a high density of mathematical notation. The first two ‘middle’ chapters are
introductory.

• Chapter 2 contains definitions in the area of logic. It contains definitions

of propositional logic, which is the basic logic of which all other logics are
extensions, modal logic and epistemic logic.

• Chapter 3 is a concise introduction and overview of game theory. It defines

the concepts of game theory that are used later on.

The remainder of this dissertation, the so-called ‘content’-part, describes original

research that I have conducted over the last three years. It can be divided roughly
in two parts. The first three chapters deal with logics that are interpreted over
extensive games of perfect information. In these games it is assumed that agents
have perfect information about the current state of the world. They know what
other agents have done, but are uncertain about what other agents will do. In
order to express properties of such situations, different logics are examined.

• Chapter 4 describes a logic for reasoning about extensive games called

efl. This logic deals with reasoning about whether coalitions of agents can
achieve certain goals, without help of the other agents. Game-theoretically
this is a simple situation, and one can therefore efficiently check properties
in this logic. A complete proof system is also given, together with a proce-
dure to automatically constructs protocols for given properties. The main
result of this chapter has been accepted for ESSLLI 2005 [107].

• Chapter 5 introduces logics that are more expressive than efl. First of all
the logic efls can be used to express more subtle properties involving side
effects of using certain strategies. One way to look at this logic is by saying
1.3. Outline of this Dissertation 7

that it deals with the situation where agents are initially not aware of the
preferences of other agents, a situation that is not normally considered in
game theory. This logic can thus be used to express more properties of
game forms, but has the same model checking complexity as efl.
A second logic introduced in this chapter, called efls, allows one to reason
about agents that want other agents to be able to do something. This is an
example of reasoning about preferences on the whole play of the game, not
just on the outcome. One can apply this logic to reason about polite agents
that want to give other agents the ability to choose. This chapter extends
work presented at the AAMAS 2004 conference in New York [113].

• Chapter 6 reasons about game forms and preferences explicitly. It uses a

special logic for reasoning about preferences. As an example we study the
backward induction solution concept in this chapter, in which agents use
their knowledge of each others preferences in order to anticipate each others
choices. This chapter is exceptional because, unlike the other chapters of
this thesis, it is not the sole work of Sieuwert van Otterloo, but is based on
joint, yet unpublished, work done in pleasant cooperation with Olivier Roy
and Johan van Benthem at the ILLC in Amsterdam.

These three chapter can be seen as an attempt to understand game forms using
more and more precise languages. For the logic efl many protocols appear to
be the same. The next two chapters offer logics that give more detailed views, so
that one can discover subtle differences in protocols.
Chapters 7 and 8 deal with the case where agents are not fully aware of all
aspects of the current situation. They have imperfect information about certain
facts. Since information is very important to agents, they might act in order to
get more information. In other situations agents act so that others obtain no
information.

• Chapter 7 discusses knowledge condition games. In this new type of games,

agents act in order to achieve a certain knowledge situation: they want to
know that others do not know that something happened. Two variants with
different computational complexity are introduced, and some tractable vari-
ants are described. The work has been presented incrementally at GTDT
in New York [110], at the first Knowledge and Games Workshop in Liver-
pool [111] and at the European workshop on Multi-Agent Systems [115], and
have finally been accepted for publication in the Journal of Logic, Language
and Information [114].

• In chapter 8, a similar problem is treated in a different way. We assume

that certain agents want to keep their preferences secret. Using techniques
from information theory, we determine what strategy agents must use in
8 Chapter 1. Introduction

order to maximize the uncertainty of an observer. This chapter is based on

a paper presented at the AAMAS 05 conference in Utrecht [108].

A recurrent theme is this dissertation is the idea that agents can have complex
goals in a protocol, and that nondeterministic strategies can be used for achieving
these goals(see for instance the situation on page 89 where Bob makes Alice unable
to decide, or the quiz master example on page 134 where nature does not favour
the candidate). Normally in game theory agents have preferences over outcomes,
and the goal is to achieve a certain outcome. A complex goal on the other
hand depends on the whole game, including properties of the strategies used and
other outcomes than the actual one. For instance in chapter 5 agents care about
whether other agents can achieve outcomes or not. In knowledge condition games
coalitions act in order to make sure certain knowledge is achieved in the end, and
in chapter 8 agents care about how predictable their strategy is.
It is easiest to use strategies that recommend single best actions for any sit-
uation. These strategies are called pure strategies and are often sufficient for
reaching simple goals. In this dissertation we often use nondeterministic strate-
gies. These strategies can recommend multiple actions and are thus potentially
more powerful. In knowledge condition games and the privacy games of chapter 8,
agents can deliberately use these strategies to become unpredictable. In the logics
efls and efln these nondeterministic strategies are used because in many case
several actions are equally good. One cannot know beforehand which action an
agent will take in this case, so we model this uncertainty using nondeterministic
strategies.

1.4 Conclusion
Multi-agent protocols have not been discovered recently. The term can be used
to describe common situations, such as auctions, voting and cake cutting. These
protocols can be studied from different disciplines, such as game theory, eco-
nomics and social science. Furthermore one can test these protocols for many
different properties, for instance envy-freeness. The different frameworks and
logical languages defined in this dissertation make it possible to formally analyse
these multi-agent protocols, and to test them on many different properties. The
precision of a logical approach makes it possible in principle to use the computer
to find the right protocol for any situation. In this dissertation, it is determined
in which cases this is also practical. This is done by looking at the complexity of
these computing problems.
Chapter 2
Logic

2.1 Introduction
Logic is one of the oldest disciplines of science. It has been studied more or less
continuously from Aristotle to the present day. For example, it was an important
part of the Medieval academic curriculum: together with grammar and rhetoric,
logic formed the ‘trivium’, the relatively simple arts that one should master before
one could move on to the more advanced arts of the Quadrivium (arithmetic,
astronomy, geometry, and music) [122].
Given the rich history of logic, it is not possible to give a complete overview
of the area. The goal of this chapter is merely to provide the necessary defini-
tions of propositional, modal and epistemic logic that will subsequently be used
throughout this dissertation. For readers not so familiar with logic, this chapter
can serve as a concise introduction. For other readers, this chapter introduces
the notational conventions that I use in the remainder of this dissertation. First
propositional logic is defined. Then in section 2.3 modal logic and epistemic logic
are defined. In section 2.4 we discuss the different ways in which theorem proving,
satisfiability and model checking can be used for protocol verification.
A logic typically consists of three elements: the logical language, the semantics
and the proof system. A logical language is a set L of formulas. The semantics
is a relation between formulas and models, that says when a formula is true on
a model. If a formula φ is true on a model M we write M |= φ, otherwise we
write M 6|= φ. We are often interested in formulas that are true in any model,
and these formulas are called valid formulas, validities or tautologies. In order to
indicate that φ is a tautology, we write |= φ. If a formula φ holds in at least one
model, the formula is called satisfiable.
The final typical element of a logic is the proof system. Such a proof system S
consists of axioms and derivation rules and allows one to formally derive formulas.
2.1.1. Definition. Let L be a logical language. A proof system S is a pair
(A, R) where A ⊆ L and R ⊆ L∗ .

9
10 Chapter 2. Logic

description language proof system

propositional logic Lp Sp
standard modal logic L2 S2
epistemic logic LK SK

Figure 2.1: Proof Systems for different logics

The set A is called the set of axioms of S, and R is the set of reasoning rules. If a
proof system S proofs a formula φ, then we write S`φ. The notion of proof that
we use here is that of a finite list of statements S`φ1 , S`φ2 , . . . S`φn such that
each formula φi is either an axiom of S, or (φ( m1 ), . . . , φ( mn ), φi ) is a reasoning
rule of S, with m1 , . . . , mn < i. Thus, axioms count as self-evident, and the
reasoning rules allow one to derive a formula from formulas proven before.

2.1.2. Definition. Let L be a a logical language. A proof system S is sound if

S`φ implies |= φ. It is complete if |= φ implies S`φ.

All logics in this thesis make use of a symbol ¬ for negation of a formula. In such
logics, a formula φ is called consistent if its negation cannot be proven: S 6 `¬φ.
In a complete proof system (for a logic where negation is defined in the classical
way), every consistent formula is satisfiable.
Ideally, a proof system should be sound and complete. Furthermore, the
axioms and rules should not be arbitrary sets, but one should be enumerable
in an automatic fashion: a mechanical procedure should be able to generate all
axioms. In practice, this means that the sets of axioms and reasoning rules consist
of a finite number of patterns, so that any formulas can be inserted in the open
places of the pattern. This constraint ensures that one can effectively generate
all proofs, which means that there is a procedure to find all proofs and thus all
theorems.
Table 2.1 lists the languages and proof systems that are defined in this chapter.

2.2 Propositional Logic

Propositional logic is a logic for reasoning on a sentence level: It explains how
complex sentences follow from simple sentences. We assume that there is a set P
of basic or atomic propositions. These represent sentences that cannot be broken
down in smaller sentences. In the next example this set contains the atomic
propositions p and q, that capture ‘It rains’ and ‘The weather is good’. These
atomic propositions are combined using logical connectives or operators, which
stand for some semantical relation between facts.
2.2. Propositional Logic 11

sentence proposition
It rains p
It does not rain ¬p
The weather is good q
It rains and the weather is good p∧q
It rains or the weather is good p∨q
If it rains then the weather is good p→q
It rains if and only if the weather is good p↔q
It rains if and only if the weather is not good p∇q
Contradiction ⊥

It is convenient to have so many operators available, so that one can concisely

and naturally express complex formula structures. At the same time, it is cum-
bersome to deal with all the different operators in all theorems and proofs. It is
also redundant, because many operators can be expressed in terms of each other.
For instance, p ↔ q is equivalent to (p → q) ∧ (q → p). The common solution
to this dilemma is to treat some of these connectives as fundamental, and others
as abbreviations for something expressed using the fundamental connectives. In
the next subsection, a variant of propositional logic with a minimal set of fun-
damental operators is presented, and a proof system for this logic is developed.
In the following subsection, it is shown in detail how propositional logic with all
operators reduces to this language.

2.2.1 Minimal Propositional Logic

The language of minimal propositional logic has a set of basic operators that
is minimal in the following sense: Every function from truth values to a truth
value can be expressed by composing the basic operators, but none of the basic
operators can be expressed as a composition of the other basic operators. One
can choose such a minimal set in different ways. Two well-known minimal sets are
{∨, ¬} and {∧, ¬}, see for instance [53, p.71]. Our approach is based on the basic
operators ⊥ (the constant ‘false’ that is never true) and → (implication). An
argument for this particular choice would be that implication plays an important
role in the proof system defined for modal logic.

2.2.1. Definition. Let P be a set of atomic propositions and p ∈ P an element

of P . Minimal propositional logic Lp (P ) consists of formulas φ generated by the
grammar
φ ::= p | φ → φ | ⊥

Parentheses indicate how certain formulas are constructed, and can be used,
for instance, to make a distinction between (p → q) → r and p → (q → r). If no
parentheses are given then the second reading is intended: p → q → r should be
read as p → (q → r).
12 Chapter 2. Logic

This language is interpreted in the following way. A model M for this logic
is a subset of P . The atomic propositions in M are assumed to be true, the ones
that are not in M are false. The next definition determines when M |= φ for any
formula φ.
2.2.2. Definition. Let M ⊆ P be a model. The satisfaction relation |= for
minimal propositional logic is defined recursively by the following three rules:
M |= ⊥ never
M |= p where p ∈ P iff p ∈ M
M |= φ → ψ iff M |= φ implies M |= ψ
Like many logics, propositional logic is closed under uniform substitution.
This means that if one has a valid formula in which p occurs, and one replaces
all occurrences of p for any other formula φ, one again has a valid formula. For
example, since φ = p ∨ ¬p is valid, the formula ψ = ¬q ∨ ¬¬q is also valid. A
formula ψ that is obtained from φ by uniform substitution is called an instance
of φ.
In the remainder of this section we define a proof system Sp for the language
Lp . The next three formulas serve as the axioms for this proof system.
A1 = φ → (ψ → φ)
A2 = (φ → (ψ → ξ)) → ((φ → ψ) → (φ → ξ))
A3 = ((φ → ⊥) → (ψ → ⊥)) → (ψ → φ)
We have defined axioms as sets of formulas, and thus A1 to A3 are sets of formulas.
To be precise, A1 = {φ → (ψ → φ)|φ, ψ ∈ Lp }, but it is hoped that the notation
without set brackets is more readable. We can write φ ∈ Ai to indicate that φ
has the stated form. If φ ∈ Ai we say that φ is an instance of the axiom scheme
Ai .
Suppose that L is a logical language in which → has its usual interpretation.
If both φ and φ → ψ are validities in this logic, then ψ must be a validity as well.
This fact forms the basis of the reasoning rule Modus Ponens. The set M PL that
expresses this rule is the following.
M PL = {(φ, φ → ψ, ψ)|φ, ψ ∈ L}
A more traditional way of presenting this rule is the following.
φ φ→ψ
M PL =
ψ
2.2.3. Definition. The standard proof system Sp for minimal propositional
logic consists of the three axioms A1 , A2 , A3 and the rule Modus Ponens.
The system Sp is sound and complete for minimal propositional logic. A proof
of this claim is beyond the scope of this dissertation, but proofs for similar systems
can be found in logic textbooks, for instance [53].
2.2. Propositional Logic 13

2.2.2 Full Propositional Logic

Logical connectives can be seen as functions that take as input a number of truth
values, and return a truth value. There are two truth values, and thus two cor-
responding truth constants: ⊥ (false) and > (true). There are two one-place
functions, namely the identity, which does not have a connective, and negation,
for which the notation ¬ is used. A simple counting argument can be used to
show that there are 24 = 16 different two-argument functions. Only a few of
these are commonly used as connectives, namely ∧ (and), ∨ (or), → (implica-
tion), ↔ (double implication) and ∇ (exclusive or). In this section, a version of
propositional logic based on these connectives is presented. This logic is called
full propositional logic.
A formula of the form ¬φ is called a negation. Similarly we call φ ∨ ψ a
disjunction, φ ∧ ψ a conjunction, φ ∇ ψ an exclusive disjunction, φ → ψ an
implication and φ ↔ ψ a double implication. The two constants > and ⊥ can
be called verum and falsum. Negation is assumed to be the strongest binding
connective, so that ¬q∧r is the same formula as (¬q)∧r. For all other connectives,
operators that appear further to the right in the expression bind stronger. Thus,
p ∧ q ∨ r is the same formula as p ∧ (q ∨ r).

2.2.4. Definition. Let P be a set of atomic propositions and p ∈ P an element

of P . Full propositional logic Lfp (P ) consists of formulas φ generated by the rule

φ ::= p | ⊥ | > | φ → φ | φ ↔ φ | φ ∨ φ | φ ∧ φ | φ ∇ φ

This full language is interpreted in the following way. The model M is again
a subset of the set of all propositions P .

M |= > always
M |= ⊥ never
M |= p where p ∈ P iff p∈M
M |= ¬φ iff not M |= φ
M |= φ ∨ ψ iff M |= φ or M |= ψ (or both)
M |= φ ∧ ψ iff M |= φ and M |= ψ
M |= φ ∇ ψ iff M |= φ or M |= ψ but not both
M |= φ → ψ iff M |= φ implies M |= ψ
M |= φ ↔ ψ iff M |= φ and M |= ψ or neither

It is not hard to show that under this interpretation the following formulas hold
14 Chapter 2. Logic

on any model M .

M |= > ↔ (⊥ → ⊥)
M |= ¬φ ↔ (φ → ⊥)
M |= (φ ∨ ψ) ↔ ((φ → ⊥) → ψ)
M |= (φ ∧ ψ) ↔ ((φ → ψ → ⊥) → ⊥)
M |= (φ ∇ ψ) ↔ ((φ → ψ) → (ψ → φ)) → ⊥)
M |= (φ ↔ ψ) ↔ ((φ → ψ) → (ψ → φ) → ⊥) → ⊥)

One can thus define all other operators in terms of the two connectives ⊥ and →.
Conjunction and disjunction have as a feature that changing the order and
nesting of these operators does not change their truth value: φ ∨ ψ is equivalent
to ψ ∨ φ, and (φ ∨ ψ) ∨ χ is equivalent to φ ∨ (ψ ∨ χ). These properties make it
possible to apply these operators to finite sets. Thus, we define a disjunction of
a set by _
{φ0 , φ1 , . . . , φn } = φ0 ∨ φ1 ∨ . . . ∨ φn
Similarly we define the conjunction of a set as
^
{φ0 , φ1 , . . . , φn } = φ0 ∧ φ1 ∧ . . . ∧ φn

A useful property of these operators is that any propositional logic formula

is equivalent to a conjunction of disjunctions of possibly negated atomic proposi-
tions.

2.2.5.VDefinition.
W A formula φ is in conjunctive normal form iff it has the form
φ = i j ψij , where ψij is of the form ψij = ¬a or ψij = a, for some atomic
proposition a ∈ P .

2.2.6.WDefinition.
V A formula φ is in disjunctive normal form iff it has the form
φ = i j ψij , where ψij is of the form ψij = ¬a or ψij = a, for some atomic
proposition a ∈ P .

Two example propositional logic formulas, one in conjunctive and one in dis-
junctive normal form, are the following.

Conjunctive normal form (p ∨ q) ∧ (¬p ∨ ¬q)

Disjunctive normal form (p ∧ ¬q) ∨ (¬p ∧ q)

For logics other than propositional logic, one can also define the notions of
conjunctive and disjunctive normal form in a similar way. For instance for modal
logic (defined in
V the
W next section), a formula is conjunctive form is a formula of
the form φ = i j ψij where ψij is either a or ¬a, for some formula a that is
either an atomic proposition, or of the form 2X φ.
2.3. Modal Logic 15

2.3 Modal Logic

Modal logic can be seen as an extension of propositional logic with operators for
expressing ‘modal’ concepts, such as provability, knowledge or belief. It has a
long history (see [12, pp. 37–48] for a brief exposition) and many applications in
logic, mathematics, computer science and artificial intelligence. It originates in
the study of necessity. Consider the following pair of statements:
If it does not rain then the weather is good (¬p → q)
If it rains then the weather is not good (p → ¬q)
At my time and place of writing, it rains and the weather is not good. Therefore,
both these sentences are true in my current situation. We assume that people do
not like rain, so rain is not called good weather. Many philosophers feel that, given
this assumption, a sentence such as the first is true merely by accident, whereas
the second sentence is necessarily true. In order to express this difference a new
symbol is needed. Here, we use the symbol 2 in front of a formula in order to
express necessity. Thus, if M expresses the current state of the world, and w my
current time and place, then the following hold.
M, w |= (¬p → q)
M, w |= 2(p → ¬q)
The dual of the box 2 is the diamond 3. It expresses that something is possible,
and can be defined as 3φ = ¬2¬φ. One can for instance say that it is possible
that it rains and the weather is not good (3(¬p ∧ ¬q)), but that it is impossible
that it rains and the weather is good at the same time ¬3(p ∧ q). One can define
modal logic formally in the following way. The set P is again a set of atomic
propositions, and the set ∆ contains the different modalities that we allow. Thus,
if ∆ contains only one element, we get basic modal logic. If ∆ contains multiple
elements, we get a multi-modal logic with multiple different modal operators.
2.3.1. Definition. Suppose the finite sets ∆ and P are given, and let X ∈ ∆
and p ∈ P be typical elements. Multi-modal logic L2 (P ) consists of formulas φ
generated by the rule
φ ::= p | 2X φ | φ → φ | ⊥
If the set of modalities ∆ is a singleton ∆ = {X} then the subscript X can be
omitted and we have single-agent modal logic. The notation 3X φ is used as a
shorthand for ¬2X ¬φ.
It took a while before logicians discovered a good way to interpret the new
operators. One of the reasons is that one can read the operator in different ways.
The next table shows a provability reading, a temporal reading, ethical reading,
doxastic reading and an epistemic reading. For many readings an alternative
notation is sometimes used, so that one can mix these different readings without
chance of confusion.
16 Chapter 2. Logic

meaning notation reference

φ can be proven 2φ [16]
φ is always true in the future 2φ page 24
φ ought to be true 2φ or Oφ [70]
A believes φ 2A φ or BA φ [71]
A knows φ 2A φ or KA φ [32]
In the last two examples, the logical language contains multiple modal operators,
one for each agent. These logics are thus multi-modal logics, whereas in the other
examples we have single-agent modal logic.
A general semantics for modal logics was finally found around 1960, and is for
a large part due to Saul Kripke, and is therefore called Kripke semantics [12].

2.3.2. Definition. A Kripke model M is a tuple M = (∆, W, {RX }X∈∆ , P, π),

where ∆ is a set of agents, W is a set of worlds, {RX }X∈∆ is a collection of binary
accessibility relations RX between worlds, one for each modality X ∈ ∆, P is a
set of atomic propositions and π is a function π : W → 2P .

The function π is typically called an interpretation function. The statement 2X φ

is interpreted as saying that φ is true in all possible worlds. This semantics
is therefore called the possible world semantics. Which worlds are possible is
determined by the accessibility relation RX .

2.3.3. Definition. Suppose that M = (∆, W, {R}∆ , P, π) is a Kripke model,

w ∈ W , p ∈ P and X ∈ ∆.
M, w |= p iff p ∈ π(w)
M, w |= ⊥ never
M, w |= φ → ψ iff M, w |= φ implies M, w |= ψ
M, w |= 2X φ iff ∀v : (w, v) ∈ RX ⇒ M, v |= φ
Different operators 2X can have different accessibility relations RX and thus
satisfy different properties. There are some properties that hold for all modal
logics. Other formulas are only true under some readings of modal logic. One
important principle is that of necessitation.

2.3.4. Lemma. Let 2X φ ∈ L2 . If φ is valid, then 2X φ is valid. [12]

It is perhaps interesting to remark that this rule preserves logical truth, but not
actual truth. Thus, it is not the case that if φ is true in a situation, then 2X φ is
true in that situation. The theorem only claims that validity is preserved, which
is a weaker statement. This lemma can therefore not be used to introduce an
axiom along the lines of φ → 2X φ, but it can be turned into a reasoning rule.
This rule is called Necessitation.

N ecL = {(φ, 2X φ)|2X φ ∈ L}

2.3. Modal Logic 17

Or expressed in the more traditional format:

φ
N ecL =
2X φ
Another principle of any normal modal logic is distribution of the box oper-
ator. From the truth of 2X (p → q) one can derive 2X p → 2X q. This can be
used to formulate the axiom Distribution or K.
K = 2X φ → 2X (φ → ψ) → 2X ψ
An important question is of course whether one can formulate a proof system for
modal logic. In such a proof system one could reuse all axioms of propositional
logic, but this is not normally done. It is more convenient to assume familiarity
with propositional logic, and to allow any propositional logic tautology as an
axiom. Thus, the following is an axiom in our proof system for modal logic.
prop = τ where τ is an instance of a propositional logic tautology
This axiom allows one to substitute any number of atomic propositions by arbi-
trary modal logic formulas. One cannot only use this rule to derive the tautology
p → p, but also to derive 2X p → 2X p.
One can of course question the legitimacy of this axiom. Is it not too easy
to allow any tautology in a proof? Does this not lead to uncheckable proofs?
The answer is ‘no’. One can test whether a propositional logic formula is valid
(and thus provable) by checking all different models: there is a finite number of
atomic propositions in any propositional logic formula, and thus a finite number
of models. Furthermore one can convert proofs that use this axiom by replacing
each usage of this axiom by the corresponding Sp proof. Thus, at least in theory,
such an axiom can be allowed.
2.3.5. Definition. Let L2 be the language of modal logic. The proof system S2
for this language has prop and K as axioms, and Modus Ponens and Necessitation
as reasoning rules.
2.3.6. Theorem. Suppose that φ ∈ L2 is valid. Then S2 `φ.
This property is called weak completeness, and a proof of some version of this
theorem can be found in most modal logic books, for instance the one by Black-
burn et al [12, p.194] or Meyer and Van der Hoek [71, p.18]. Below we sketch the
general idea, which uses maximally consistent sets.
2.3.7. Definition. Suppose a logic is given with language L and with a proof
system S that uses Modus Ponens as a reasoning rule (and possibly other rules as
well). A set of formulas S ⊆ L is maximally consistent if the following conditions
are all met: ⊥ ∈ / S, all instances of axioms φ ∈ A are in S, if φ, φ → ψ ∈ S then
ψ ∈ S, and for any formula φ either φ ∈ S or ¬φ ∈ S.
18 Chapter 2. Logic

For each consistent formula φ one can find a maximally consistent set S so
that φ ∈ S [12]. If the proof system S is sound, then for any pointed model M, w,
the set {φ | M, w |= φ} is maximally consistent.
In order to prove the theorem, consider a consistent formula φ with atomic
propositions from the set P . It is necessary to show that there is a model
M such that M, w0 |= φ. Such a model W = (Σ, W, R, P, π) can be con-
structed using maximally consistent sets as the worlds as the model: W =
{w|w is a maximally consistent set}. The relation RX is then defined such that
(v, w) ∈ RX if 3X φ ∈ v for all φ ∈ w, and π(w) = {p ∈ P |p ∈ w}. For w0 one
can take any maximally consistent set that contains φ. This construction defines
one large model M called the canonical model for a logic, such that any consis-
tent formula holds in some world of this model. This technique to use maximally
consistent sets as possible worlds not only works for plain modal logic, but can
often be adapted for modal logics with a different interpretation [12, p.194].
The multi-modal logic presented here is the weakest possible version of a
modal logic with a possible worlds semantics. The next table lists some formulas
that one might expect to hold for certain readings of the 2 operator, but that
cannot be proven in S2 .

4 2X φ → 2X 2X φ What is necessary (for X) is necessarily necessary

D 2X φ → ¬2X ¬φ What is necessary (for X) is possible
T 2X φ → φ What is known to X is true

The names 4, D and T are the standard names for these axioms [12, p.192]. One
can make these formulas valid by putting constraints on the relations RX . This
idea is used in order to get an epistemic reading in the next section.
It is often important to determine whether two models satisfy the same for-
mulas, and for this purpose the notion of bisimulation can be used. Two Kripke
models are bisimilar if one can find for any world in one model a corresponding
world in the other model. Such a relation between worlds is called a bisimula-
tion. A formal definition of this concept for single-agent modal logic would be
the following.

2.3.8. Definition. Let M1 = ({X}, W1 , R1 , P, π1 ) and M2 = ({X}, W2 , R2 , P, π1 )

be single-agent Kripke models. A non-empty relation S ⊆ W1 × W2 is a bisimu-
lation if the following conditions hold

• If (w1 , w2 ) ∈ S then π(w1 ) = π(w2 )

• If (v1 , w1 ) ∈ R1 and (v1 , v2 ) ∈ S then there is a world w2 such that (v2 , w2 ) ∈

R2 and (w1 , w2 ) ∈ S

• If (v2 , w2 ) ∈ R2 and (v1 , v2 ) ∈ S then there is a world w1 such that (v1 , w1 ) ∈

R1 and (w1 , w2 ) ∈ S
2.3. Modal Logic 19

For logic with more than one agent one has to use a set of relations SX as bisim-
ulation. For simplicity reasons this extension has been omitted.
A bisimulation matches worlds that behave similarly with respect to modal
logic formulas. Thus, if M1 , w1 |= φ and (w1 , w2 ) ∈ S then also M2 , w2 |= φ.
In order to decide whether two models are equivalent one thus has to find a
bisimulation between the models, or prove that no such relation exists.

2.3.1 Epistemic Logic

Epistemic logic is an extension of propositional logic with operators that express
that a proposition is known. It originates from philosophy [48], but has found
applications in computer science and artificial intelligence [71, 32]. It is one of
the best known modal logics.
In epistemic logic, the operator 2 is usually written K. In the case of multiple
operators these are written KX instead of 2X . The X indicates which agent’s
knowledge one is talking about. Thus, Kφ denotes that φ is known in the single
agent case, and KA φ means that A knows φ. Instead of 3 we use M . The
operators M and MX express that something is considered possible, the dual of
knowledge.
The following statements about knowledge make use of these operators.
It rains p
Alice know it rains KA p
Bob does not know that it rains ¬KB p
Bob thinks it is possible that it rains MB p
Alice or Bob knows that it rains K A p ∨ KB p
Bob does not know that Alice knows that it rains ¬KB KA p
Alice knows that she thinks it is possible that it rains K A MA p
2.3.9. Definition. Suppose the finite sets Σ and P are given, and let X ∈ Σ and
p ∈ P be typical elements. Epistemic logic LK consists of formulas φ generated
by the rule
φ ::= p | KX φ | φ → φ | ⊥
For epistemic logic we identify each modality with an agent. Since the notation
Σ is used for the set of all agents in this dissertation, we here use Σ for the set of
modalities, rather than ∆ as we did on page 15.
A relation R is an equivalence relation over W if for any worlds v, w, x ∈ W it
is the case that (v, v) ∈ R (reflexivity), if (v, w) ∈ R then (w, v) ∈ R (symmetry)
and if (v, w) ∈ R and (w, x) ∈ R then (v, x) ∈ R (transitivity). If a relation is an
equivalence relation we often use the symbol ∼ for this relation. Furthermore we
write w ∼X v instead of (w, v) ∈ ∼X .
2.3.10. Definition. A Kripke model M = (Σ, W, {RX }X∈Σ , P, π) is an epis-
temic model if each relation RX is an equivalence relation over W .
20 Chapter 2. Logic

The next table lists some examples of formulas that hold over epistemic models
M.
|= KX φ → φ Truth
|= KX φ → KX KX φ Positive Introspection
|= ¬KX φ → KX ¬KX φ Negative Introspection
These validities can be used as axioms in a reasoning system for epistemic logic.

T = KX φ → φ
4 = K X φ → K X KX φ
5 = ¬KX φ → KX ¬KX φ

For epistemic logic one also has a proof system.

2.3.11. Definition. The proof system SK for LK is defined as

(prop ∪ K ∪ 4 ∪ 5 ∪ T, M P ∪ N ec)

This proof system is again sound and complete with respect to the given semantics
based on epistemic models. A proof can be found in epistemic or modal logic text
books such as [12, p.194]. The common name for this proof system is S5 or S5n
where n is the number of agents. This semantics is widely used because it is simple
and seems realistic for rational agents, but has also received criticism. First of all
there is the omniscience problem: all agents know all tautologies, so every agent
knows all mathematical theorems. This seems unrealistic in the case of human
agents, or artificial agents with a limited computing capacity. There have been
attempts to model knowledge while avoiding omniscience [32]. A second potential
objection is that it is not certain that humans have full introspection over all their
knowledge. Humans do not always know what they know, and especially it seems
doubtful that they have negative introspection. This is for instance stated in
Rumsfeld’s famous remark:
“There are known knowns. These are things we know that we know.
There are known unknowns. That is to say, there are things that
we know we don’t know. But there are also unknown unknowns.
There are things we don’t know we don’t know.” (D. Rumsfeld, USA
secretary of defense, Feb. 12, 2002)[92]
Philosophically there are many arguments against introspection [123]. In the
context of protocols, with a finite number of states and possibilities, it does
not seem to be a problem. It is not unreasonable to assume that agents have
introspection over a small, well-known domain such as the possible outcomes of
a protocol. It does not follow from this assumption that agents know everything
about the whole world, or are completely aware of omissions on their knowledge
in general.
2.3. Modal Logic 21

pq p
A
B B
A
q
Figure 2.2: Rain in Liverpool and Amsterdam: Model M1

2.3.2 Common Knowledge

If one reasons about the knowledge of multiple agents, it is natural to consider
cases where agents collectively know something. In order to make this possible
several notions of group knowledge have been defined. The first of these notions
allows one to say that ‘Everbody knows . . . ’ and is denoted Eφ. This notion can
be defined by means of a conjunction over all agents.
^
M, w |= Eφ ⇔ M, w |= KX φ
X∈Σ

This operator can also be defined using an accessibility relation, like

S the K oper-
ators. Define RE as the union of all single agent relations: RE = X∈Σ ∼X . The
everybody knows operator can be interpreted in the following way.
M, w |= Eφ iff ∀v : (w, v) ∈ RE implies M, v |= φ
The relation RE is not transitive, and thus it is not an equivalence relation.
Therefore, the principles of positive and negative introspection do not hold for
‘Everybody knows’. The formula Eφ → EEφ is not valid, and a counterexample
is presented in figure 2.2. This illustrated model M1 has two agents, A and B.
A is in Liverpool, and can thus observe the weather in Liverpool, and B is in
Amsterdam, and can see the weather in Amsterdam. The atomic proposition p
indicates that it rains in Liverpool, and q that it rains in Amsterdam. Suppose
that in the actual situation s it rains in both Amsterdam and Liverpool. The
following formulas hold.
M1 , s |= KA p ∧ KB q A and B know that it rains in their city
M1 , s |= KA (p ∨ q) A knows that it rains in Liverpool or Amsterdam
M1 , s |= ¬KA KB q A does not know that B knows it rains in Liverpool
M1 , s |= E(p ∨ q) Everybody knows that it rains somewhere
M1 , s |= ¬EE(p ∨ q) Not everybody knows that everybody knows it rains
The conclusion of this example is that given what everybody knows, one cannot
conclude anything about what agents know about each other’s knowledge. Since
22 Chapter 2. Logic

one often does want to reason about this higher order knowledge, a stronger
notion of group knowledge is useful. Lewis therefore introduced the notion of
common knowledge [65]. Intuitively something is common knowledge if everybody
knows it, everybody knows that everybody knows it, everybody knows everybody
knows everybody knows it, etcetera. Common knowledge turns out to be a more
powerful notion than ‘everybody knows’. First of all it is hard to obtain, but
on the other hand it can be a necessary condition in order to coordinate [71] or
to make linguistic conventions work [65]. Other phenomena, such as the pricing
of TV commercials, can also be explained by the need of advertisers to achieve
common knowledge instead of plain knowledge [21].
The notation Cφ is used to convey that φ is common knowledge. It can be
defined in the following way. Let ∼C be the smallest equivalence relation such
that for all X we have ∼X ⊆ ∼C . We can interpret Cφ in the following way.
M, w |= Cφ iff ∀v : w ∼C v implies M, v |= φ
The following formulas are valid under this interpretation.
|= Cφ → CCφ
|= ¬Cφ → C¬Cφ
|= Cφ → C(φ → ψ) → Cψ
|= Cφ → Eφ
|= φ → E(φ → Eφ) → Cφ
The common knowledge operator C thus satisfies positive and negative introspec-
tion, which means that it behaves as a knowledge operator.
One can show that if a formula φ holds in every world w of a model M , then φ
is common knowledge in M . Thus, if we present a model of a certain situation in
which there is no state where φ does not hold, then we have implicitly assumed
that φ is common knowledge. In most examples in this dissertation, common
knowledge is not introduced in the language. However, common knowledge of at
least the protocol is assumed everywhere in this dissertation, and in some cases
also the preferences of agents are common knowledge.

2.4 Theorem Proving, Satisfiability and Model

Checking
Theorem Proving, Satisfiability and Model Checking are three different problems
that one can formulate for a logic. In this section, it is explained how these
problems are relevant for multi-agent protocols.
Theorem proving, the problem of finding derivations that prove a given the-
orem, has always been one of the main uses of logic. Automated theorem prov-
ing has received much attention in the field of AI. As described by for instance
2.4. Theorem Proving, Satisfiability and Model Checking 23

MacKenzie [66], automated theorem proving has important applications in the

verification of hardware and software.
A related problem to theorem proving is satisfiability. In the satisfiability
problem one has a formula φ such that ¬φ cannot be proven, and one would like
to find a model M such that M |= φ. In many cases one can use the same method,
for instance a tableau-based method [96], for theorem proving and satisfiability: If
φ is satisfiable a model is produced by the method, otherwise the method returns
a proof for ¬φ.
Model checking is the problem of verifying whether a formula φ holds on a
given model M , and is also widely used for verification of systems [53]. For many
logics, including propositional logic, model checking is substantially easier than
satisfiability. Intuitively this can be explained by the fact that in order to solve
a satisfiability problem, one has to find a model, whereas for model checking one
has been given one specific model.
Model checking can be used for verification of computer hardware and pro-
grams, in the following way. The system to be checked is translated in a logical
model M , and the property that one would like to verify is translated in a formula
φ. The following set of correspondences illustrates the correspondence between
the original verification problem and the model checking problem.

protocol model
⇔
property formula
What is meant here is that protocols correspond to models, and properties
with formulas. The double-headed arrow indicates that one can go back and forth
from the informal description on the lefthand side to the formal description of
the situation on the righthand side.
Model checking was first done for properties involving time [22, 10]. One rea-
son that model checking has become popular is the invention of symbolic model
checking [69]. Using this technique the model that is checked is not stored explic-
itly, but represented in a symbolic way, using for example ordered binary decision
diagrams. The model can thus have more states than one can store explicitly in
the memory of the computer that is used. The use of symbolic model checking has
made it possible to check systems with billions of states, instead of only millions
of states. Recent work aims to develop model checking techniques for epistemic
properties [104, 101, 56].
For some logics one can use theorem proving for system verification. This is
possible if one can translate the system into a formal structure (for instance a tree
or a graph) and then describe this structure using a formula φ1 . The property
to be verified is represented by a formula φ2 , and one uses the theorem prover
to prove that `φ1 → φ2 . This results into a proof that any system of the given
structure has the desired property. Schematically one can display this in the
following way.
24 Chapter 2. Logic

protocol φ1
⇔
property φ2
Hence both the protocol and the property correspond to a formula. This
method of verification by theorem proving is only possible for logics in which
one can express the structure of a system. For some logics, such as Pauly’s
coalition logic [85] (discussed in section 3.4), this is possible because the logic has
a modal operator that corresponds to exactly one step in the protocol. Thus,
one can construct, for each game tree T describing a protocol, a formula φ1 that
describes this game tree. For other logics, such as Van Benthem’s logic for process
models [99], also discussed in section 3.4, this is not possible. In this logic game
trees cannot be described in detail, and hence no suitable formula φ1 can be
found.
Another long term goal in computer science is the automatic design of systems.
In this case the user indicates the properties that a system, protocol or program
must have, and the computer constructs a system. One can always use the AI-
heuristic ‘generate and test’ in order to solve this problem, but this heuristic is
not very efficient. In general this problem is more difficult than verification, just
as satisfiability is more difficult than model checking: again one has to construct
something, instead of just computing a yes-no answer.
In economics, the classical name for the problem of finding a suitable auction
or procedure is mechanism design [83]. This problem has been picked up by com-
puter science researchers, and several researchers are now active in the area called
automated mechanism design or computational mechanism design [29, 25]. For
all logics in which models can be seen as representing a system, the mechanism
design problem can be reduced to satisfiability checking. Unfortunately satisfi-
ability checking is often a difficult problem, and hence mechanism design using
logical methods is often hard as well.

Linear Temporal Logic

A logic that is often used successfully in combination with model checking is
linear temporal logic LTL. A formula of this logic is interpreted over a sequence
of states. Such a sequence represents the different states a system can go through
during a computation. The different operators of LTL can be used to refer to
states after the current state.

2.4.1. Definition. The logic LTL contains formulas φ generated by the follow-
ing rule. In this rule, p is a typical element of P

φ ::= p | ⊥ | φ → φ | φ U φ | fφ
2.4. Theorem Proving, Satisfiability and Model Checking 25

The sequences of states that are considered for LTL are not always finite. There-
fore, it is more convenient to use functions f : N → 2P as sequences. These func-
tions assign subsets of true atomic propositions to each natural number. Such a
sequence is called a history or a run. Each formula φ is interpreted over a pair
f, n where f is such a function, and n ∈ N indicates the current state.

f, n |= ⊥ never
f, n |= p where p ∈ P iff p ∈ f (n)
f, n |= φ → ψ iff f, n |= φ implies f, n |= ψ
f, n |= fφ iff f, n + 1 |= φ
f, n |= φ U ψ iff ∃m : m ≥ nf, m and |= ψ and ∀m > k > n : f, k |= φ

The ‘until’ operator φ U ψ indicates that at some point in the future ψ is true,
and until that time φ holds. The next operator fφ expresses that φ is true at the
next state. A commonly defined shortcut is the ‘sometimes’ operator 3φ = > U φ
that indicates that φ is true somewhere in the future. One can define an ‘always’
operator 2φ that indicates that φ holds forever from now on, with the following
definition: 2φ = ¬3¬φ.
A system to be verified is not modeled as a single run but as a set of runs.
Since these runs can be infinite, such a set must be specified implicitly. The
most common way to do this is to define a labeled graph (V, E, π) where π is a
function from V to 2P . A possible path in this graph is a sequence w : N → V
such that (w(n), w(n + 1)) ∈ E for all n ∈ N. Any possible path w defines a
possible run f : n 7→ π(w(n)). A shorthand notation for this is f = π(w), since
f (n) is constructed by first computing w(n), and then applying π to the result.
Let W be the set of all possible paths, and R the set of all possible runs.
A model checker for LTL takes a description of a system and an LTL formula
φ. From the description it computes a set of possible runs R, and then it checks
whether for all runs r ∈ R it is the case that r, 1 |= φ. If so the model checker re-
turns true, otherwise the model checker returns the run r such that r, 1 |= ¬φ. An
LTL model checker thus returns counter-examples that can be very informative
for system designers. A well-known model checker for LTL is SPIN [51].

Branching Time Temporal Logic

Computation tree logic is a temporal logic that is suitable for reasoning about
models that have multiple possible futures [23]. The formulas can express whether
certain events happen in all possible futures, or only in some possible future.
This has turned out to be an important feature in the verification of for instance
concurrent computer systems.
26 Chapter 2. Logic

2.4.2. Definition. The logic CTL contains formulas φ generated by the follow-
ing rule. In this rule, p is a typical element of P .

φ ::= p | φ → φ | ⊥ | ∀ψ | ∃ψ
ψ ::= 2φ | φ U φ

The formulas φ generated by the grammar rules stated above are called state
formulas because they are interpreted over states. The formulas ψ are called
path formulas, because they are interpreted over paths. As the name indicates
this logic can be interpreted over tree structures, where the nodes of the tree
are states of a system, and the edges indicate how the system can go from one
state to the next. Typically such a tree is constructed by considering all possible
paths through a labeled graph (V, E, π), combined in a set W. Each path w
is a finite sequence w(1)w(2) . . . w(n) or infinite sequence w(1)w(2) . . . of states
w(n). Thus the notation w = v... ∈ W is used to indicate that w is an infinite
path that starts in v. CTL formulas φ can be interpreted over a set of paths
combined with an interpretation function π : V → 2P and a current state v ∈ V .
The path formulas are however interpreted over a set of paths combined with an
interpretation function π : V → 2P and a current path w ∈ W.

W, π, v |= ⊥ never
W, π, v |= p where p ∈ P iff p ∈ π(v)
W, π, v |= φ1 → φ2 iff W, π, v |= φ1 implies W, π, v |= φ2
W, π, v |= ∀ψ iff ∀w = v... ∈ W : W, π, w |= ψ
W, π, v |= ∃ψ iff ∃w = v... ∈ W : W, π, w |= ψ

W, π, w |= 2φ iff ∀n ≥ 1 : W, π, w(n) |= φ
W, π, w |= φ1 U φ2 iff ∃m ≥ 1 : W, π, w(m) |= φ2 and
∀k : m > k ≥ 1 ⇒ W, π, w(k) |= φ1

Both CTL and LTL can be model checked in polynomial time in terms of the
number of states of the system [90]. This makes the verification of systems feasi-
ble. Unfortunately, the number of states of a system can increase exponentially
with the number of components, or the amount of memory cells, that a system
has. The conclusion that model checking is tractable, is thus perhaps somewhat
misleading, since model checking is intractable when the input is measured in the
size of the description of the system. Summarizing and interpreting the current
state of the art, one could say that model checking using LTL and CTL is feasi-
ble, but not (yet) very tractable for real world systems. In practice, techniques
like symbolic model checking and other heuristics can be used to make model
checking for ever larger systems feasible.
2.5. Computational Complexity 27

2.5 Computational Complexity

In the field of computational complexity, researchers contemplate why certain
problems are hard to solve for computers [81, p. v]. A problem in this context
has to be defined precisely. A problem is function f : Prob → Sol, that takes
problem instances d ∈ Prob to their solution f (d) ∈ Sol. In the case of a decision
problem, the solution space consists of only two answers: Sol = {0, 1} where
0 means ‘no’ and 1 means ‘yes’. An algorithm for problem f is a mechanical
procedure that takes an input d and produces its answer f (d).
To give an example of a problem in this sense, consider satisfiability. For
each logic, its satisfiability problem is a logical problem of interest to complexity
theorists. Satisfiability is usually phrased as a decision problem f such that
f (φ) = 1 iff φ is satisfiable.
Since there are many different computer architectures, one can have many
different ideas of what counts as a mechanical procedure. However, it turns out
that many of these architectures are equivalent with respect to the computational
resources that are required to solve a problem, and therefore it does not matter
which architecture or notion of algorithm one chooses. A common choice is to
consider Turing machines, introduced by Alan Turing, because these machines are
very convenient from a theoretical perspective [81, p. 19]. In practice, algorithms
for Turing machines can be converted to programs for actual computers, that
have a comparable efficiency.
Turing machines that compute f , do not work on problem instances sim-
pliciter, but on representations thereof. Every problem instance can be repre-
sented by a string of symbols. In practice, it is sufficient to consider only two
distinct symbols, 0 and 1, to represent any object. A two-valued variable is called
a bit, and the size kdk of an object d can thus be measured in the number of bits
one needs to encode the object. For instance formulas could be encoded by using a
sequence of say 8 bits to encode each symbol in the formula. In that case a formula
p∨q would be 24 bits in size. The exact number of bits needed in a representation
is not always important. It is enough to know that “any ‘finite’ mathematical
object can be represented by a finite string over an appropriate alphabet” [81, p.
26]. We simply assume a “reasonably succinct representation” [81, p. 26] is used
for objects such as formulas.
One computational resource we are interested in is the time it takes for an
algorithm to compute the answer to a problem, and this is called the computation
time. The question is how the computation time (measured in steps of the head
of the Turing machine) depends on the size of the input. In particular, one would
like to find monotone functions b : N → R that provides an upper bound on
the computation time. If for all d ∈ Prob larger than a certain fixed length c,
the algorithm can compute f (d) in less than b(kdk) steps, then we say that the
running time of the algorithm is bounded by b.
Similarly one can consider bounds on the amount of bits that an algorithm
28 Chapter 2. Logic

needs as working memory in order to compute its answer. Thus, if there is a

constant c such that for instances d with kdk > c, the algorithm can compute
f (d) using less than b(kdk) bits of memory, then we say that the space needed by
the algorithm is at most b.
This notion of a bound is overly precise as it greatly depends on the machine
architecture chosen: for instance sorting a list can have a bound b(x) = 3·x2 +167
on one machine and b0 (x) = 3.5·x2 +14 on another machine. Therefore, the bound
functions are put into equivalence classes. A bound b0 is in the same equivalence
class as b if there are constants c, e ∈ R such that for all inputs d with kdk > e
we have that b0 (kdk) ≤ c · b(kdk). We write O(b(kdk)) to denote the equivalence
class of b(kdk). It is not hard to see that both bounds given above sit in the same
equivalence class O(kdk2 ). They are called quadratic bounds. Similarly one has
linear bounds O(kdk) and exponential bounds O(2kdk ).
An algorithm runs in polynomial time if there is some bound b on its running
time that is in O(xn ) for some n. Similarly an algorithm can be in polynomial
space if there is a bound b on the memory needed that is in b = O(xn ) for some
n.
In figure 2.3 a graph is displayed. If one sees such a graph as a network
of roads between cities, a question of practical importance would be what the
shortest path is between two nodes of the graph. Thus, the path-finding problem
would be a function f that takes (V, E, v1 , v2 , n) as input, and returns a path
w1 . . . wm with m ≤ n, w1 = v1 and wm = v2 , if such a path exists. Otherwise it
should return ‘no’. This problem can be solved in polynomial time, for instance
using Dijkstra’s algorithm [27].
The model checking problem for propositional logic is the function g such
that g(M, φ) = 1 if and only if M |= φ. This function can be computed in time
O((kM k + kφk)2 ) and thus this problem can be solved in polynomial time. Only
a small amount of memory is needed, bounded by kφk, and thus the problem is
also in polynomial space. In general a Turing machine can only use a polynomial
amount of space in polynomial time, so all polynomial time problems are in
polynomial space.
Algorithms can be classified into classes of algorithms whose bounds are in the
same equivalence class. These classes are called complexity classes. The following
complexity classes are well-known and turn out to be relevant for the results in
this dissertation. The class P contains problems that can be solved in polynomial
time. The class PSPACE contains problems that need a polynomial amount of
memory.
Problems can also be divided into the same complexity classes. A problem
belongs to a class C if there is an algorithm in C that solves the problem. Thus,
the problem of finding the shortest path between two points in a graph is in P
because there is a P algorithm that solves this problem.
The class P is often called that class of tractable of problems or problems that
can be solved efficiently. In this dissertation we follow this convention and indeed
2.5. Computational Complexity 29

Figure 2.3: A graph with a Hamiltonian cycle

uses these words as meaning ‘solvable in polynomial time’. The term intractable
means that the problem is not in the class P.

Nondeterministic Computation
There are many decision problems f such that f (d) = 1 if there exists some object
w that satisfies certain criteria. A good example is the problem, for a given
graph, to decide whether this graph has a Hamiltonian cycle. A Hamiltonian
cycle visits each node of the graph, but does not use the same edge twice. In
figure 2.3, Hamilton’s original problem is displayed. In this particular graph
there is a Hamiltonian cycle, and the reader is invited to find one as an exercise
(a possible answer is displayed in figure 2.4 on page 29). The reader will most
likely experience that finding a Hamiltonian cycle is harder than verifying that a
given path is a Hamiltonian cycle.
Suppose that the problem f : Prob → {0, 1} is defined such that f (d) = 1 iff
d is a graph that has a Hamiltonian cycle. Suppose also that we have a problem
g ∈ P that checks whether a path w is a Hamiltonian cycle on d. The relation
between f and g is that f (d) = 1 ⇔ ∃w : g(d, w) = 1.
The Hamiltonian cycle w is called a witness for d, since the existence of a
path w is evidence for the fact that f (d) = 1. We have assumed that g ∈ P, and
hence that we have a polynomial time algorithm for g. A very naive algorithm for
solving f would be the following. Guess some value w, and check whether g(d, w).
If you are very lucky in guessing w, then this algorithm works in polynomial time
as well. If you are not a good guesser, this algorithm is not efficient.
Any problem f for which there exist a polynomial time function g that checks
witnesses, is called solvable in nondeterministic polynomial time [26]. The class
of these problems is called NP . There are many problems that have practical
relevance in this class [35]. A logical example of an NP problem is the proposi-
30 Chapter 2. Logic

tional logic satisfiability problem f . In order to determine whether a formula φ

is satisfiable, one can guess a model M ⊆ P such that M |= φ. If such a model
is guessed correctly, then we know that f (φ) = 1. If no lucky guess can be made
at all, then φ is not satisfiable and thus f (φ) = 0.

Reductions and Completeness

It is often possible to translate an instance of one problem into an instance of
another problem. Such a translation is called a reduction in the context of com-
plexity theory. Suppose that f and f 0 are two problems. A reduction r from f
to f 0 is a function such that ∀d : f (d) = f 0 (r(d)). If a reduction r exists that
is relatively easy to compute, then solving an instance of f cannot be harder
than solving an instance of f . In order to determine f (d) one first computes the
reduced problem r(d) and then uses the algorithm, if one is known, for f 0 (r(d)).
If the reduction function r is in P, then we call f reducible in polynomial time to
f 0 [27].
A problem f is called C-hard if all problems in the complexity class C can
be reduced in polynomial time to problem f . If the C-hard problem f is itself a
member of class C, then f is called C-complete. Such a problem can be said to be a
representative for all problems of this class. Consider for instance the satisfiability
problem f for propositional logic. This decision problem can be defined by saying
that f (φ) = 1 if M |= φ for model M .

2.5.1. Theorem (Cook’s theorem). Deciding whether a propositional logic

formula φ is satisfiable is NP-complete.

This was the first theorem to be proven NP-complete, presented in 1971 [81, p.
176]. The proof contains a general method how a reduction function r can be
found for any NP decision problem that has a verification method g. Such a
general proof has to be given once for each class. In order to prove that another
problem in NP is NP-complete, it suffices to select a known NP-complete problem,
and then give a specific reduction function r from the complete problem to the
next problem.
In many NP-completeness proofs it is convenient not to use the general sat-
isfiability problem in a reduction argument, but to give a reduction from 3-CNF
formulas. A formula φ is in 3-CNF if it is in conjunctive normal form, and each
disjunction contains exactly three literals. Such a formula thus has the following
form. ^
φ = (±ai ∨ ±bi ∨ ±ci )
i

The sign ± can be either a negation or nothing, and ai , bi , ci ∈ P are atomic

propositions. The satisfiability problem for 3-CNF formulas is called 3SAT and
this problem is NP-complete [81, p. 183].
2.5. Computational Complexity 31

For the class PSPACE, we also use a logical problem in the reduction argu-
ments. We use the satisfiability problem for a quantified boolean formula, since
this problem is PSPACE-complete.
2.5.2. Definition. A quantified boolean formula φ is a formula of the form

∀x1 ∃x2 ∀x3 . . . ∃xn−1 ∀xn φq

such that φq is a propositional logic formula with {x1 , . . . xn } as atomic proposi-

tions.
An example formula is ∀p∃q(p ∨ ¬q) ∧ (¬p ∨ q). This formula is true if for all
truth values of p one can find a truth value for q such that the propositional
logic formula holds. Intuitively, ∀pφ is true if φ holds regardless what truth value
one chooses for p, and similarly ∃pφ holds if φ is true for some truth value for
p. Formally, we can define the following interpretation for quantified boolean
formulas, which is an extension of the interpretation of propositional logic given
on page 12. Let S ⊆ P be a set of atomic propositions, φ a quantified boolean
formula, and φq ∈ Lp a propositional logic formula.

S |= ∀xφ iff (S ∪ {x}) |= φ and S |= φ

S |= ∃xφ iff (S ∪ {x}) |= φ or S |= φ
S |= φq iff S |= φq in propositional logic

The QBF decision problem f is defined such that for any quantified boolean
formula φ we have

f (φ) = 1 if ∅ |= φ
f (φ) = 0 otherwise

2.5.3. Theorem (Stockmeyer and Meyer). The QBF decision problem is

PSPACE-complete.

The completeness proof for this problem was presented in 1973 [81, p. 487].
The satisfiability problem for propositional logic is of course a special case
of the QBF problem, where we allow only existential quantifiers. Instead of
considering whether p → q is satisfiable, one can consider the QBF problem of
deciding whether ∃p∃q(p → q) holds. Similarly one can consider other restrictions
on the number of quantifier series in a QBF problem. The following table lists a
few variants.
∃p1 . . . ∃pn φq satisfiability, in class Σ1 P or NP
∀p1 . . . ∀pn φq tautology, in class Π1 P or co-NP
∃p1 . . . ∃pn ∀q1 . . . ∀qn φq Σ2 P
∀p1 . . . ∀pn ∃q1 . . . ∃qn φq Π2 P
...
32 Chapter 2. Logic

Figure 2.4: A solution to Hamilton’s problem

The classes Σn P and Πn P are defined using oracles. A Turing machine with
an oracle is a machine that is allowed to ask certain difficult questions to a
supernatural being (an oracle). The oracle returns the right answer to these
questions in one time step. The class Σ2 P contains problems that can be verified
by a Turing machine that has an oracle for some NP-complete problem. The
problems given above are again complete for these classes: any Σ2 P problem can
be reduced to a QBF problem of the form ∃p1 . . . ∃pn ∀q1 . . . ∀qn φq [81, p. 428].

Open Problem
Intuitively it seems easier to verify a problem than to solve it. For instance it is
easy to see that the path given in figure 2.4 is a Hamiltonian cycle, whereas it
is less easy to find such a path. It is therefore widely believed [35] that not all
NP problems can be solved in polynomial time, and thus that no NP-complete
problems can be solved in polynomial time. If we assume that this is the case,
then NP-complete problems are intractable, and the same holds for the classes
Σ2 P and PSPACE. Unfortunately whether P 6= NP is one of the most famous
open problems in computer science [26].
In this dissertation we show for several problems that they are NP-complete,
Σ2 P complete or PSPACE-complete, and we use this as evidence for the in-
tractibility of these problems. Of course this is only partial evidence, since it is
still possible that all problems in NP, contrary to popular belief, are tractable.
Chapter 3
Game Theory

3.1 Overview
One can define game theory as the area of mathematics that is about games. In
this case, game theory is older than most people think. In the sixteenth century
the mathematician and physician Jerome Cardano wrote his Book on Games of
Chance [78]. The book opens with a statement describing the various forms that
games can take.

Games depend either on agility of body, as with a ball; or on strength,

as with discus and in wrestling; or on industriously acquired skill, as
at chess; or on chance, as with dice and with knucklebones; or on
both, as fritillus. [78, p. 185]

As Cardano indicates, certain games can depend on both skill and luck at the same
time. Nowadays Cardano’s book is classified as being about probability theory,
as opposed to game theory. The reason for this is that his book is concerned
with calculating the probabilities of certain events, but does not consider various
strategies and the influence of an opponents’ strategy.
The origin of game theory is therefore better placed in the first half of the
twentieth century. One of the first mathematical papers that focused on the
strategic aspects of games was Zur Theorie der Gesellschaftspiele by John von
Neumann [118]. The central question of this paper is about the optimal strategies
for players in a parlour game.

n Players, S1 , S2 , . . . , Sn , are playing a given parlour game G. How

should one of those players, Sm , play, in order to get a most beneficial
result? [118, p. 295]

John von Neumann also co-authored the first book on game theory, Game
theory and economic behaviour, which appeared in 1944 [74]. The title already

33
34 Chapter 3. Game Theory

Figure 3.1: Jerome Cardano

indicates that game theory is not merely about recreational games, but can be
applied to economics. Von Neumann and Morgenstern see a game as an optimisa-
tion problem in which multiple parties simultaneously try to optimize their own
outcome. According to them, this is ‘nowhere dealt with in classical mathemat-
ics’ [74, p.11]. This influential book introduced game theory to a wide audience,
summarized results that were ‘already known, but lacked formal proof’ [74, p.6]
and gave many game-theoretic terms their meaning.
The following list contains some of the terms introduced by Von Neumann
and Morgenstern.
A game: A description of a set of interactions between agents. The description
should include which agents can participate, what these agents can do, and
what these agents try to achieve.

A play: A specific sequence of interactions between agents. A game consists of

many possible plays.

A player: An entity that can make decisions in a certain game.

A move: An action that one can choose, a possibility.

A choice: An action that one has chosen.

A solution: ‘plausibly a set of rules for each participant which tell him how to
behave in every situation which may conceivably arise.’ [74, p31]
The words ‘game’ and ‘play’ still have the same meaning in most of the literature.
The word ‘player’ is nowadays often replaced by the synonym ‘agent’. A possible
reason for this change of terminology is that the word ‘player’ reminds people of
recreational games, whereas game theorists often consider less leisurely situations.
An agent can be a human player, but also an organisation or a computer program.
3.1. Overview 35

The word ‘solution’ should be used with caution. Not every game has a unique
solution, so the phrase ‘the solution of a game’ is misleading. A solution is
always a set of rules for all players. A set of rules for a single player can be called
a strategy, and if it is a good set of rules it can be called an ‘optimal’ or ‘rational’
strategy.
In order to decide what the best set of rules for an agent is, one must take
into account what information the agent has. First of all, it is important to know
whether the agent knows exactly which game it is playing. Even the game of
chess has several variants, and one can imagine a player who is not sure what
the current variant is. Furthermore, in chess an opponent can try to win at all
cost, or an opponent can be trying to draw. These two opponents may require
different strategies. Von Neumann and Morgenstern boldly state that

... we cannot avoid the assumption that all subjects of the econ-
omy under consideration are completely informed about the physical
characteristics of the situation in which they operate and are able
to perform all statistical, mathematical, etc., operations which this
knowledge makes possible. [74, p.30].

They call this assumption complete information. Another question is whether

a player can observe or remember every aspect of the current situation. If this
is the case we say that a game has perfect information, but if some aspects are
hidden the game has imperfect information. Chess is a good example of a perfect
information game, whereas poker is an imperfect information game.
In games of imperfect information, one has to consider the question of whether
a player can remember his own observations, and its own previous actions. If the
description of the game indicates that a player can remember both observations
and previous actions, then a game has perfect recall . If a player can remember
all its previous observations the game has perfect memory. An interesting but
perhaps artificial example of a game with imperfect recall was constructed by Von
Neumann and Morgenstern when they argued that one can treat teams of players
with the same objective as single players. ‘Bridge is a two-player game, but the
players 1 and 2 do not play it themselves’ [74, p. 53]. The four real participants
of a bridge game become ‘agents’, acting on behalf of the two absent players.
Von Neumann and Morgenstern focus on games with two players in which
the preferences of the players are exactly opposite. They have less to say about
what they call general games, which are games with more than two players. The
main problem of solving these general games is that in these games, the out-
come depends on the possible co-operations between two players. Game theory
is nowadays split in two almost unrelated parts: in cooperative game theory it
is assumed that agents can make binding agreements between each other, and
these agreements are enforcable [46]. In this case the exact strategies that are
used are not so important. The important aspect is which outcome the agents
36 Chapter 3. Game Theory

should collectively aim for, and how they should split the profits of their collab-
oration. Throughout this thesis it is not assumed that agents can make binding
agreements, and this is called non-cooperative game theory.
The problem of how to treat general games was addressed by John Nash in
1951 [73]. Nash showed that a solution of a game should be a set of strategies
such that when all agents use these strategies, no player has any incentive to use
another strategy. He called such a solution an equilibrium, and nowadays it is
called a Nash equilibrium.
The Nash equilibrium is a solution concept that allows the same game to have
many solutions. A solution concept is a general rule that for each game predicts
which strategies are good. Many researchers have argued that the Nash equilib-
rium allows more solutions than it should. Various refinements have thus been
proposed. One of the first was Selten’s subgame perfect (Nash) equilibrium [79].
This concept makes most sense when applied to perfect information extensive
games. Every decision point of such a game can be seen as the starting point
of some game, and these games are called the subgames of the original game.
Selten argued that a solution should not only be an equilibrium of the original
game, but also of every subgame. All finite perfect information extensive games
have a subgame perfect equilibrium, and this equilibrium can efficiently be calcu-
lated by a procedure named backward induction. The procedure is sometimes also
called Zermelo’s algorithm, since Zermelo applied the same procedure in 1913 to
analyse chess. Other refinements exist, for instance the trembling hand perfect
equilibrium [93], the proper equilibrium [72] and the sequential equilibrium [62].
Even though Von Neumann and Morgenstern limited themselves to the study
of complete information games, it was only a matter of time before incomplete
information games were considered. Harsanyi proved in 1967 that incomplete
information games can in certain cases be reduced to complete, imperfect infor-
mation games [47]. He did not assume that all agents know all other’s preferences.
Instead he assumed that each agents’ preferences depended on the type of the
agent. The number of types was limited, and a probability distribution for the
types should be commonly known. In that case the incomplete information game
can be considered an imperfect information game where in the first move the
types of all agents are determined at random according to the given probability
distribution. After that the game would proceed as normal.
The applications of game theory have not been limited to the economic realm.
Early on it was already realized that game theory could be applied to political and
military conflict situations. An early and influential book applying game theory
to political science is The Strategy of Conflict by Thomas C. Schelling [88]. This
book contains various ideas. First of all Schelling showed using experiments that
people are able to coordinate their actions. They can do this because real world
problems have so-called focal points, even if these points are no longer present in
the abstract models of game theory. For example, Schelling asked people to try to
meet each other on a given day in a given city, without giving them a specific time
3.2. Strategic Games 37

and place, and without allowing them to communicate to each other. One might
expect that this is not possible, since there are so many possibilities. However,
Schelling’s subjects were remarkably successful in meeting each other. Many of
them were able to select the same time and place, by reasoning about which
points were most obvious to the average person. For instance for a meeting in
New York, people often choose to meet at 12.00 at Central Station. Schelling
also discusses the rationality of promises and threats. An equilibrium that is not
subgame perfect, can for instance contain an unreliable threat. In that case a
player threatens to do something in a certain situation, even though it is not
rational to really do this action if the subgame in which the action can be done
is reached.
More surprising applications were found in biology. This may seem strange, as
animals or plants are not usually ascribed rationality or intelligence. The players
are thus not assumed to reason about their strategies, but to repeat behaviour
that has been successful in the past. The Nash equilibrium can also be applied
in these circumstances. In classical game theory attention is focused on the
solutions itself, whereas the process by which a solution is reached is ignored. In
evolutionary game theory it is also studied how certain strategies are replaced
by others, using dynamic systems theory. John Maynard Smith is one of the
originators of this field [68]. Evolutionary game theory is a maturing and popular
research area [37, 120].

3.2 Strategic Games

Games can be presented in different forms. A very natural but detailed form is as
an extensive game. In this form there is a number of decision points in each play
of the game, and the outcome is determined by all these decisions. This model
is very detailed. Often a less detailed perspective is taken, and thus games are
studied in strategic or normal form. In this form, each agent has a number of
strategies available at the beginning of the game, and each agent independently
picks a strategy. We can calculate the payoff of the game directly, without going
into details which actions have been played. The general definition for an n-agent
normal form game is the following. We let Σ be the set of all agents, and assume
that Σ = {1, 2, . . . , n} for some n > 0. Thus, in this chapter, and in chapter 8, we
use the natural numbers as labels for agents. This is necessary to simplify some
of the definitions.

3.2.1. Definition. A strategic game G is a tuple (Σ, {S X }X∈Σ , U) where Σ =

{1, . . . , n} is a set of agents, for each X ∈ Σ the set S X is a set of strategies for
agent X, and UX : (S 1 × . . . × S n ) → R is a utility function for agent X.

The utility function UX takes a strategy for each agent as input, and return a real
number for agent X, which represents that agent’s utility. The notation UX (~s)
38 Chapter 3. Game Theory

denotes the Xth element of the vector U(~s), and thus represents the utility of
agent X when the strategy profile ~s is used. One can see U as a function that
returns a vector of real numbers, one for each agent.
In a strategic game, each agent tries to maximize its utility. They can choose
any strategy from their set of strategies, and these sets can be infinite. One can
combine the strategies that agents have chosen in a so-called strategy vector . A
tuple ~s = (s1 , s2 , . . . , sn ) is a strategy vector for game G if G = (Σ, {S X }X∈Σ , U)
and for all agents X we have sX ∈ S X . In order to manipulate these strategy
vectors, two constructs are needed. The construct s−j denotes the vector s with
the jth element removed. Thus, (a, b, c)−2 = (a, c). The construct [s, x] is used
to denote the vector s with x inserted in an appropriate place.
For example [(a, c), d] = (a, d, c), or [(a, c), d] = (a, c, d), depending on what is
appropriate. Determining what the appropriate place is can be difficult, therefore
the construction [s, x] can only be used if s is of the form s−X for some agent X.
For example [(a, b, c)−2 , d] = (a, d, c). In practice this means that these constructs
can be used to replace one strategy of a strategy vector by another strategy. The
combination [s−j , tj ] or, depending on author’s preferences, (s−j , tj ) is standard
in game theory [79, p. 7].
In many situations, every agent X has a finite number of basic actions mi to
choose from. The total utility of a strategy somehow depends on the payoff of
each action. The number of strategies can still be infinite. The payoff of each
action is typically given in the form of a matrix A. We first present the case for
two agents, and then extend this to an arbitrary number of agents.

Two player games

3.2.2. Definition. An m × n bi-matrix is a function A such that for each vector
(a, b) where a ∈ {1, . . . , m} and b ∈ {1, n}, and for each X ∈ {1, 2}, the function
A returns a real number AX (a, b) ∈ R.

The next table shows how a 2 × 3 bi-matrix is usually displayed. This matrix can
be used to define a game where agent 1 has two actions, and agent 2 has three
actions.
 1
A (1, 1), A2 (1, 1) A1 (1, 2), A2 (1, 2) A1 (1, 3), A2 (1, 3)


A1 (2, 1), A2 (2, 1) A1 (2, 2), A2 (2, 2) A1 (2, 3), A2 (2, 3)

In a pure strategy game, the strategy of both agents consists of a single action.
We can use the bi-matrix A given above to define a pure strategy game G =
({1, 2}, ({1, 2}, {1, 2, 3}, T ), U). The set of agents would be {1, 2}, the strategy
set of agent 1 would be {1, 2}, and the strategy set of agent 2 would be {1, 2, 3},
and the utility function would be defined by U(a, b) = (A1 (a, b), A2 (a, b)).
In a mixed strategy game, a strategy consists of a probability for each ac-
tion. Thus a mixed strategy game G based on the bi-matrix A would be G =
3.2. Strategic Games 39

({1, 2}, (S 1 , S 2 ), U), where S 1 = {(a, b)|a, b ∈ [0, 1], a+b = 1} and S 2 = {(a, b, c)|a,
b, c ∈ [0, 1], a + b + c = 1}. An example strategy for player 1 would be (0.25, 0.75).
If the agent follows this strategy it should take action 1 twenty-five percent of
the time, and action 2 seventy-five percent of the time. The utility function U
returns the expected payoff, and is defined as

U((a, b), (c, d, e)) =

(acA1 (1, 1) + adA1 (1, 2) + aeA1 (1, 3) + bcA1 (2, 1) + bdA1 (2, 2) + beA1 (2, 3),
acA2 (1, 1) + adA2 (1, 2) + aeA2 (1, 3) + bcA2 (2, 1) + bdA2 (2, 2) + beA2 (2, 3))

Multi-player games
3.2.3. Definition. An m1 × m2 . . . × mn multi-matrix is a function A such that
for each vector a1 a2 . . . an where aY ∈ {1, . . . , mY } for all Y ∈ {1, . . . , n}, and for
each X ∈ {1, . . . , n}, the function A returns a real number AX (a1 i2 . . . an ) ∈ R.

The term A(a1 , a2 . . . an ) denotes a vector v ∈ RΣ such that v1 = A1 (a1 a2 . . . an ),

v2 = A2 (a1 a2 . . . an ) etcetera. A bi-matrix is a multi-matrix where n = 2. The
notation RΣ , which is for instance used by Gamut [34, p. 84], denotes the set of
all functions f : Σ → R. The set Σ is often finite and is assumed to have some
kind of natural ordering, such as the set {1, 2, 3}. If this is the case then the
elements f ∈ RΣ can be seen as tuples. Each element f would correspond to the
tuple (f (1), f (2), f (3)). Thus, the set R{1,2,3} is isomorphic to R3 .
For a given multi-matrix A one can in fact define different games. The simplest
type of game is the pure strategy game. In this game, the strategy of each agent X
consists of a single action aX and the payoff is then A(a1 . . . an ). This definition
does not allow agents to play randomly.
In a mixed strategy game, the strategy of an agent is a probability distribution
over the available actions. The utility is the expected (weighed average) value
of A. This type of game is defined below. The shorthand AX i (~ s) denotes the
expected payoff of action i for agent X when the other agents use strategies
from ~s. It can be defined in the following way. Define the set ViX = {~v |∀Y ∈
Σ \ {X} : vY ∈ S Y , vX = i}. Thus, this set contains the pure strategy profiles in
which agent X selects action i. For instance if A is a 2 × 2 multi-matrix we have
V21 = {(2, 1), (2, 2)}.

3.2.4. Definition. For any multi-matrix A, agent X and action i, and vector ~s
of mixed strategies sY for each agent Y , we define the expected payoff of action
i for agent X by:
X
AX
i (~
s ) = ((s1 )v1 · · · (sX−1 )vX−1 (sX+1 )vX+1 · · · (sn )vn )AX (~v )
(v1 ...vn )∈ViX
40 Chapter 3. Game Theory

In this definition, the element v1 denotes a possible action for player 1, s1 is the
strategy of player 1, and therefore (s1 )v1 denotes the probability that player 1
will play action 1. Although this definition is hard to read, in practice it is not
hard to see how this expected payoff is computed. To give an example, let A be
again a 2 × 2 multi-matrix, assume that the first player plays action 1 with ninety
percent probability, and that the second player plays the first action with forty
percent probability. Then the expected payoff of action 2 for agent 1 is computed
in the following way.

A12 (((0.9, 0.1), (0.4, 0.6))) = 0.4A1 (2, 1) + 0.6A1 (2, 2)

The following set Pm is used in the definition of mixed strategies. It contains

vectors that sum up to one. These vectors can be seen as specifying probabilities
for all actions.
X
Pm ={x ∈ [0, 1]m | xi = 1}
i

3.2.5. Definition. Let A be an m1 × m2 . . . × mn multi-matrix. The mixed

X
strategy game Mm (A) of A is a tuple (Σ, {S
P },XU)Xwhere Σ = {1, 2, . . . , n}, the
X mX X
strategy sets are S = P , and U (~s) = i si Ai (~s).

Recall that the notation UX (~s) denotes the Xth element of the vector U(~s). It
represents the utility of agent X when the strategy profile ~s is used.
The fact that agents can play mixed strategies is explicitly defined in this
definition of a mixed strategy game. We assume that all agents are equipped with
random number generators (coins, dice or whatever) so that they can randomize
their behaviour exactly as specified in their strategy. This definition of a mixed
strategy game is such that each mixed strategy game is in fact a strategic game.
For the next definition we need the function argmax that returns all inputs
that maximize a given function. argmaxx f (x) = {x|¬∃y : f (x) < f (y)} We
use the function argmax to define what a ‘good’ strategy is: A good strategy
is a strategy that returns a maximal utility. The function bX returns the best
response strategies for agent X for a given game and strategy vector.

3.2.6. Definition. Let (Σ, {S X }X∈Σ , U) be a game and ~s ∈ ( X S X ) a strategy

Q
profile. The best response b(~s) = b1 (~s) × · · · × bn (~s)) is defined by

bX (~s) = argmaxt UX ((s−X , t))

The set b(~s) thus contains the strategy vectors t such that tX is optimal if all
opponents Y use the strategy sY . We could assume that the strategy of the
opponents is fixed. The set bX (~s) is the set of best decisions for agent X.
When playing a game an agent cannot always predict what strategy the other
agents use, because the other agents might want to change their strategy once
3.3. Extensive Games 41

they learn that X uses a strategy in bX (~s). The notion of a best response is
therefore not a solution concept in itself. One can however search for fixed points
in the best response function, and this is called a Nash equilibrium.

3.2.7. Definition. Let (Σ, {S X }X∈Σ , U) be a game and ~s ∈ ( X S X ) a strategy

Q
profile. The vector ~s is a Nash equilibrium iff ~s ∈ b(~s).

Every mixed strategy game has at least one Nash equilibrium [73]. There has
been some discussion in the literature whether the notion of a Nash equilibrium
needs to be refined. Several refinements have been proposed [72], but none of
them has the appealing simplicity of the Nash equilibrium.
A special class of games for use in logic are the win-loss games. In these
games the utility functions only takes two values, which can be associated with
winning and losing. Typically these values are 1 and 0. The utility function can
then be specified by stating what the winning positions are. These sets can then
be specified by stating a formula, so that a position is winning if it makes the
formula true.
Another special class are the constant-sum games. We define this property
only for games with exactly two players. A game ({A, B}, {S A, S B }, U) is a
constant-sum game if there is a constant c ∈ R such that for any strategy profile
(σA , σB ) it is the case that UA ((σA , σB )) + UB ((σA , σB )) = c. If the constant c is
0 then we call it a zero-sum game. The next bi-matrix A defines a constant-sum
game where the constant is 2.
 
1, 1 0, 2 2, 0
2, 0 1, 1 0, 2
Examples of strategic games can be found in many game theory text books.
All classical examples can be found in the primer by Osborne and Rubinstein [79],
but more playful examples are given by Binmore [11]. Textbooks on evolutionary
game theory such as Gintis’ [37, 120] also make much use of strategic form games.
In each case the examples of strategic games are often presented in the form of
finite multi-matrices, and this can give readers the false impression that strategic
games are always finite, small and simple. In chapter 8, it is shown that there
are many other strategic games that do match definition 3.2.1, but are not pure
or mixed strategy games.

3.3 Extensive Games

In an extensive game the agents have to make sequences of choices that ultimately
lead to an outcome. There are multiple decision points and at each decision point
one of the agents has to decide what to do next. The rules of the game specify
exactly which sequences of actions are legal. We represent these rules in a very
simple way, by listing all sequences that are allowed.
42 Chapter 3. Game Theory

a b a b b b b b

a b a a a b

a b a b

Figure 3.2: A game tree and a set of sequences

3.3.1. Definition. A non-empty set H of sequences is a sequence set if for

any sequence h and action a it is the case that ha ∈ H implies h ∈ H. For any
sequence set H and h ∈ H we define the set of next actions A(H, h) = {a|ha ∈ H}
and the set of terminal sequences Z(H) = {h ∈ H|A(H, h) = ∅}.

Another term for a sequence set could be a non-empty prefix-closed set.

If H is a sequence set then one can define a graph G = (V, E) by defining
V = H and E = {(h, ha)|ha ∈ H}. This graph is a tree with the empty sequence
 as root. Such a tree is often called a game tree. In figure 3.2 a game tree and
the corresponding set of sequences is displayed.
Extensive games can be played as perfect information games. In this case every
agent can distinguish all sequences, and thus the agent can select the action that
is best for that specific decision point.

3.3.1 Perfect Information

In order to play an extensive game, one must know which agent can influence
which decision. Therefore, we augment the game tree with a function turn that
returns the agent that is in control of a certain history.

3.3.2. Definition. A game form F is a tuple F = (Σ, H, turn), where Σ is

a finite set of agents, H is a finite sequence set and turn is a function turn :
H \ Z(H) → Σ.

A game form by itself is often not what one needs. One typically want a game
form with a utility function (if you are a game theorist) or a game tree annotated
with atomic propositions (if you are a logician). In certain cases you might want
both. We use the word ‘interpreted’ to indicate a structure that is labeled with
atomic propositions. If a structure contains utilities it is called an extensive game,
otherwise a game form.
3.3. Extensive Games 43

name contains
(extensive) game form tree
interpreted game form tree, atomic propositions
extensive game tree, utility function
interpreted game tree, utility function, atomic propositions
The word ‘extensive’ emphasizes that we deal with games in which the order of
moves is explicitly present. We omit it if it is not necessary, and thus we speak
of game forms rather than extensive game forms. An extensive game form is
thus synonymous to game form, and so are interpreted extensive game form and
interpreted extensive game.
3.3.3. Definition. An extensive game F is a tuple F = (Σ, H, turn, U), such
that (Σ, H, turn) is a game form and U : Z(H) × Σ → RΣ .
The function U is called a utility function. It returns the utility for each agent
and each agent tries to maximize its utility.
3.3.4. Definition. Let (Σ, H, turn) be a game form. A pure strategy σ for agent
X in game form F is a function σ with domain {h ∈ H|turn(h) = X} such that
σ(h) ∈ A(H, h).
The notion of a strategy can be extended to strategies for coalitions Γ ⊆ Σ. A
pure coalition strategy σΓ for Γ is a function f with domain {h ∈ H|turn(h) ∈ Γ}
such that σ(h) ∈ A(H, h).

3.3.5. Definition. Let F = (Σ, H, turn) be a game form. A behavioural strategy

σ for agent X ∈ Σ in game form (Σ, H, turn) is a function σ with domain {h ∈
H|turn(h) = X} such that for each h, σ(h) is a probability distribution over
A(H, h).

There is a difference between mixed strategies and behavioural strategies [79].

The concept of mixed strategies applies to strategic games. A mixed strategy
is itself a probability distribution. A behavioural strategy is a function that
returns probability distributions for nodes of an extensive game. For imperfect
information games without perfect recall the two kinds of strategies are not equiv-
alent [79].

3.3.6. Definition. Let F = (Σ, H, turn) be a game form. A nondeterminis-

tic strategy σ for agent X in game form F is a function σ with domain {h ∈
H|turn(h) = X} such that σ(h) is a non-empty subset of A(H, h).

The notion of a strategy for a coalition Γ ⊆ Σ can also be introduced for be-
havioural and nondeterministic strategies.
For each game form F , we define SpX (F ) to be the set of pure strategies of
agent X in F , the set SbX (F ) to be the set of behavioural strategies of X in F , and
44 Chapter 3. Game Theory

SnX (F ) the set of nondeterministic strategies. The notion of behavioural strategy

is a more general notion than that of a pure strategy. For any pure strategy σ
one can find a behavioural strategy σ 0 by defining σ 0 (h)(a) = 1 if a = σ(h), and 0
otherwise. Thus, one action gets probability one and the other actions get prob-
ability zero. For each behavioural strategy σ one can define a nondeterministic
strategy σ 0 by defining σ 0 (h) = {a ∈ A(H, h)|σ(h)(a) > 0}. Thus, σ 0 returns that
actions that have a nonzero probability in σ. A nondeterministic strategy is a less
detailed description of a behavioural strategy, in which the exact probabilities are
omitted. We use nondeterministic strategies when the exact probabilities are not
important.
Extensive games can be reduced to strategic games. This observation was
already made by Von Neumann and Morgenstern [74]. Below, we do this for both
pure strategy and behavioural strategy games.

3.3.7. Definition. Let (Σ, H, turn, U) be an extensive game. The correspond-

ing pure strategy strategic game is (Σ, {SpX }X∈Σ , U0 ), where U0 (~s) is defined by
U0 (~s) = U(r(~s, )) where

h iff h ∈ Z(h)
r(~s, h) =
r(~s, (h, sX (h))) iff turn(h) = X

3.3.8. Definition. Let (Σ, H, turn, U) be an extensive game. The correspond-

ing behavioural strategy strategic game is (Σ, {SbX }X∈Σ , U0 ). The function U0 (~s)
is defined by U0 (~s) = h∈Z(h) p(~s, h) · U(h) where
P

p(~s, ) = 1
p(~s, ha) = ~s(h)(a) · p(~s, h)

A Nash equilibrium of an extensive game is defined as a Nash equilibrium of

the corresponding strategic game. It should be clear from the context whether
the corresponding strategic game is the pure strategy game or the behavioural
strategy game. In figure 3.3, a small extensive game E1 is displayed. In this game
two PhD students Alice (A) and Bob (B) have the choice of cleaning their shared
office (action c), or to ignore the mess (action i). Since Alice arrives first, she has
to decide first what she will do. If she does not clean the office, Bob is faced with
the same choice. A clean office is worth 2 utility units, but cleaning the office
costs an agent 1.
Each agent has two pure strategies called σc and σi , listed in the next table.
B
A\ σc σi
σc (1, 2) (1, 2)
σi (2, 1) (0, 0)
3.3. Extensive Games 45

A
i
B
c
c i

1 2 0




2 1 0

Figure 3.3: To clean or not to clean: Game E1

The two Nash equilibria of this game are indicated in bold. In the lower left
equilibrium, A ignores the problem and B cleans the room (payoff (2, 1)). In the
other equilibrium B plans to ignore the problem and A cleans the room (payoff
(1, 2)). Are both of these Nash equilibria equally good? Many people tend to say
‘no’. The reasoning is as follows. Each decision node of an extensive game can
be seen as the starting point of a smaller extensive game. Such a game is called
a subgame of an original game. One would expect each agent to act rationally
in each subgame. If an equilibrium has such a property, it is called a subgame
perfect equilibrium.

3.3.9. Definition. Let F = (Σ, H, turn) be a game form, and h ∈ H. The

sub-‘game form’ of F starting at h is defined as subg(F, h) = (Σ, H 0 , turn 0 ) where
H 0 = {h0 |h · h0 ∈ H} and turn 0 is the same as turn but with the domain restricted
to H 0

This definition can be extended to interpreted game forms, games and interpreted
games in a straightforward way. The set of all subgames of a given game is defined
as the set allsub((Σ, H, turn, U)) = {subg((Σ, H, turn, U), h)|h ∈ H}.

3.3.10. Definition. Let G = (Σ, H, turn, U) be an extensive game, f a function

that reduces extensive games to strategic games. The strategy profile ~s is a
subgame perfect equilibrium if for all subgames G0 of G it is the case that ~s is a
Nash equilibrium of f (G0 ).

The example game of figure 3.3 has one subgame perfect Nash equilibrium in
which agent B cleans the office. The other Nash equilibrium is not subgame
perfect, since it is not optimal for agent B to choose the action ignore if A does
not clean the room.

3.3.11. Theorem (Kuhn). Every perfect information game has at least one
subgame perfect equilibrium in pure strategies [64].
46 Chapter 3. Game Theory

A
y n
B B
B
y y
n n
B C

Figure 3.4: An imperfect information game form FI

If the utility function U of a game G is such that for any agent X and history
h ∈ Z(h) we have that h 6= h0 ⇒ UX (h) 6= UX (h0 ) then the optimal action a in
a certain situation is always unique, and thus the subgame perfect equilibrium is
unique.

3.3.2 Imperfect Information

In imperfect information games it is possible that an agent X does not see the
difference between histories h and h0 . Thus when agent X is in the situation
represented by h, it considers it possible that it might be in the situation h0 .
We use equivalence relations ∼X to store this information, and write h ∼X h0
to indicate this lack of information. For instance in a game of Poker, agent X
may not know the hand of cards that an opponent Y holds. If the only difference
between situations h and h0 would be the hand of cards of Y , then h ∼X h0 .

3.3.12. Definition. An imperfect information game form F is a tuple F =

(Σ, H, turn, ∼), where Σ is a finite set of agents, H is a non-empty, prefix-closed
set of finite sequences, turn is a function turn : H \ Z(H) → Σ, for each X ∈ Σ
the relation ∼X ⊆ H × H is an equivalence relation between states. Furthermore
∼X has to satisfy the following condition: if turn(h) = X and h0 ∼X h then also
turn(h0 ) = X and A(H, h) = A(H, h0 ).

An example imperfect information game form is displayed in figure 3.4. In

this figure, the lack of information is indicated by dashed lines. Hence agent B
cannot distnguish the histories y and n, and thus has no information what action
agent A has chosen: y ∼B n.
It often happens that an agent X has to make decisions in situations h and h0
that it cannot distinguish, i.e. h ∼X h0 . Since X cannot see a difference between
these situations, strategies for X must prescribe the same behaviour in both
situations. The definitions of the different strategies thus have to be modified.

3.3.13. Definition. A strategy σ for X in game form F = (Σ, H, turn, ∼, π) is

uniform if for all h ∼X h0 it holds that σ(h) = σ(h0 ).
3.3. Extensive Games 47

This modification can be applied to all kinds of strategies: pure, behavioural

and nondeterministic. For imperfect information games we only consider uniform
strategies, even if the word uniform is not mentioned.
An example of a uniform pure strategy σB for agent B in game form FI would
be σB (y) = σB (n) = y. The strategy σB (y) = y and σB (n) = n would not B
uniform, and thus this would not be an acceptable strategy.
For games of imperfect information we use a different notion of a subgame.

3.3.14. Definition. Let F = (Σ, H, turn) be a game form, and h ∈ H. Let

F 0 = (Σ, H 0 , turn 0 , ∼0 ) where H 0 = {h0 |h · h0 ∈ H} and turn 0 , ∼0 are the same as
turn, ∼ but with their domain restricted to H 0 instead of H. The structure F 0 is
a subgame form iff for all j ∈ H 0 and X ∈ Σ it is the case that j ∼X j 0 implies
j 0 ∈ H 0.

Any decision node h0 in a subgame should only be indistinguishable from histories

that are also in the subgame. This means that all agents ‘know’ that they are
in the subgame, and anything that is outside the subgame does not influence the
agents’ decisions. Some imperfect information games have subgames, but others
do not have any because certain information is private from start to end. The
notion of a subgame perfect equilibrium is therefore not often used on imper-
fect information games. Instead, people use extensions such as the sequential
equilibrium or the trembling hand perfect equilibrium [79].
It is often reasonable to suppose that agents remember information. This
means that if two histories h, h0 can be distinguished, then any pair of extensions
of these histories can also be distinguished: h X h0 implies h · h2 X h0 · h02 .
This property is called perfect memory. Another useful assumption to make is
that agents remember their own decisions. Thus, if turn(h) = X then ha ∼X ha0
implies a = a0 . If a game form has this property and perfect memory then the
game form has perfect recall .

Examples

Agent C in game form FI does not ave perfect memory. The agent can distinguish
y and n (thus y 6∼C n), but it cannot distinguish yn and nn (hence yn ∼C nn).
In the game form FI , agent B can forget its own action, since yy ∼B yn. Thus
the game form FI does not have the property and hence does not have perfect
recall.
The assumption of perfect recall makes it easier to compute Nash equilibrium
strategies. In fact, for two player constant-sum games with perfect recall, one
can find Nash equilibrium strategies in polynomial time. If perfect recall is not
assumed, the problem is Σ2 P-complete and thus believed to be intractable [57].
48 Chapter 3. Game Theory

3.4 Existing Work on Logic and Games

Logicians and game theorists often work on the same problems, and their respec-
tive fields are becoming more and more connected due to the efforts of many
researchers in both fields [97].
It is surprising to see that logic and games can be connected in many different
ways.
Logic can be used to understand and make transparent the reasoning behind
game-theoretic solution concepts. In this case logic is a tool used to understand
the assumptions made in game theory. The focus is often on the knowledge that
is required for agents in order to ensure that a certain outcome is reached. Exam-
ples of work in this direction include Aumann’s discovery that a Nash equilibrium
can arise without common knowledge [7], or De Bruin’s analyses of iterated elim-
ination of dominated strategies [30].
Epistemic logic and its extensions can be used to understand situations that
occur in imperfect information games. A typical example is the game of Clue.
In order to play this board game one must reason about knowledge of cards, and
thus this game lends itself well to modeling using dynamic epistemic logic [105].
Probabilistic epistemic logic is useful for modeling games in which probabilities
play a role [59]. The dynamic epistemic logic approach can be extended to include
even complex game actions such as cheating and deceiving other players [8]. The
focus in this area of research is on the imperfect information of players.
The subgame perfect Nash equilibrium, also known as backward induction,
is the most popular solution concept for extensive games of perfect information.
Modal logicians are always interested in determining the expressivity of modal lan-
guages, and at least two authors have thus determined what language one needs
to characterise this solution concept. Bonanno [14] has given a characterisation
of backward induction using branching time temporal logic. Harrenstein [45] has
used a different multi-modal logic.
Since a game form lacks preferences, one cannot ask what agents want in a
game form, or which agents will win in a game form: the concept of winning is not
defined if there are no preferences. One can however investigate the effectivity of
coalitions of agents: whether agents can, by choosing the right strategy, ensure
that a certain outcome holds. This can be formalized in logic. Pauly’s coalition
logic [85] and Van Benthem’s logic for process models [99] do exactly this. These
logics are suitable for reasoning about what agents, or coalitions of agents, can
achieve by their choice of strategy. This is called effectivity. Since both these
logics are closely related to the work in this chapter, they are introduced in more
detail in section 3.4.1.
In order to study games instead of game forms, one must introduce the notion
of preferences. This leads to the idea of preference logic. Our work on preference
logic goes back to Von Wright [119]. Von Wright used an intuitive approach,
based on identifying likely axioms. Van Dalen [100] subsequently proved several
3.4. Existing Work on Logic and Games 49

completeness results for certain basic semantics. Rescher [86] also developed a
semantic for Von Wright’s language, this time based on the fact that the value
of a formula φ (in a certain model) should be the average of the utilities of all
worlds in the model that satisfy φ. This is certainly an interesting idea, but hard
to characterize in an axiomatic way. Chisholm and Sosa [20] investigated the
philosophical aspects of preference logic further. A recent overview is presented
by Hansson [44, 320].
What these sources have in common is that they use a binary construction
φP ψ in order to express preferences. This is different from the usual modal logic
approach, which is normally based on an unary operator 2φ (see for instance [45]).
The binary approach seems to correspond better with natural language, where
one can say things such as “I prefer coffee over tea”. The sort of relations that
is used to indicate preferences can also be used to express relative likelihood [42].
As a result, the technical results stated in terms of likelyhood can be applied
to preferences. Huang [52] also introduces preference logics, in order to model
agents with bounded rationality.
Another way to combine logic and games is to interpret a formula as a game
between two players, one of which wants to reach a ‘true’ outcome, the other one
a ‘false’ outcome. This idea has been used in the interpretation of independence
friendly logic (IF logic). This logic was introduced by Hintikka with the bold
goal of replacing first order logic as the primary logic of scientific discourse [49].
This idea has not materialized, partly because IF logic is quite complicated, and
has several interesting ‘features’ (that some call ‘bugs’). It has a compositional
semantics but it is not the simplest semantics [50], for this logic the falsity condi-
tions are not the mirror image of the truth conditions [31], and one can question
whether the interpretation of IF is faithful to game theory [94].

3.4.1 Coalition Logics

Coalition logic [84, p.46] is a logic for reasoning about effectivity in general game
frames. The language is very similar to efl (to be defined on page 58), and thus
coalition logic is examined here in detail first.
3.4.1. Definition. Assume that finite sets P, Σ are given. A coalition logic
formula φ is defined by the following rule. In these rules p ∈ P and Γ ⊆ Σ.

φ ::= p | φ → φ | ⊥ | [Γ]φ

Coalition logic is interpreted over general game frames, which combine features
of strategic games and extensive games. A general frame is similar to a tree, but
at each decision node all agents have to select an action, like in a strategic game.
The next state depends on the actions chosen by all agents. This semantics is
described in detail in Pauly’s dissertation [84, p.46]. This class of models is more
general than the interpreted game forms that are used for efl. The interpretation
50 Chapter 3. Game Theory

of a coalition logic formula φ over a general game frame G in state s is defined

below.

3.4.2. Definition. A coalition model M is a tuple (S, {EΓ |Γ ⊆ Σ}, π) where S

is a set of states, π : S → P an interpretation function, and for each coalition Γ
S
we have EΓ : S → 22 is a function that to each s ∈ S assigns a set of sets of
states. The functions EΓ must be monotonic, i.e. if T ∈ EΓ (s) and T ⊂ T 0 then
T 0 ∈ EΓ (s)

The following rules define the interpretation of coalition logic formulas over
pointed models M, s, where s ∈ S is a state in the coalition model M
G, s |= ⊥ never
G, s |= p for p ∈ π(s)
G, s |= φ → ψ iff not G, s |= φ or G, s |= ψ
G, s |= [Γ]φ iff φG ⊆ EΓ (s)
where φG = {t ∈ S|G, t |= φ}
This logic does not make a distinction between intermediate states and end states
or outcomes. The internal structure of the protocol can therefore be described in
coalition logic, and one can use satisfiability for protocol verification.
The fact that coalition logic does take intermediate states into account, means
that one should read formulas from efl and coalition logic in a different way. In
chapter 4 we see that the efl formula [A]2p expresses that A can enforce that
a p outcome is reached. Syntactically the closest coalition logic formula is [A]p.
This formula means A can make p true in the next state. If the next state is not
an outcome state, then this formula does not say anything about which outcomes
A can reach. In order to express something about outcomes, one can however
use extended coalition logic. This logic is an extension of coalition logic with
operators [Γ∗ ]φ and [Γ× ]φ. The first operator expresses that φ can eventually be
reached by Γ, and the second operator [Γ× ]φ expresses that Γ can keep φ true in
the entire future. The first operator can be used to refer to outcome states, and
indeed Pauly introduces a special notation to do so.

[Γt ]φ = [Γ× ]([∅]⊥ ∧ φ)

def

Proof Theory
There are sound and complete proof systems for several variantes of Coalition
logic. The full details are given by Pauly [85, 85]. Here we present as an example a
sound and complete proof system for coalition models that have a weakly playable
effectivity function.
S
3.4.3. Definition. An effectivity function EΓ : S → 22 is Γ-maximal iff for all
T , if S \ T ∈
/ EΣ\Γ then T ∈ EΓ
3.4. Existing Work on Logic and Games 51

S
3.4.4. Definition. An effectivity function EΓ : S → 22 is superadditive iff
for all T1 , T2 , Γ1 , Γ2 such that Γ1 ∩ Γ2 = ∅, if X1 ∈ EΓ1 and X2 ∈ EΓ2 then
X1 ∩ X2 ∈ EΓ1 ∪Γ1
S
3.4.5. Definition. An effectivity function EΓ : S → 22 is weakly playable if it
/ E(Σ), (2) if ∅ ∈ E(Γ) and Γ0 ⊂ Γ
satisfies the following five conditions: (1) ∅ ∈
then ∅ ∈ E(Γ0 ), (3) If ∅ ∈/ E(∅) then S ∈ E(Γ) for all Γ ⊆ Σ, (4) E is Σ-maximal
and (5) E is superadditive.

These following axioms are sound on coalition models with weakly playable
effectivity functions.

|= ¬[Σ]⊥ (N ⊥)
|= [Γ ∪ Γ2 ]⊥ → [Γ]⊥ (⊥)
|= ¬[∅]⊥ → [Γ]> (>)
|= ¬[∅]¬φ → [Σ]φ (N )
|= ([Γ1 ](φ1 ) ∧ [Γ2 ](φ2 )) → [Γ1 ∪ Γ2 ](φ1 ∧ φ2 ) (S)
where Γ1 ∩ Γ2 = ∅

There are two reasoning rules for coalition logic. The first is Modus Ponens, the
second one is called Monotonicity.
φ↔ψ
[Γ]φ ↔ [Γ]ψ
The proof system consisting of these two rules and the five axioms is sound and
complete on coalition models with weakly playable effectivity functions [84, p.
55].
Coalition logic can be used for reasoning about extensive games. In that case
the following formula, valid on extensive game forms, should be added as an
axiom. _
[Σ]φ → [X]φ
X∈Σ

This formula can be read as saying that if something can be done, it can be done
by one of the agents.
Pauly presents several completeness proofs for different classes of models, and
for a detailed presentation we refer to his dissertation [84, p. 54]. The most
general proof is similar to the standard completeness proof of modal logic based
on a canonical model.
For extended coalition logic, a complete axiomatization is also given, and this
axiomatisation was later used by Goranko [40, 41] to develop a complete proof
system for ATL. These proof systems are more complex than the system given
here for efl, since these proof systems deal with systems with infinite runs. Our
logic efl is only a small fragment of these logics.
52 Chapter 3. Game Theory

3.4.2 Power Level Logic and Bisimulation

Another language for reasoning about what agents can effect has been introduced
by Van Benthem [99]. Van Benthem introduces the notation {G, X}φ, where X
is a single agent and G refers to a strategic game. This operator is interpreted in
the following way.

M, s |= {G, X}φ ⇔ ∃S : ρX
G s, S ∧ ∀t ∈ S : M, t |= φ

The relation ρX G s, S is interpreted as saying that agent X has a strategy for

playing game G from state s onwards such that all next states are within the set
S. Van Benthem remarks that the argument G can be omitted if the game does
not change, and this makes the language even more similar to efl.
This language allows one to write {A}{B}φ. This does not add to the expres-
sivity of the language, since the second operator can be omitted without changing
the meaning of the formula.

|= {A}{B}φ ↔ {A}φ

Variants on this language, presented in the same paper [99], combine this language
with features of coalition logic.
As explained on page 18, the notion of bisimulation is used in standard modal
logic in order to decide when two models are the same. For the logic described
here, one cannot use the same definition directly. Instead Van Benthem defines
the notion of a power bisimulation.
3.4.6. Definition. Suppose two models M and M 0 with sets of worlds W, W 0
are given. A binary relation E ⊆ W × W 0 is a power bisimulation if the following
conditions hold.
• If (x, y) ∈ E then they satisfy the same atomic propositions
• For any agent X, if (x, y) ∈ E and ρX M xU then there is a set V such that
ρX
M 0 yV and ∀v ∈ V ∃u ∈ U : (u, v) ∈ E
• Vice versa: For any agent X, if (y, x) ∈ E and ρX
M yV then there is a set U
X
such that ρM 0 xU and ∀u ∈ U ∃v ∈ V : (u, v) ∈ E
This definition captures the idea that agents have the same abilities in the models
M and M 0 . Two models that are power bisimilar satisfy the same formulas [99].

3.4.3 Alternating-time Temporal Logic

Alternating-time Temporal Logic (ATL) is a multi-agent extension of CTL [6].
The language of ATL contains temporal operators similar to CTL, but instead
of the quantifiers ∀ and ∃ that appear in CTL, strategy operators hhΓii are used,
where Γ can be any set of agents.
3.4. Existing Work on Logic and Games 53

3.4.7. Definition. Let Σ be a set of agents, and P a set of atomic propositions.

The logic ATL contains formulas φ generated by the following rule. In this rule,
p is a typical element of P and Γ ⊆ Σ

φ ::= p | φ → φ | ⊥ | hhΓiiψ
ψ ::= 2φ | φ U φ

The meaning of a formula hhΓiiφ is that the agents in Γ can use a strategy such
that φ holds.
This logic is interpreted over alternating transition systems [6]. These are
defined as tuples (P, Σ, Q, π, δ). As usual P is a set of atomic propositions and Σ
a set of agents. The set Q is a set of states the system can be in, and π : Q → P
Q
adds propositions to these states. The function δ : Q × Σ → 22 assigns to each
agent in each state a set of sets of states. Each agent can choose one set of states,
and the next state of the system will be from that set.
An example would be a system where Q = {0, 1, 2, 3, 4}. Suppose that
δ(0, X) = {{1, 2}, {3, 4}} and δ(0, Y ) = {{1, 3}, {2, 4}}. Agent X can now choose
{1, 2} and Y can choose {2, 4}. They make these choices simultaneously. The
next state of the system will be 2, because that is the only common state in their
chosen sets. It is necessary to put some constraints on δ so that a next state can
always be chosen.
The interpretation of this logic uses the notion of strategy to interpret the
coalition operator hhΓii. A strategy for Γ is any function that makes a choice
σΓ (X, q) ∈ δ(q, X) for any agent X ∈ Γ in any state q ∈ Q. Based on a strategy
σΓ , one can define the set of possible walks W(σΓ ) through Q so that all choices
for agents X ∈ Γ are made as recommended by the strategy. This set of walks is
used in the following interpretation of ATL.

M, q |= ⊥ never
M, q |= p where p ∈ P iff p ∈ π(v)
M, q |= φ → ψ iff M, q |= φ implies M, q |= ψ
M, q |= hhΓiiφ iff ∃σΓ : ∀w = v... ∈ W(σΓ ) : M, w |= φ

M, w |= 2φ iff ∀n > 0 : M, w(n) |= φ

M, w |= φ U ψ iff ∃m > 0 : M, w(m) |= ψ and
∀m > k > 0 : M, w(k) |= φ

The model checker Mocha can be used to verify ATL properties of system speci-
fications [5].
54 Chapter 3. Game Theory

3.4.4 Dynamic Epistemic Logic

Strictly speaking, dynamic epistemic logic is not a game logic because its defini-
tion does not make any reference to games at all. It is however frequently applied
to game-like situations [105], and it seems to be the right tool to model knowledge
change in imperfect information games.
Dynamic epistemic logic is an extension of epistemic logic and contains the
usual logical connectives, ordinary modal operators KX φ, and update operators
[φ]ψ.

3.4.8. Definition. Assume that finite sets P, Σ are given. A dynamic epistemic
logic formula φ is defined by the following rule. In these rules p ∈ P and X ∈ Σ.

φ ::= p | φ → φ | ⊥ | KX φ | [φ]φ

Formulas are interpreted over a pointed epistemic models M, w. The ordinary

operators are interpreted in the same way as in epistemic logic. The construct
[φ]ψ is interpreted by first computing an updated model M φ and then determining
whether M φ |= ψ.

3.4.9. Definition. Let M = (Σ, W, ∼, P, π) be an epistemic model and w ∈ W .

Dynamic epistemic logic is interpreted in the following way.

M, w |= p iff p ∈ π(w)
M, w |= ⊥ never
M, w |= φ → ψ iff M, w |= φ implies M, w |= ψ
M, w |= KX φ iff ∀(w, v) ∈∼X : M, v |= φ
M, w |= [φ]ψ iff M, w |= φ implies M φ , w |= φ

Where M φ = (Σ, W 0 , ∼0 , P, π 0 ) is defined such that W 0 = {w ∈ W |M, w |= φ},

π 0 : W 0 → 2P is defined by π 0 (w) = π(w) and for all agents X we have ∼0X =∼X
∪(W 0 × W 0 )

In the interpretation of an update formula φ = [ψ]χ, the model M is changed.

We have that M, w |= [ψ]χ if and only if M ψ , w |= χ. The model M ψ is a model
for the situation that you get if you update M with the information ψ. In the case
of dynamic epistemic logic, this update is done by removing from M the worlds in
which ψ does not hold. The updates in dynamic epistemic logic can be compared
to announcements, because if you announce a simple formula p, then everybody
knows p afterwards: W, w |= [p]p. For more complicated formulas this does not
hold, consider for instance φ = [p ∧ ¬KB p](p ∧ ¬KB p). After the announcement,
B of course knows that p holds, so φ is not a tautology.
A complete proof system for dynamic epistemic logic exists, because one can
use the following reduction axioms to reduce any DEL formula to a formula of
3.4. Existing Work on Logic and Games 55

epistemic logic.

At [φ]p ↔ (φ → p)
PF [φ]¬ψ ↔ (φ → ¬[φ]ψ)
Dist [φ](ψ1 ∧ ψ2 ) ↔ ([φ]ψ1 ∧ [φ]ψ2 )
KA [φ]KX ψ ↔ (φ → KX [φ]ψ)

This reduction method can also be used to obtain a complete proof system for
dynamic epistemic logic with common knowledge [61], and for more complicated
actions involving knowledge and beliefs [9].
Chapter 4
Logics for Protocols

4.1 Introduction
In this chapter, the logic efl is presented, that can be used for reasoning about
multi-agent protocols. The acronym efl stands for effectivity logic, because this
logic can be used to express whether coalitions have strategies that are effective
in achieving certain goals. Thus, the logic contains statements such as [X]φ,
meaning that X can achieve φ. The statement [X]φ does not mean that X wants
φ, but rather that X would have a strategy for φ at hand if it would ever need
one.
As an example of how a logical approach can be useful for people inter-
ested in multi-agent protocols, the following informal situation description is used
throughout this chapter.

Three agents Alice, Bob and Caroline (or A, B and C) have to

select one of the alternatives x, y and z. They are looking for a
suitable voting protocol to select exactly one of these three alterna-
tives as the outcome. The protocol should be democratic, so that
any majority can enforce any outcome x, y or z.

The goal of this chapter is to capture these requirements in logic, and then to
find protocols that satisfy these requirements.
In this chapter, we give several examples, and we have tried to give the smallest
interesting examples of each phenomenon. The following more basic decision
problems are used for these examples.

joint decision problem A decision p can be taken if either A or B thinks that

p should be the case. If both agents do not want p, it should be rejected.

independent decision problem An agent A can decide whether a should hold

or not, and agent B can decide whether b should hold or not.

57
58 Chapter 4. Logics for Protocols

B C

x y z x y z

Figure 4.1: A voting protocol FV

In the next section, section 4.2, the language efl is defined. Section 4.3
discusses the model checking problem. Bisimulation is discussed in section 4.4,
and section 4.5 presents a proof system.
In the second part of this chapter, we look at the different ways in which
one can represent protocols. It is shown in section 4.6 that the way in which
protocols are represented influences the model checking complexity. This is done
by specifying a more efficient way of representing protocols, and proving that the
model checking problem becomes harder when this input format is used. Within
this chapter we also present many alternative protocols for the example voting
problem. The last section, section 4.7, contains conclusions.

4.2 Defining Effectivity Logic

Protocols are modeled in this chapter as (extensive) game forms. In order to use
logical formulas for the properties, these game forms are extended with atomic
propositions. These atomic propositions are added only to the outcome states of
each game form. Such game forms are called interpreted game forms.

4.2.1. Definition. An interpreted game form F is defined as a tuple F =

(Σ, H, turn, P, π), so that (Σ, H, turn) is a game form, P is a finite set of atomic
propositions, and π : Z(H) → 2P returns the true atomic propositions of any
terminal history.

An example game form that represents a protocol for the voting problem is
displayed in figure 4.1. The outcomes are marked with propositions x, y, z. In this
protocol, A decides whether B or C can decide on the outcome of the protocol.

4.2.2. Definition. Assume that finite sets P, Σ are given. An efl formula φ
is defined by the following two rules. In these rules p ∈ P and Γ ⊆ Σ

φ ::= 2ψ | φ → φ | ⊥ | [Γ]ψ
ψ ::= p | ψ → ψ | ⊥
4.2. Defining Effectivity Logic 59

The second line of this definition defines a typical propositional logic formula ψ.
These propositional logic formulas are not efl formulas. They can only appear
as 2ψ or [Γ]ψ. Thus, formulas such as p ∨ ¬p and p → q are propositional logic
formulas, but are not themselves efl formulas.
Propositional logic is interpreted in the usual way. The logic efl is interpreted
over an interpreted game form F = (Σ, H, turn, P, π). The definition makes use
of pure strategies σΓ and updates with these strategies.

4.2.3. Definition. Let F = (Σ, H, turn, P, π) be an interpreted game form and

σΓ a pure strategy for coalition Γ. The updated model F 0 = U p(F, σΓ ) is defined
as F 0 = (Σ, H 0 , turn 0 , P, π 0 ) where H 0 is the unique subset of H such that
• the empty sequence  is a member of H 0

• if h ∈ H 0 and turn(h) ∈ Γ then hσΓ (h) ∈ H 0 , but for all other actions b we
have hb ∈/ H0

/ Γ then ha ∈ H 0 for any ha ∈ H.

• if turn(h) ∈
The new elements P 0 , turn 0 and π 0 are identical to P and π respectively, except
that they are restricted to H 0 .

The idea behind an update F 0 = U p(F, σΓ ) is that it calculates a reduced game

form F 0 , in which no action is taken that is excluded by the strategy σΓ . The
strategy σΓ is only defined for agents X ∈ Γ. The other agents are not restricted
in any way by the strategy. The notion of an update is used in the following
interpretation of efl.

F |= ⊥ never
F |= φ → ψ iff not F |= φ or F |= ψ
F |= 2φ iff ∀h ∈ Z(H) : π(h) |= φ
F |= [Γ]φ iff ∃σΓ ∀h ∈ Z(H 0 ) : π 0 (h) |= φ
where (Σ, H 0 , turn 0 , P, π 0 ) = U p(F, σΓ )

Intuitively, the box 2φ is a universal quantifier. It expresses that φ holds in every

outcome state. The construction [Γ]φ expresses that Γ has a strategy so that if
it uses this strategy, any reachable outcome satisfies φ.
The language efl is expressive enough to express the first two properties that
are required for the example protocol. First of all, it can be used to express that
a protocol F selects exactly one action.

F |= 2(x ∨ y ∨ z)
F |= 2¬(x ∧ y)
F |= 2¬(x ∧ z)
F |= 2¬(y ∧ z)
60 Chapter 4. Logics for Protocols

Secondly, one can express that any two agents can enforce any outcome.

F |= [AB]x ∧ [AB]y ∧ [AB]z

F |= [AC]x ∧ [AC]y ∧ [AC]z
F |= [BC]x ∧ [BC]y ∧ [BC]z

It is not hard to verify that the example protocol FV displayed in figure 4.1
indeed satisfies these formulas. Therefore, there exists a suitable protocol for the
example voting problem.

4.3 Model Checking for EFL

The main point of this section is to show that the model checking problem for
efl is tractable. This is not a very deep point. However, it is valuable in practice
and it serves as a test case for the notation chosen.

4.3.1. Definition. Let F = (Σ, H, turn, U) be an extensive game. For each

agent X the value function v X is defined recursively by v X (h) = UX (h) if h ∈
Z(H) and v X (h) = maxa∈A(H,h) v X (ha) when X = turn(h), and v X (h) = mina∈A(H,h)
v X (ha) when X 6= turn(h).

4.3.2. Lemma. The function v X can be computed in time O(kF k).

Proof. Suppose that we walk through the game tree using a post-order tree
walk [27, p.245]. At each node h, we can compute v X (h) since either h is a leaf,
or we have already computed the value of all children ha of h, in which case we
take the maximum maxa∈A(H,h) v turn(h) (ha) of all children. This walk takes time
O(kF k). 

The value of a game indicates how much payoff agents can expect if they act
optimally. If one knows the value of each node, one also knows which moves are
good. Suppose that h is a nonterminal history and turn(h) = X. Intuitively
an action a is a ‘good’ action iff v X (ha) = v X (h), and thus knowing the value
function helps agents to select the best actions.

4.3.3. Theorem. For a given formula φ ∈ efl and interpreted game form F ,
checking whether F |= φ takes time O(kF k · kφk)

Proof. Assume that a formula φ and a model F = (Σ, H, turn, P, π) are given,
and furthermore assume that F is represented explicitly by listing all elements of
pairs and sets.
Determining for any propositional logic formula ψ and terminal history s
whether π(s) |= ψ can be done in time proportional to kψk. For any formula
4.4. Bisimulation 61

φ = 2ψ, determining whether F |= φ can thus be done in time proportional to

kHk · kψk ≤ kF k · kφk.
Suppose now that φ = [Γ]ψ. We can define a game F 0 = (Σ0 , H, turn 0 , U)
where Σ0 = {Γ, Ξ} and turn 0 (h) = Γ iff turn(h) ∈ Γ. Otherwise turn 0 (h) = Ξ. The
utility function is defined such that UΓ (h) = (1, 0) iff π(h) |= ψ. For each terminal
history, this takes time at most O(kφk), thus computing the whole function takes
time O(kF k · kφk). For this game, one can compute the value function v Γ . If
v Γ () = 1 then F |= φ. Otherwise F 6|= φ. Computing the value function is thus
sufficient for determining whether F |= φ holds. According to lemma 4.3.2 this
can be done in time O(kF k).
For other formulas φ one can prove the theorem by using induction over the
formula structure. The case φ = ⊥ is a trivial case. The base cases are formed
by φ = 2ψ and φ = [Γ]ψ, and we have seen that these cases take time at most
O(kF k · kφk), and the induction hypothesis is that this holds for all formulas.
In case φ = ψ1 → ψ2 , one can see that determining whether F |= φ takes time
O(kF k · kφ1 k) + O(kF k · kφ2 k) ≤ O(kF k · kφk). Since these are all the cases, we
conclude that for any formula φ and game form F the theorem holds. 

Combining logic and game theory can potentially lead to problems with high
complexity, but the result given here shows that this is not always the case. This
model checking problem is easy due to two factors. First of all the preferences of
agents are expressed by means of propositional logic. In this format one cannot
express complicated, higher order preferences. Secondly, this logic essentially de-
scribes two-player constant-sum games, since one group of agents tries to achieve
something under the assumption that the other agents do not cooperate. Two
player constant-sum games with perfect information are well understood and
computing optimal strategies for such games is not computationally costly.

4.4 Bisimulation
An important question is to decide when two protocols are the same. In general,
this is a complicated question, because one can compare protocols with more or
less scrutiny. One way out of this dilemma is to use logical equivalence as a
deciding factor. Given a suitable logic one can define two protocols to be the
same when their corresponding game forms satisfy the same logical formulas.
This leaves us with the problem of deciding when two interpreted game forms
satisfy the same formulas.
For the logic efl one cannot apply the notion of bisimulation of standard
modal logic directly, because it is not clear what the ‘sets of worlds’ are that
should act as domain of the bisimulation relation. For some logics defined on
extensive games one can define a bisimulation between the sets of histories. Each
position in the game tree of the first model is matched by the bisimulation to an
62 Chapter 4. Logics for Protocols

equivalent position in the other model. For efl this idea does not work, because
efl does not use the structure of the game tree directly in its semantics. Two
models in which agents move in a completely different order can still satisfy the
same efl formulas. Thus, for efl one must use a relation similar to the power
bisimulation discussed on page 52.

4.4.1. Definition. Suppose that the two models F = (Σ, H, turn, P, π) and
F 0 = (Σ, H 0 , turn 0 , P, π 0 ) are given. A binary relation E ⊆ Z(H) × Z(H 0 ) is an
outcome bisimulation if the following conditions hold.
• If (h, h0 ) ∈ E then π(h) = π 0 (h0 )

• For any coalition strategy σΓ there exists a strategy σΓ0 such that the fol-
lowing holds: Let Z be the set of terminal histories of U p(F, σΓ ) and let
Z 0 be the set of terminal histories of U p(F 0 , σΓ0 ). Then ∀z 0 ∈ Z 0 ∃z ∈ Z :
(z, z 0 ) ∈ E

• Vice versa. For any coalition strategy σΓ0 there exists a strategy σΓ such
that the following holds: Let Z be the set of terminal histories of U p(F, σΓ )
and let Z 0 be the set of terminal histories of U p(F 0 , σΓ0 ). Then ∀z ∈ Z ∃z 0 ∈
Z 0 : (z, z 0 ) ∈ E

If two game forms are outcome bisimilar, then they satisfy the same formulas.
This is proven in two steps. Below we define which formulas are called basic and
simple. In lemma 4.4.3 we show that two bisimilar models satisfy the same basic
and simple formulas. Lemma 4.4.5 can then be used to show that if two models
satisfy the same basic and simple formulas, they satisfy the same formulas.

4.4.2. Definition. A formula of the form 2φ is basic. A formula of the form

[Γ]φ is called simple.

Basic formulas can be seen as representing global constraints on the possible

outcomes, or as powers of the empty coalition. Each simple formula expresses a
power of a certain coalition.

4.4.3. Lemma. Suppose that the two models F = (Σ, H, turn, P, π 0 ) and F 0 =
(Σ, H 0 , turn 0 , P, π 0 ) are given and that E is a bisimulation between F and F 0 .
Then these models satisfy the same basic and simple formulas.

Proof. Suppose that F |= 2φ. The empty coalition has only one strategy σ∅
and this strategy has the property that U p(F, σ∅ ) = F . The same holds for F 0 .
Take Z as the set of terminal histories of F and Z 0 of F 0 . Take any state z 0 ∈ z.
The second clause of the bisimulation tells us there a bisimilar state z ∈ Z.
Since F |= 2φ and π(z) = π 0 (z 0 ) it follows that π 0 (z 0 ) |= φ. Since z 0 was chosen
arbitrarily, it follows that F 0 |= 2φ.
4.4. Bisimulation 63

Suppose that F |= [Γ]φ. This means that there is a strategy σΓ such that
U p(F, σΓ ) |= 2φ. According to bisimulation one can find a matching strategy
σΓ0 . Take Z, Z 0 to be the terminal histories of U p(F, σΓ ), U p(F 0 , σΓ0 ) respectively.
Take any state z 0 ∈ Z 0 . One can find a state z ∈ Z such that (z, z 0 ) ∈ E. Since
π(z) |= φ and π 0 (z 0 ) = π(z) it follows that π 0 (z 0 ) |= φ. Since z 0 was chosen
arbitrarily, we conclude that F 0 |= [Γ]φ.
Since the definition of outcome bisimulation is symmetric, one can repeat the
argument to show that F 0 |= 2φ implies that F |= 2φ, and that F 0 |= [Γ]φ
implies F |= [Γ]φ. 

In the next lemma, we use specific formulas instead of simple formulas. These
specific formulas are simple formulas such that no stronger simple formulas exists.

4.4.4. Definition. Take any set S of formulas and suppose that φ1 = [Γ]ψ ∈ S
and φ2 = [Γ]χ ∈ S. The formula φ2 is more specific than φ1 if 2(χ → ψ) ∈ S and
2(ψ → χ) ∈ / S. The formula φ1 is specific if there is no more specific formula in
S.

To give an example of a specific formula, take S = {[A]a, [A]b, 2(b → a)}. In this
case [A]b is the only specific formula in S, because this formula is more specific
than [A]a.
The next lemma tells us that it is enough to check only formulas that are sim-
ple and specific to ensure that two maximally consistent sets are the same. Since
the set of all formulas satisfied by a model is always a maximally consistent set,
one can also use this lemma to show that two models satisfy the same formulas.

4.4.5. Lemma. Suppose that S and T are maximally consistent sets. Let S 0
contain all basic and all specific formulas of S and T 0 all basic and all specific
formulas of T . If S 0 = T 0 then S = T .

Proof. Suppose that S and T are maximally consistent sets. Let S 0 contain
all basic and all specific formulas of S and T 0 all basic and all specific formulas
of T . Suppose also that S 0 = T 0 . Let ξ = [Γ]ψ ∈ S. We have to show that
ξ ∈ T . If ξ is specific, then ξ ∈ S 0 , thus ξ ∈ T 0 and ξ ∈ T . If not, then
there is some ‘more specific’ formula [Γ]χ ∈ S so that 2(χ → ψ) ∈ S . This
formula itself need not be specific, since there might be an even more specific
formula that rules out [Γ]χ. However, since P is finite, there is only a finite
number of non-equivalent propositional logic formulas. This means there must
be a specific formula [Γ]χ ∈ S 0 and 2(χ → ψ) ∈ S. Since S 0 = T 0 we know that
[Γ]χ ∈ T 0 and thus [Γ]χ ∈ T . Since 2(χ → ψ) ∈ S is basic, we can conclude that
2(χ → ψ) ∈ S 0 = T 0 ⊆ T . Using the the validity ([Γ]χ ∧ 2(χ → ψ)) → [Γ]ψ and
the fact that T is maximally consistent, we conclude that [Γ]ψ ∈ T .
It is now proven that S and T contain the same simple formulas. Consider now
a formula of the form ¬[Γ]ψ. If ¬[Γ]ψ ∈ S, the validity of the axiom determined,
64 Chapter 4. Logics for Protocols

proven in the next section, can be used to show that [Σ \ Γ]¬ψ ∈ / S. Since this is
a simple formula, we conclude that [Σ \ Γ]¬ψ ∈ / T . Using determined again we
obtain ¬[Γ]ψ ∈ T .
A useful property of maximally consistent sets is that if φ ∧ ψ ∈ S then φ ∈ S
and ψ ∈ S. Moreover if φ ∨ ψ ∈ S then φ ∈ S or ψ ∈ S (or both). For every
formula φ ∈ S in conjunctive normal form we can conclude that φ ∈ T . Since
every propositional formula is equivalent to a formula in conjunctive normal form,
we may conclude that for any formula φ it is the case that φ ∈ S ⇔ φ ∈ T .
Therefore, S = T . 

One can also prove the reverse of lemma 4.4.3.

4.4.6. Lemma. Suppose that the two models F = (Σ, H, turn, P, π 0 ) and F 0 =
(Σ, H 0 , turn 0 , P, π 0 ) are given and that these models satisfy the same formulas.
Then there is a outcome bisimulation E between F and F 0 .

Proof. Suppose that F = (Σ, H, turn, P, π 0 ) and F 0 = (Σ, H 0 , turn 0 , P, π 0 ) are

given. Define a relation E ⊆ Z(H) × Z(H 0 ) by stating that zEz 0 if π(z) = π 0 (z 0 ).
We have to show that this relation is an outcome bisimulation. That the first
condition of definition 4.4.1 holds, follows directly from the definition of E. Below
we show that the second condition holds. The argument for the third condition
is completely parallel to the argument for the second condition.
Take any strategy σΓ on F , and compute Z = Z(U p(F, σΓ )). Suppose that
P = {p0 , p1 , . . . , pn } and that Z = {zV0 , . . . , zm }. Each state zj ∈ Z can be com-
pletely described by a formula ψjW= ni=0 ±i pi where ±i pi = pi if pi ∈ π(zj ), and
±i pi = ¬pi otherwise. Let φ = m j=0 ψj . It follows that F |= [Γ]2φ. Since F
and F 0 satisfy the same formulas, F 0 |= [Γ]2φ. Therefore there exists a strategy
σΓ0 such that U p(F 0 , σΓ0 ) |= 2φ. Now take Z 0 = z(U p(F 0 , σΓ0 )) and take any state
z 0 ∈ Z 0 . Since π 0 (z 0 ) |= φ it must hold that π 0 (z 0 ) |= ψj for some j. Therefore
π 0 (z 0 ) = π(zj ) and thus z 0 Ezj for some zj ∈ Z, which is what we had to show. 

The notion of outcome bisimulation can thus be used to test whether two
protocols have the same properties.

4.5 Completeness
Using the notations from the previous section, one can determine whether two
protocols are equivalent. In this section the focus is on the more difficult problem
of determining whether there exists a protocol F that satisfies a given property
φ. In order to do so, a proof system SEF L is defined, so that one can prove that
certain formulas hold for any model. If SEF L `¬φ, then there is no model F such
that F |= φ. The proof system we present is complete, and thus the opposite also
4.5. Completeness 65

holds: If SEF L 6 `¬φ, then there exists a model F such that F |= φ. The proof
given is constructive, in the sense that it provides a method for constructing such
a model.
First the validity of the formulas that are used as axioms is proven. Then the
proof system SEF L is defined, and the completeness proof is given.
The next table lists four axioms that can be written without the coalition
operator [Γ]φ. These axioms are thus formulas of ‘normal’ modal logic [12].
For the Greek letter τ one may substitute any instance of any propositional
logic tautology that one can obtain using uniform substitution. For instance
[Σ]p∨¬[Σ]p is an instance of p∨¬p. For all other Greek letters one may substitute
any propositional logic formula.
prop = τ tautology
prop2 = 2τ box-tautology
S = 2φ → 3φ seriality
K = 2(φ → ψ) → (2φ → 2ψ) distribution
All instances of these axioms are valid. For the axiom tautology this follows
from the fact that the connectives ⊥, → are interpreted in the same way as in
propositional logic. For the axiom box-tautology, one can remark that the
definition of 2φ uses the semantics of propositional logic. For the axiom seri-
ality, it follows from the fact that each game form must have a non-empty set
of outcome states. The distribution axiom is the same as the standard modal
logic distribution axiom. Its validity can be shown in the same way. This works
because the operator 2 is also defined as a universal operator: 2φ holds if φ is
true in all reachable states. For efl, the reachable states are all the outcome
states.
These axioms are complete for the fragment of efl in which the construction
[Γ]φ is not used. To give a proof sketch: consider the completeness proof of
standard modal logic [12]. One can adapt the completeness proof so that no
nesting of boxes occurs. In that case, the proof exactly matches the language of
efl without [Γ]φ. The remaining axioms for efl are listed below.
C =([Γ]φ ∧ [Ξ \ Γ](φ → ψ)) → [Γ ∪ Ξ]ψ combination
M =[Γ]φ ↔ ¬[Σ \ Γ]¬φ determined
N =[∅]φ ↔ 2φ nobody
4.5.1. Lemma. All instances of combination, determined and nobody are
valid

Proof. Let F be any interpreted game form.

• Take an instance ([Γ]φ ∧ [Ξ](φ → ψ)) → [Γ ∪ Ξ]ψ of combination so that
Γ ∩ Ξ = ∅. Take any model F so that F |= [Γ]φ and F |= [Ξ](φ → ψ). It
66 Chapter 4. Logics for Protocols

follows that there are two strategies σΓ and σΞ such that U p(F, σΓ ) |= 2φ
and U p(F, σΞ ) |= 2(φ → ψ). The two strategies σΓ and σΞ are two functions
with separate domains. One can form a combined strategy σΓ∪Ξ = σΓ ∪ σΞ .
If an outcome state is pruned by either σΓ or σΞ , then it is also pruned by
the combined strategy. Therefore, U p(F, σΓ∪Ξ ) |= 2φ ∧ 2(φ → ψ) and thus
F |= [Γ ∪ Ξ]ψ.

• Take an instance [Γ]φ ↔ ¬[Σ \ Γ]¬φ of determined. One can define a two
player constant-sum extensive game based on the game tree of F between Γ
and Ξ = Σ\Γ so that Γ wins in an outcome s if s |= φ and Ξ wins if s |= ¬φ.
In such an extensive game of perfect information, the two coalitions must
have a winning strategy. Therefore, either F |= [Γ]φ or F |= [Ξ]¬φ.

• Take an instance [∅]φ ↔ 2φ of nobody. First the left to right implication

is proven, then we do right to left. Suppose that F |= [∅]φ This means there
is a strategy for the empty coalition σ∅ so that U p(F, σ∅ ) |= 2φ. The empty
coalition has only one strategy (the function with the empty domain), and
this strategy σ∅ does nothing: U p(F, σ∅ ) = F . Thus, F |= 2φ. For the right
to left implication, assume that F |= 2φ. It follows from U p(F, σ∅ ) = F
that F |= [∅]φ.

One property that one can derive is specificity and can be derived using
combination and nobody. Another property is monotonicity, which follows
from box-tautology, combination and nobody.

([Γ]φ ∧ 2(φ → ψ)) → [Γ]ψ specificity

[Γ]φ → [Γ ∪ Γ2 ]φ monotonicity

4.5.2. Definition. The proof system SEF L consists of the seven axioms tau-
tology, box-tautology, seriality, distribution, combination, deter-
mined, nobody given above and the reasoning rule Modus Ponens.

As an example of how this proof system can be used, assume that we have
three agents (Σ = {A, B, C}) and three propositions P = {a, b, c}. We are looking
for a protocol that has the following properties.

1 2(a ∨ b ∨ c)
2 ¬[AB]2c
3 ¬[AC]2b
4.5. Completeness 67

One can use the proof system SEF L to show that from these three properties, it
follows that ¬[A]2¬a. The following derivation proves this property.

4 [C]2¬c 2, determined
5 [B]2¬b 3, determined
6 [BC]2(¬b ∧ ¬c) 4, 5, combination
7 2((¬b ∧ ¬c) → a) 1, box-tautology, K
8 [BC]2a 6, 7, specificity
9 ¬[A]2¬a 8, determined

4.5.3. Theorem. The proof system SEF L for efl is sound.

Proof. In lemma 4.5.1 it is shown that all axioms are sound. On page 12
it is remarked that the rule modus ponens preserves validity. From these facts
it follows that only valid formulas can be derived, which means that the proof
system is sound. 

The proof system defined here is also complete, and this is proven below in a
constructive sense. This proof differs from the standard completeness proof for
modal logic, that is sketched on page 17. The proof is a bit more complicated
because the semantics of this logic do not refer to single steps in the game trees,
but on the possible outcomes that agents can effect.

4.5.4. Theorem. The proof system SEF L is complete for efl

Proof. We have to show that for each consistent formula φ ∈ efl there is a
model F such that F |= φ. Let a consistent formula φ ∈ efl be given. Let S be
a maximally consistent set so that φ ∈ S and let S 0 contain all basic and specific
formulas of S. Below a model F is constructed so that ∀ψ ∈ S 0 : F |= ψ. Lemma
4.4.5 can then be used to conclude that F |= φ.
The model F we are about to construct is defined recursively using a function
f (C, A, r). The outcome of this function depends on a set of basic and simple
formulas C, on a set of active agents A ⊆ Σ and on a representation function
r : Σ → 2Σ . The set r(X) contains the agents that are represented by agent
X. The model F is defined as F = f (S 0 , A0 , r0 ). Initially, all agents are active
agents: A0 = Σ, and each agent initially only represents itself: r0 (X) = {X}.
The function r can also be applied to coalitions of agents. This is defined by
r(Γ) = ∪X∈Γ r(X). The pair A, r can be used to calculate a new set of simple and
basic formulas S(C, A, r) from a given subset C.

S(C, A, r) = {2ψ | 2ψ ∈ C} ∪ {[Γ]ψ | Γ ⊆ A, [r(Γ)]ψ ∈ C}

68 Chapter 4. Logics for Protocols

The game form f (C, A, r) is defined in the following way. If A contains exactly
one active agent X, then we define a model f (C, A, r) = (Σ, H, turn, P, π) where
H = {, ψ | [X]ψ ∈ C, [X]ψ is specific}. Define turn() = X. If 2ψj ∈ C,
then because of box-tautology 2(ψ → (ψ ∧ ψj )) ∈ C. Using the specificity
property, we conclude [X](ψ ∧ ψj ) ∈ C. Repeating
V this reasoning for any simple
formula ψj ∈ C, we obtain a formula [X](ψV ∧ j ψj ) ∈ C. Let π(ψ) be a set of
atomic propositions such that π(ψ) |= (ψ ∧ j ψj ). One can now show that any
formula χ ∈ C is satisfied by this model.
If A has two or more members, define f (C, A, r) = (Σ, H, turn, P, π) as follows.
Take any agent X ∈ A. Define turn() = X, so that this becomes the acting
agent of the current situation. The set of options A(H, ) consists of two parts:
A(H, ) = E ∪ J. Agent X can thus choose from two different types of actions:
formulas from set E or ‘joining’ an agent from set J.

• The set E consists of all specific choices of agent X: E = {ψe |[X]ψe ∈

C is specific}. These choices lead to a subgame in which the formula ψe
holds in all outcomes. This subgame is defined as f (C 0 , A, r) where

C 0 = {2ψe , 2ψ, [Γ]χ | 2ψ, [Γ ∪ {X}](χ ∧ ψe ) ∈ C}

This definition ensures that ψe holds in the submodel. Axiom combination

ensures that no inconsistent formulas appear in C.

• The set J contains all other active agents: J = {Y ∈ A | Y 6= X}.

These choices Y lead to the subgames f (C Y , A \ {X}, r 0 ) where C Y =
S(C, A \ {X}, r 0 ), and r 0 is such that r 0 (Y ) = {Y, X} and r 0 (Z) = {Z}
for Z 6= Y . Intuitively, choosing Y means that agent Y will now make all
decisions for agent X.

We must show that f (C, A, r) satisfies all formulas in C. This is done using
induction. The induction hypothesis is that submodels of the current model have
this property. The base case is formed by models with one active agent, and this
has been done with above.
First, consider basic formulas, of the form 2φ ∈ C. These formulas are present
in each set C 0 that is used to construct a subgame. Using the induction hypothesis
we know that all outcomes of all choices satisfy φ, and thus f (C, A, r) |= 2φ.
Consider [Γ]ψ ∈ C with X ∈ / Γ. This formula is also present in any set C 0 and
by induction hypothesis we know that there is thus a strategy in each subgame
for Γ to ensure ψ. We can combine these subgame strategies into a strategy σΓ
for the whole game that guarantees ψ, and thus f (C, A, r) |= [Γ]ψ.
Secondly, consider [Γ]ψ ∈ C with X ∈ Γ. If Γ = {X} then there is some spe-
cific ψe ∈ E so that 2(ψe → ψ) ∈ C. This choice leads to a submodel f (C 0 , A, r).
From ψe ∈ C 0 and the induction hypothesis it follows that there is a strategy σX
0
for this submodel that guarantees ψ. Agent X can now use a strategy σX so that
4.5. Completeness 69

A
y n
B B
y n y n
a, b a b
Figure 4.2: A simple game form F1

σX () = {ψe } and within the subgame f (C 0 , A, r), strategy σX 0

makes the same
choices as σX . This strategy guarantees ψ and thus f (C, A, r) |= [Γ]ψ. If there
is more than one agent in Γ, then X can join any of the other agents Y ∈ Γ.
By induction the coalition Γ \ {X} will have a strategy for guaranteeing ψ in the
subgame f (C 0 , A \ {X}, r 0 ), and thus f (C, A, r) |= [Γ]ψ.
The model F = f (S 0 , A0 , r0 ) thus satisfies all formulas ψ ∈ S 0 . From lemma
4.4.5 it follows that F satisfies all formulas in S and thus F |= φ. 

Because this proof is constructive, it provides us with a standard method for

constructing game trees. These trees have a specific format.

4.5.5. Corollary. For any protocol F there is an equivalent protocol F 0 in

which each agent only moves once, and all agents move in a given order.

In figure 4.2 an example interpreted game form is shown. In this game form agent
A first decides whether a should hold or not. Then agent B can decide whether
proposition b should hold or not. A possible story could be that a indicates that
A dresses in black, and b indicates that B dresses in black. The next table lists
properties that are true for the example F1 .

F1 |= [A]a ∧ [A]¬a
F1 |= [B]b ∧ [B]¬b
F1 |= [B](a ↔ b) ∧ [B](a ∇ b)

One can conclude that agent B, because it goes second, can control more. This
corollary can be illustrated for the example protocol of figure 4.2. According to
the proof there should be an equivalent protocol in which agent B moves first.
This is indeed the case, and the protocol is illustrated in figure 4.3. One can see
that B in this case can choose from four options.
In the construction of the proof, each agent has a choice whether it wants to
use one of its abilities (set E) or whether it wants to join a specific agent (set J). In
order to illustrate these two possibilities, consider the property φ3 = [A]p ∧ [B]p ∧
[AB]¬p. There is only one atomic proposition in this example, so P = {p}. There
are only four distinct formulas that one can express: p, ¬p, ⊥, p → p. Suppose
70 Chapter 4. Logics for Protocols

B A
b A ¬b a↔bAa ∇ b p B
A A B
p p ¬p
a ¬a a ¬a a ¬a a ¬a

ab b a ab a b p

Figure 4.3: Alternative F2 Figure 4.4: Game form F3AB

that S is a maximally consistent set containing φ3 . The ability [A]p is more

specific than [A](p → p). Thus, agent A has one specific ability p. In the game
form F3AB that is constructed in the proof, agent A has two options. It can use
this ability, or it can join agent B. The game form is depicted in figure 4.4. In
this protocol agent A and B have exactly the same amount of influence on the
outcome, thus one could call this protocol fair.

4.6 Linear Representations

The definition of a game form does not allow one to compactly specify the inter-
preted game form FV of figure 4.1 (page 58). A more compact format, similar
to the compact formats used in LTL and CTL model checking, is useful. For
this purpose we describe here a new way of representing those game forms, called
linear representation. The idea behind this description method is that a game
tree can be summarized by describing a typical path. Consider for instance the
solution to the independent decision problem given in figure 4.2. This protocol
can be informally described by saying that first agent A chooses whether a should
hold, and then agent B decides whether b should hold. Schematically one would
like to represent this protocol in the following way
... ...
R1 = A −
→B−
→ ...
In this section, such a notation is defined, and used in two ways. First of the
notation is used for giving more examples of protocols. Then, it is used to show
how the model checking complexity of efl depends on the way protocols are
specified.
In this section we need to make a distinction between a description R of
a protocol, and the protocol itself. Suppose that R is a linear representation
of a protocol, such as the string R1 given above. We use pRq to indicate the
protocol denoted by R. We hope that R1 is smaller than the protocol pR1 q that
it represents, and this hope can be expressed as kR1 k < kpR1 qk.

Examples
In this subsubsection we present a few example linear representations
4.6. Linear Representations 71

To start with a simple example, consider the trivial protocol F where agent
A can only chose one action, after which an outcome in which p holds is reached.
Such a protocol F could be described by the linear representation R = A → − {p}.
The set {p} at the end of the representation is the set of propositions that is true
at the end of the protocol. The agent A in front of the arrow indicates that agent
A is the agent that can chose. In this protocol agent A can only chose one action.
It has no real choice, which makes the protocol trivial.
In order to describe more complicated protocols, two additional constructions
can be used. The first one is parallel composition. Suppose that we have two linear
representations R1 = A → − {p} and R2 = A → − {q}. In each of these protocols
agent A can only chose one action, but the two protocols have different outcomes.
We define R3 to be the parallel composition of the two linear representations:
R3 = R1 ||R2 , or:
R3 = (A → − {p})||(A →− {q})
In the protocol described by R3 , agent A can choose two actions, and end up in
either the outcome of pR1 q or the outcome of pR2 q. Thus parallel composition
of protocols gives the starting agents more choice.
The other construction is the use of variables. Take a protocol in which agent
A can choose which of the tree propositions {p, q, r} is true. The agent is allowed
to chose one of these propositions. This protocol can be described by the linear
v∈{p,q,r}
representation R = A −−−−−→ {v}. The symbol v is used here as a variable that
can takes the values p, q or r. The protocol pF q thus has three possible outcomes,
in which either p, q or r holds.
The follow grammar defines how one can form expressions that denote boolean
values, sets, lists and objects. Assume that a set of propositions P = {p0 , p1 , . . .}
and a set of agents Σ = {X0 , X1 , . . .} are given.

Bool ::= Objct = Objct | Bool ∨ Bool

Set ::= ∅ | {List } | Set ∪ Set | Set ∩ Set | Set \ Set | {List ‘|’ Bool}
List ::= Objct | Objct , List
Objct ::= Prop | Set | Ag
Ag ::= X0 | X1 | . . .
Prop ::= p0 | p1 | . . .

All these operators are interpreted in the usual way, except perhaps for {x|φ}.
The x in this example is a concrete object, not a variable. Thus, if pφq holds
then p{x|φ}q = {pxq}, otherwise p{x|φ}q = ∅.
The denotation pEq of an Objct expression E can be computed in polynomial
time, since efficient algorithms for all operations exist. In fact the operations used
are polynomial shrinking (see page 87), which means that the denotation pEq of
an expression E is also not bigger than E: kpEqk ≤ kEk
72 Chapter 4. Logics for Protocols

The next table gives some examples of Objct expressions and their denotations

p{p, q, r}q ={p, q, r}

p{A} ∪ {B}q ={A, B}
p{p|A = B}q =∅

Substitution of variables is needed in order to define linear representations.

Assume that a set of variables V is given,and that v ∈ V. The notation s[v \ x]
is used to obtained by replacing all occurrences of v ∈ V in s by x. Thus,
{3, v}[v \ 1] = {1, 3}.
A linear representation can now be defined recursively, in three steps. First
of all an Objct expression that denotes a set of propositions, like s = {x, y}, is
a linear representation of a protocol. Such a set represents a protocol with no
choices and only one outcome. It serves as a base case. Since computing the
denotation of an expression is a tractable problem, one can always compute psq
in polynomial time O(kskn ) for some n ∈ N.
v∈A
Secondly, one can use an expression of the form X −−→ R(v). Here X is an
agent, v ∈ V is a variable, pAq is a set of objects. This expression denotes a game
form such that agent X can choose any action a from the set pAq, after which
the protocol proceeds with R(a). An example of this construct is the following
p∈{x,y,z}
voting protocol, in which A decides whether x, y or z holds: A −−−−−→ {p}.
a∈A b∈B
Finally, one can join the options of two different protocol X −−→ R1 , X −−→
a∈A b∈B
R2 using the construct (X −−→ R1 )||(X −−→ R2 ). In the resulting protocol, agent
X can choose either an action from pAq or an action from pBq. The following
description of the protocol F3AB displayed in figure 4.4 uses this construct.

p∈{P } B s∈{{p},∅}
R3AB = (A −−−→ {p})||(A −
→ B −−−−−→ s)
Note that one can represent the same protocol in different ways. At the start
of this section as example is given where an agent A can choose whether p, q
or r holds. This protocol F was described using variables as F = pRq where
v∈{p,q,r}
R = A −−−−−→ {v}. One can also use parallel composition to describe the same
protocol. Thus F = pR0 q where

R0 = (A →
− {p})||(A →
− {q})||(A →
− {r})
Note that R is a shorter description than R0 . The more concise description R
is often easier to read, and thus preferred over R0 .
In the next two definitions we formally define what we can allow for a repre-
sentation, and how these representations are translated into game forms. Then
we give more example game forms for the voting problem, and give a complexity
result.
4.6. Linear Representations 73

4.6.1. Definition. Assume a set of variables V is defined and that the finite
sets P, Σ are given. The set of all linear representations is defined recursively as
follows.
• If R is a Objct-expression, such that R denotes a set of atomic propositions
pRq ⊆ P , then R is a linear representation
v∈S
• The construct X −−→ R is a linear representation if the following conditions
are met. We demand that X ∈ Σ, that v ∈ V is a variable, that S is a linear
representation such that pSq is a set of objects, and that for all si ∈ pSq
we have that R[v \ si ] is a linear representation.
v∈S v∈S
• If R0 = X −−−→ 0
R00 and R1 = X −−−→ 1
R10 are linear representations and
pS0 q ∩ pS1 q = ∅ then R0 ||R1 is a linear representation.

Using this definition we can now fill in the details in the linear representation of
the independent decision problem given above.
s1 ∈{{a},∅} s2 ∈{{b},∅}
R1 = A −−−−−−→ B −−−−−−→ s1 ∪ s2
Thus, agent A chooses whether atomic proposition a appears in s1 , agent B
chooses whether b appears in s2 , and the final outcome of the protocol is the
union of their respective decisions. If one had 100 agents, then this method of
specification would be much more efficient than a description in tuples and sets.
The next definition defines a function f F that translates linear representations
into interpreted game forms. This function is defined in the following way.
4.6.2. Definition. Assume the finite sets P, Σ are given, Define f F (R) = f1F (, R),
where f1F is the function defined below. Let h be a sequence of actions.
• If pRq ⊆ P then f1F (h, R) = (Σ, {h}, ∅, P, π) where π(h) = pRq.
s∈S
• Assume R = X −−→ R0 is a linear representation. For any si ∈ pSq compute
= f1F (hsi , R0 [s \ si ]).S The result f1F (h, R)Sis defined
(Σ, Hi , turn i , P, πi ) S
F
as f1 (h, R) = (Σ, i Hi ∪ {h}, turn, P, i πi ) where turn = ( i turn i ) ∪
{(h, X)}.

• f1F (h, R0 ||R1 ) = (Σ, H0 ∪H1 , turn 0 ∪turn 1 , P, π0 ∪π1 ) where (Σ, Hi , turn i , P, πi ) =
f1F (h, Ri )
The voting protocol FV has the following linear representation.
X∈{B,C} p∈{x,y,z}
R1A = A −−−−−→ X −−−−−→ {p}

It says exactly what the protocol is : A chooses between B and C, which in turn
chooses his favorite alternative.
74 Chapter 4. Logics for Protocols

4.6.3. Fact. For any linear representation R, f F (R) is an interpreted game form.

Proof. An induction on R proves this fact. 

One of the uses of linear representations is to describe example protocols

succinctly. Another use is to show how the complexity of model checking depends
on the choice of input format. The linear representation of an interpreted game
form can be more compact than a naive representation of a game form. If one
specifies the input using linear representation, the efl model checking problem
has a high computational complexity.
4.6.4. Theorem. Deciding whether an efl formula φ holds on a linearly repre-
sented game form F is PSPACE-complete.

Proof. In the proof of theorem 4.3.3 and lemma 4.3.2 it is explained how model
checking efl depends on the ability to do a post-order tree walk. If we can do
such a post-order tree walk in polynomial space, then we can model check efl
formulas in polynomial space. An algorithm for such a walk typically uses a
stack. On this stack a description of the current node is stored, after which the
description of a successor is computed, which is also stored on the stack, etcetera,
until a final node is reached. We therefore show the following facts.
• For a linear description R of a terminal node and a propositional logic
formula φ, one can determine whether pRq |= φ in polynomial time. This
follows immediately from the assumption that one can compute the set
of atomic propositions pRq in polynomial time. A naive polynomial time
algorithm is to compute pRq and then determine whether pRq |= φ. Hence
this takes at most b(kRk + kφk) for some polynomial bound R
• Each linear description of a successor of R is smaller that R itself. This is
easy to see. In the case of R = R1 ||R2 , both R1 and R2 are smaller than
v∈pSq
R. In the case of R = X −−−→ R0 (v), it is also clear that kR0 (v)k ≤ kRk.
Thus, each element on the stack needs as most memory kRk
• The maximal number of descriptions on the stack is also bounded by kRk,
because the maximal depth of the model denoted by R is at most the number
of arrows that occur in R.
From the last two facts one can compute that one needs at most kRk · kRk
memory for the stack. Therefore, the total amount of memory that one needs
to determine whether φ holds on pRq for a linear representation R is less than
kRk2 + b(kRk + kφk), and thus is polynomial.
It remains to be proven that the problem is PSPACE-hard. This can be done
by reducing the QBF problem described on page 31 to the efl decision problem.
4.6. Linear Representations 75

Assume that a QBF formula ∀x1 ∃x2 ∀x3 . . . ∀xn φ is given. We have to con-
struct an equivalent efl decision problem, consisting of a representation R and a
formula φ0 . Let Σ = {X, Y } and P = {xi |0 ≤ i ≤ n}. The atomic proposition x0
is a dummy atomic proposition, since it does not appear in φ. The representation
of the interpreted game form is the following:
v1 ∈{x0 ,x1 } v2 ∈{x0 ,x2 } vn−1 ∈{x0 ,xn−1 } vn ∈{x0 ,xn }
R = X −−−−−−→ Y −−−−−−→ . . . Y −−−−−−−−−→ X −−−−−−→ {v1 , v2 , . . . , vn }
Take φ0 = [Y ]φ. The agent Y thus makes all existential choices (it tries to
pick values for the xi that make φ true), and agent X is used for the universal
choices. If ∀x1 ∃x2 ∀x3 . . . ∃xn−1 ∀xn φ, then f F (R) |= φ0 and vice versa. 

It has to be remembered that for some protocols without a lot of structure,

the linear representation format is not more efficient. For very irregular protocols
all linear representations can be larger that the game form itself. Nevertheless,
one can give a linear representation of any protocol.
4.6.5. Theorem. For any interpreted game form F = (Σ, H, turn, P, π) where
H consists of sequences of propositions, there is a linear representation R such
that f F (R) = F

Proof. The set H of any game form F consists of sequences of actions. It

does not matter what kind of objects these actions are, as long as they can be
distinguished. In the examples throughout this dissertation, we have used natural
numbers, propositions and agents as actions. In this proof we restrict the action
to be propositions because propositions are part of the Objct notation.
For any subset S ⊆ P of propositions, one can find an expression R such that
pRq = S, by listing all elements of S. For instance if S = {p, q} then one can
simply take R = {p, q}.
For any interpreted game form F = (Σ, {}, turn, P, π) that has only one
outcome, one can take an expression R so that pRq = π(). This expression R
consists of a list of atomic propositions that hold in the single end state  of the
interpreted game form F . This simple case can be used as a base case for an
inductive proof.
Consider now an interpreted game form F = (Σ, H, turn, P, π) that has more
than one outcome, and assume that for all smaller game forms F 0 one can con-
struct representation R0 such that f F (R0 ) = F 0 . Let A(H, ) = {a1 , . . . , an } be
the set of possible first actions, and let X = turn() be the agent to move first.
Define, for each action ai , the interpreted subgame form Fi = subg(F, ai ). By
induction hypothesis, there is a representation Ri such that f F (Ri ) = Fi . Let
v ∈ V be some variable. One can now create a game form R by using the ||
construction.
v∈{a1 } v∈{an }
R = (X −−−−→ R1 )|| . . . ||(X −−−−→ Rn )
76 Chapter 4. Logics for Protocols

B B B
... ...
C C C
x

x x
y z

Figure 4.5: A second voting protocol FV 2

For this linear representation R, it holds that f F (R) = F . Therefore, by induc-

tion, one can find a linear representation R for any interpreted game form F that
uses propositions for actions. 

The new representation format can be used to define more candidate protocols
for the example problem defined in the beginning of this chapter. These new
protocols all satisfy the requirements of the example, while being very different
from protocol FV . Below we define two more protocols, called FV 2 = f F (R2ABC )
and FV 3 = f F (R3x ). Part of the game tree of protocol FV 2 is depicted in figure
4.5.
a∈{x,y,z} b∈P \{a} c∈{a,b} a
R2ABC = A −−−−−→ (B −−−−−→ C −−−−→ {c})||(B →
− {a})
In FV 2 , A and B choose from the three possible outcomes. If they choose the
same outcome, then that is the final outcome. Otherwise C can choose from the
two outcomes they selected.

a∈{x,y,z} b∈{x,y,z} c∈{x,y,z}

R3x = A −−−−−→B −−−−−→ C −−−−−→
{x|{a, b, c} = {x, y, z}} ∪ {a|b = a} ∪ {c|a = c} ∪ {c|b = c}

In our third example FV 3 , the agents A, B, and C vote sequentially for one
of the three outcomes; the outcome that gets the most votes is elected. If A, B
and C disagree then a pre-determined outcome x is elected.
It is not hard to verify that these three protocols satisfy exactly the same
efl formulas. One can thus conclude that these three protocols are equally fair,
and that there is no reason to prefer one of those protocols above the others. This
conclusion seems counter-intuitive, because the protocol R3x seems biased towards
outcome x. If the agents cannot come to an agreement, then outcome x results.
4.7. Conclusion 77

Perhaps efl is the right tool and the bias mentioned for R3x is an irrelevant
detail. It is also possible that the logic efl is not sensitive enough. Both of these
viewpoints are valid, depending on the application one has in mind.
Making more use of the new notation, one can make even more complicated
protocols. Similar to R3x , one can define protocols R3y and R3z , in which y and
respectively z are the default outcomes. One can offer one of the agents the choice
of selecting the default outcome.

d∈{x,y,z}
R4A = A −−−−−→ R3d

The new protocol f F (R4A ) again satisfies the same efl formulas. If one how-
ever thinks that R3x is biased towards x, then one must conclude that R4A is skewed
towards A. The word skew is used here to express that a certain agent is treated
differently in a significant way from other agents, whereas bias means that an
outcome is treated in a different way than the other outcomes. To give another
example of how one can make more subtle protocols, consider the case where
agent A chooses whether B or C selects the default outcome.

S∈{B,C}
R5A = A −−−−−→ R4S

The result is that there are indeed many different protocols for the example
problem. Many of these can be elegantly described using a linear representation,
even when it would have been very hard to draw a picture of the game tree. For
efl however all these protocols are equivalent.

4.7 Conclusion
The logic efl is a high level logic for reasoning about multi-agent protocols. In
this chapter, the problem of finding a good voting protocol has been used as a
motivating example for the construction of such logic. Different protocols have
been presented and modeled as game forms. Using efl we have shown that the
candidate protocols indeed satisfy the requirements of the problem. Using an
effectivity logic such as efl one can thus reason about the powers of agents and
coalitions in protocols.
In order to efficiently discuss multiple protocols, one needs a good way of
representing different protocols. The naive way, as a tuple of sets and functions,
is rather cumbersome. Another option is to specify protocols by means of a
picture of a game tree. However this is also only practical for small protocols. In
this chapter, a new input format is defined, called the linear representation. The
idea behind this format is that one can specify a protocol by describing a typical
execution of the protocol. For the example voting problem, one can elegantly
describe many protocol variants using the linear representation.
78 Chapter 4. Logics for Protocols

One big question is whether verification of multi-agent protocols in efl is

tractable. Thus, one would like to know the model checking complexity of efl.
The answer to this question depends on how one wants to specify the input. If
one allows protocols to be specified in a linear representation, then this problem
is very intractable: It is PSPACE complete. This result does not only say some-
thing about efl, but applies to solving games in general. With a sufficiently
advanced notation, determining the winner of a game is an intractable problem.
For instance winner determination in the games of Go and Geography is also
PSPACE-complete [81, p. 463].
On the other hand, the verification problem becomes tractable if one uses the
naive representation for game forms. The interpretation of the logic efl is defined
in terms of winning a perfect information game, and this problem is not too hard.
Verification can be done in polynomial time. This supports the conclusion that
efl is indeed simple. One can conclude that verification of properties in efl is
a feasible computational problem. Interesting future work is to find out whether
techniques that have been used to speed up model checkers for ATEL and CTL
can be adapted for efl.
In many applications one does not already have a protocol. One only has a
specification and one would like to find a protocol that meets this specification.
This corresponds to the satisfiability problem for efl: one has an efl formula
φ, and would like to know whether there exists an interpreted game form F that
satisfies this formula. This problem is also solvable for efl: a proof system
SEF L has been presented, so that one prove formally which formulas φ cannot be
satisfied. This proof system is thus complete. If no such proof exists, a method
has been sketched that allows one to construct a model. Automated protocol
design on the basis of efl specifications thus seems possible. It is interesting to
note that one has a lot of freedom in constructing these models: one can order
the agents in any way, and construct a model in which the agents make decisions
in this order. This property distinguishes this logic from modal logics that work
on the level of single actions.
One open question, concerning the example protocol, is how one can distin-
guish between all candidate protocols that have been presented in this chapter.
All protocols presented are equivalent under efl. However, it seems that these
protocols should behave differently of one considers more complicated properties.
In the next chapter, extended logics that work under different assumptions are
employed to solve this problem.
Chapter 5
Politeness and Side Effects

5.1 Introduction
Like the previous chapter, this chapter is concerned with reasoning about game
forms, which are seen as models for multi-agent protocols. In this chapter we
extend the logic of the previous chapter. It is assumed that each agent that
participates in a protocol has some private preferences about the outcome of the
protocol. The word ‘preferences’ is used here in a very loose sense, as a synonym
for ‘goal’ or ‘desire’. It is also assumed that these preferences are not determined
by the protocol. Agents can want whatever they want to want. In voting protocols
it is clear that the agent can have its own, private, preferences over outcomes. In
an auction this is less clear: auctions are often analysed under the assumption
that each agent wants to win at the lowest cost. We assume a more general
setting, where agents can also play to lose, or to maximize the amount they pay.
One of the goals of this chapter is to investigate verification of more com-
plicated properties than only the ability to enforce a certain outcome. Three
complications that are being discussed are the following.

• Groups of agents can have coalition preferences. One can express in the
logical languages that A and B together want φ. This means that they try
to reach a certain goal together and are able to cooperate.

• Agents may be interested in nested abilities: an agent can have the ability to
enable another agent to achieve something, or to make sure another agents
is not able to do something. The wish to give other people the chance
to make a decision, is often associated with politeness, we use the phrase
‘reasoning about politeness’ as an informal name for these nested ability
goals.

• We are not only interested in knowing what agents can achieve, but also in
what way they achieve it. Thus, we would like to know whether an agent

79
80 Chapter 5. Politeness and Side Effects

has to spend all his money to win an auction, or whether an agent should
vote for the candidate it like best. Thus, the side effects of acting in a
certain way are also important.

In this chapter, different logical languages are defined, so that we can determine
how considering nesting and side effects affects the analysis of protocols. In total
four languages are defined. The next table lists the languages and their features.
logic nesting side effects
efl
efls

efln

eflns

Chapter Structure
The structure of this chapter is the following. First the logic efls is defined
in section 5.2, and examples for this logic are given in section 5.3. The next
section, section 5.4, contains a theorem stating that for logics of a certain form,
the model checking problem is tractable. It is shown that this theorem can be
applied to efls. Then the question is posed whether there are more expressive
or detailed logics based on efl and efls. In section 5.5, first the language efln
for reasoning about nested abilities is introduced, and we determine the model
checking complexity of this logic. Then a more expressive language eflns is given
for reasoning about both politeness and side effects. The last section, section 5.6
is the conclusion.

5.2 Defining EFLS

5.2.1. Definition. Suppose that Σ and P are finite sets (of agents and atomic
propositions respectively). The language efls consists of formulas ψ generated
by the following rules. In these rules p ∈ P and Γ ⊆ Σ.

φ ::= p | φ → φ | ⊥
ψ ::= [Γ : φ]ψ | 2φ | ψ → ψ | ⊥

This language can be seen as an extension of efl, in the following way. An efl
formula [Γ]φ is equivalent to the efls formula [Γ : φ]2φ.
A formula [Γ : φ]ψ should be read as saying ‘Assume that Γ uses a strategy
that is supporting φ. Then ψ follows’. It is assumed that all agents are aware of
strategies that are used. The strategy that an agent uses can be said to be ‘visible’
to other agents. If one wants to express this idea in the most extreme form, one
could say that we assume that strategies are visible in the same way as people
5.2. Defining EFLS 81

can see what clothes other people are wearing on a certain day. In many real life
settings, finding out what strategies are used in a strategic setting will probably
take a bit more effort, but is not impossible. This visibility assumption is inspired
by the assumption of complete information in game theory, and is compatible with
the idea of a Nash equilibrium. Indeed if one, as a game theorist or as a strategic
consultant, intends to publish books and papers about good strategies (whether
these are chess strategies, marketing strategies, strategies for penalty taking, or
strategies for generating secure random numbers) then such strategies must work
even when public. Even if one does not wish to publish strategies, people can
often observe what action you take and deduce your strategy from this. Thus it is
known what playing styles professional chess players prefer and how professional
football players take penalties. Many professional keepers, including Hans van
Breukelen, for instance relied on Jan Reker’s booklet for this information [117].
The visibility assumption is also inspired by insights from security and cryp-
tography. One can see the definition of specification of a cryptographic algorithm
as a protocol, and the implementation details as a strategy within such protocol.
For instance the protocol for RSA key generation requires one to choose two prime
numbers p and q. An agent has many ways to do this, and common strategies in-
clude using the current time and some keyboard input for generating these prime
numbers (See Schneier [89] for a dicussion of RSA and implementation details).
The implementation details are often public information, since for many security
programs one can obtain the source code.
The effectiveness of an implementation should not lie in the fact that its inner
details are secret (Thus, one should avoid trying to obtaining security through
obscurity [89]). In order to prove that a strategy or algorithm is ‘good’ or ‘safe’,
one should assume that it is known to all opponents that the strategy is used,
and then consider how effective the strategy is. For instance if Microsoft decides
to use a certain encryption mechanism in its web server software, then anybody
with harmful intentions can buy and study the software, and find out what mea-
sures have been taken against attacks. Typically in security one wants to prove
that opponents remain ignorant of private data. If they are ignorant even when
they know the strategy used, they are certainly ignorant when they do not know
the strategy used. This assumption can also be made in the case of imperfect
information games, and indeed a similar argument is given on page 132.
The idea that strategies are ‘visible’ makes the act of deciding to use a strategy
similar to publicly announcing that you use the strategy. In complex statements,
this idea of an announcement can be used informally when reading formulas. The
following examples illustrate how formulas of this logic can be read.
[A : q]2p
This example formula expresses that if A is trying to achieve q, then as a side
effect p will hold for every possible outcome.
[A : q][B : r]2r
82 Chapter 5. Politeness and Side Effects

This example formula can be read as expressing that, assuming after A has de-
cided that it wants q, then B can select a strategy such that r becomes true. The
order of operators in the formula indicates that B knows the strategy of A, and
can use this in its selection of a strategy for r.
The next formula seems to contain contradictory assumptions.
[A : q][A : ¬q]2(¬q)
This formulas expresses that if A wants q, and then it wants ¬q, then ¬q is
guaranteed. In order for this formula to hold on a model F , it must be the
case that A cannot make q true, otherwise it would choose to do so in the first
assumption.
For the interpretation of this logic the following definitions are used.
5.2.2. Definition. Let F = (Σ, H, turn, P, π) be an interpreted game form and
h ∈ H. The reduced model r(H, h) is defined as r(H, h) = (Σ, H 0 , turn 0 , P, π 0 )
where H 0 = {h0 |h · h0 ∈ H} and turn 0 , π 0 are restrictions of the corresponding
elements of F to H 0 .
The next definition redefines the update function U p so that it works on nonde-
terministic strategies. The intuition is that in the updated model U p(F, σΓ ), the
agents in Γ only take actions that are recommended by σΓ .
5.2.3. Definition. Let F = (Σ, H, turn, P, π) be an interpreted game form and
σΓ a strategy for Γ. Define U p(F, σΓ ) = (Σ, H 0 , turn 0 , P, π 0 ) where H 0 is the
greatest subset of H such that ha ∈ H 0 implies h ∈ H 0 and turn(h) ∈ Γ ∧ ha ∈ H 0
implies a ∈ σΓ (h). The functions turn 0 , π 0 are identical to turn, π but restricted
to H 0 .
The next definition defines a strategy σΓe (φ) that is intended to be the least
restrictive, or most general, strategy that Γ can use to achieve φ.
5.2.4. Definition. Let F = (Σ, H, turn, P, π) be an interpreted game form,
Γ ⊆ Σ and φ ∈ Lp . A history j is a φ-effective position (for Γ) iff there is a
strategy σΓ such that for each terminal history h in U p(r(H, j), σΓ ) it is the case
that π(h) |= φ. The most general φ-effective strategy σΓe (φ) is now defined by

e {a|ha is a φ-effective position for Γ} if this set is non-empty
σΓ (φ)(h) =
A(H, h) otherwise
The definition above spells out what we consider a rational strategy σΓe (φ) for a
coalition Γ that wants to achieve φ. The strategy is defined such that it selects
actions a that lead to winning positions. If that is not possible, it selects all
actions. The idea is that coalition Γ tries to guarantee φ in all positions where
it can guarantee φ. This is similar to the notion of a subgame-perfect strategy.
In the definition below we use this strategy for interpreting the logic. Let F =
(Σ, H, turn, P, π) be an interpreted game form. For any formula φ ∈ efls the
relation F |= φ is defined as follows.
5.3. Examples 83

A
b, e a

a, e b

Figure 5.1: Alice and Bob eat cake

F |= ⊥ never
F |= φ → ψ iff not F |= φ or F |= ψ
F |= 2φ iff ∀h ∈ Z(H) : π(h) |= φ where (Σ, H, turn, P, π) = F
F |= [Γ : φ]ψ iff U p(F, σΓe (φ)) |= ψ

The interpretation of these formulas is similar to that of previous update logics,

such as dynamic epistemic logic, discussed in section 3.4.4 on page 54.
In efls, formulas of the form [Γ : φ]ψ can be seen as updates. In order to
determine whether F |= [Γ : φ]ψ, a new model F 0 = U p(F, σΓe (φ)) is computed.
This new model represents the situation after Γ has decided to try to achieve φ.
It holds that F |= [Γ : φ]ψ if and only if F 0 |= ψ. In an update logic, the model
can thus be changed by adding new information to it. How the model changes,
depends on the update function that is used.

5.3 Examples
5.3.1 Alice and Bob eat Cake
Alice and Bob have a cake, and they have agreed to divide it by means of a “cut-
and-choose” protocol [17]. Alice has cut the cake and unfortunately one of the
pieces is bigger than the other. Bob can now choose from three options: he can
select the big piece, select the small piece, or he can say to Alice ‘No, you choose’.
If he lets Alice choose, she can either choose the big piece or the small piece. Both
agents have common knowledge of this protocol. The interpreted game form
protocol corresponding to this situation is displayed in figure 5.1. Proposition a
means that Alice gets the biggest piece, b that Bob gets the biggest piece, and
e means that something has happened that is embarrassing to Alice and Bob,
namely that either Alice or Bob has chosen the biggest piece. In many cultures
this is considered impolite. Using efls one can express relevant properties of
this protocol. First we will provide several efls formulas (A stands for Alice, B
stands for Bob).
84 Chapter 5. Politeness and Side Effects

Figure 5.2: Model U p(M, σBe (¬e))

• [B : ¬e]2a If B does not want either of them being embarrassed, he must

take the smallest piece. Our semantics take a pessimistic view, so Bob
cannot take the risk of letting A choose. Figure 5.2 shows the updated
model U p(M, σBe (¬e)).

• [B : ¬e][A : ¬e]2a This formula is a consequence of the previous example.

It expresses that if B does not want embarrassment and that A does not
want embarrassment, then A gets the biggest piece. This may seem strange,
since there is an outcome in which ¬e and b are true. However, the order
of assumptions is important. The formula expresses that B wishes to guar-
antee the absence of embarrassment, independently of what A does. Two
possible readings of the formula are that he commits himself to his strategy
before he learns that A has the same preference, or that he thinks that this
goal is so important that he does not wish to rely on A for this property.

• [AB : ¬e][B : b]2b In this example, A and B commonly want to avoid

embarrassment, and B also prefers b. If this is the case, B can let A choose
and then A will take the smallest piece. Figure 5.3 shows the updated model
U p(M, {A, B}, ¬e).

• [A : ¬e][B : ¬e][B : b]2b This formula expresses that if A does not want
embarrassment, B does not want embarrassment, and B prefers the biggest
piece then B gets the biggest piece. The behaviour of B is influenced by the
fact that he knows that A prefers to avoid embarrassment. In this scenario
A should try to hide the fact that she has good manners, because it is not
in her advantage if B knows this.

This example illustrates that, by using efls, one can express consequences
of ordering goals in a certain way. There are several interesting side effects men-
tioned in the above formulas.
5.3. Examples 85

A
a

Figure 5.3: Model U p(M, {A, B}, ¬e)

B
p A A
p p ¬p
p

Figure 5.4: Game form F3BA

5.3.2 Joint Decision Problem

In figure 4.4, on page 70, an interpreted game form F3AB is given in which two
agents jointly decide whether p should hold or not. If either agent wants to have
p it should hold, otherwise p is rejected. In figure 5.4 another protocol is given
that satisfies the same efl formulas. In this protocol, the roles of B and A are
reversed. It seems reasonable to assume that agents care who has to give its
opinion first, and therefore one would like to have a logic that can distinguish
these protocols.
The following statements show that efls is such a logic.

F3AB |= [B : ¬p][A : ¬p]2¬p

F3BA 6|= [B : ¬p][A : ¬p]2¬p

The reason the formula does not hold in the second model is that B, because
it moves first in the protocol F3BA , has an informational disadvantage. When it
has to decide it does not know what A will do, and therefore it is not clear that
letting A choose helps towards achieving its goal. The logic efls is thus more
expressive than efl.
86 Chapter 5. Politeness and Side Effects

5.4 Model Checking EFLS

The semantics of efls is based upon the idea that one can interpret the construct
[φ]ψ by updating a model M with φ, and then checking whether ψ holds in
the updated model. Such a semantics, familiar from dynamic epistemic logic
described on page 54, can be called an update semantics. The goal of this section
is to determine the model checking complexity of the logic efls. Instead of doing
it directly, we prove a more general theorem concerning update semantics of a
certain form, and then show that the theorem applies to efls.
Below we give a general definition of an update language. This definition is
suitable for efls, but not general enough for all other update logics. The term
‘update language’ in this section thus does not refer to all logical languages that
use the idea of updates. It refers only to languages to which the given definitions
can be applied. In this section we have given this term a specific interpretation,
using the following definition. Let M be a set of models for a logic, N1 any set of
additional information objects, N2 a set of formulas in another (simpler) language.
We assume that two functions f and g are given, such that f : M × N1 → M
and g : M × N2 → {true, false}. Suppose also that for any n2 ∈ N2 and M ∈ M,
one can check in polynomial time whether g(M, n2 ) holds. One can, based on
these functions f and g, define an update logic Lf g with the following semantics.

5.4.1. Definition. Suppose that M and N are given, and assume that n1 ∈ N1
and n2 ∈ N2 . The update language Lf g consists of formulas ψ generated by the
following rules.

ψ ::= [n1 ]ψ | n2 | ψ → ψ | ⊥

This language is called an update language, because one can interpret this logic
using updates. The function f is used to compute a new model from the current
model. The next definitions captures the idea of an update semantics. In the
next definitions, M ∈ M is a model, n1 ∈ N1 , n2 ∈ N2 and ψ, ξ ∈ Lf g .

M |= ⊥ never
M |= ψ → ξ iff not M |= ψ or M |= ξ
M |= n2 iff g(M, n2 )
M |= [n1 ]ψ iff f (M, n1 ) |= ψ

The following formulas are valid under this semantics.

|= [n](φ → ψ) → [n]φ → [n]ψ

|= ¬[n]ψ ↔ [n]¬ψ

These axioms can be compared to the reduction axioms stated for dynamic epis-
temic logic stated on page 55. The axioms are not identical, and these two axioms
5.4. Model Checking EFLS 87

are not sufficient to eleminate all update operators, but they do help to simplify
formulas.
This semantics is a generalisation of the semantics of efls. For efls, the
set N1 consists of pairs (Γ, φ), but in this general semantics one can update with
anything. The set N2 consists, in the case of efls, of the formulas N2 = {2φ|φ ∈
Lp }. The question is whether such a semantics can be evaluated in polynomial
time. If so, then the model checking problem is tractable, and thus this logic can
be used in practice for protocol verification.
Whether model checking is tractable, depends on the function f . This function
should be easily computable, but it should also not create bigger and bigger
models. If a function f has these two properties, it is called polynomial shrinking.

5.4.2. Definition. A function f is polynomial shrinking iff kf (a, b)k < kak+kbk
and f can be computed in polynomial time.

If the function f is polynomial shrinking, then the model checking problem is

indeed tractable.

5.4.3. Theorem. Suppose that f is a polynomial shrinking function, and that

Lf g is the corresponding update logic. One can check for given M ∈ M and
ψ ∈ Lf g whether M |= ψ within polynomial time.

Proof. Suppose that f is a polynomial shrinking function, and that Lf g is

the corresponding update logic. In order to prove the theorem, an algorithm
and constants a and b must be given, such that the algorithm needs at most
time (kM k + kψk)a + b to determine whether M |= ψ, for any inputs M and ψ.
The algorithm works recursively on the structure of ψ, so four cases have to be
examined. The first two cases are basic cases, in the two other cases we use an
induction assumption.

• If ψ = ⊥, then M 6|= ψ. Returning this answer takes constant time, and

one can take any b larger than this constant time.

• Suppose that ψ = n2 ∈ N2 . It has been assumed that it can be determined

in polynomial time whether g(M, n2 ). Thus, there are constants c and d
such that this takes less time than (kM k + kψk)c + d. Taking a ≥ c and
b ≥ d, it follows that this takes less time than (kM k + kψk)a + b.

• Suppose that ψ = ψ1 → ψ2 . This means that kψk = 1 + kψ1 k + kψ2 k. In

order to determine whether ψ holds, one has to compute whether M |= ψ1
and whether M |= ψ2 . Using the induction hypothesis, and supposing that
a ≥ 2, one can show that this takes less than (kM k + kψk)a + b time.

• Suppose finally that ψ = [n]ψ1 . In order to determine whether M |= ψ,

one has to compute f (M, n) and then check whether f (M, n) |= ψ1 . Since
88 Chapter 5. Politeness and Side Effects

f is polynomial shrinking, there are constants c, d such that computing

f (M, n) takes less than (kM k + knk)c + d. Furthermore, kf (M, n)k ≤
kM k + knk. The induction hypothesis states that determining whether
f (M, n) |= ψ1 takes less time than (kf (M, n)k + kψ1 k)a + b, which is less
than (kM k + knk + kψ1 k)a + b. One can assume that a ≥ 2 and then derive
that both parts of this computation can be done in time (kM k + kψk)a + b.

This theorem can be used to show that update logics that are based on an update
function f with the right properties, have a tractable model checking problem.
In the next theorem it is shown that the efls update function indeed behaves
well (i.e. that it is polynomial shrinking).

5.4.4. Theorem. The function f : (M, (Γ, φ)) 7→ U p(M, σΓe (φ)) is polynomial
shrinking

Proof. The function f takes a coalition Γ ⊆ Σ and a propositional logic

formula φ ∈ Lp , calculates the strategy σΓe (φ) and returns the updated model
U p(M, σΓe (φ)). In order to show that it is polynomial shrinking, we must show
two things. First of all that it is computable in polynomial time. Secondly, that
the output is smaller than the input. The latter is easy: the reduced model
U p(M, σΓe (φ)) contains less states than M , and thus it is smaller. It remains to
be shown that the function f can be computed in polynomial time. In order to
show this, lemma 4.3.2 is used. Define an extensive game F 0 = (Σ0 , H, turn 0 , U)
where Σ0 = {Γ, Σ \ Γ}. Thus, it is a two-player game. The function turn 0 is
defined such that turn 0 (h) = Γ iff turn(h) ∈ Γ. The function U is defined such
that UΓ (h) = 1 if π(h) |= φ, and UΓ (h) = 0 otherwise. It is a constant-sum game,
thus UΣ\Γ (h) = 1 − UΓ (h). According to lemma 4.3.2, the value function v for this
game can be computed in polynomial time. Using the value function, it is not
hard to define the strategy σΓe (φ). If v Γ (h) = 1 then σΓe (φ)(h) = {a|v Γ (ha) = 1}.
If v Γ (h) = 0 then σΓe (φ)(h) = A(H, h). Once one has this strategy, one can use it
to compute the model U p(M, σΓe (φ)) in linear time: one has to apply this function
to every history exactly once. 

A simple model checking program for efls has been implemented. The program
can be found at www.csc.liv.ac.uk/~sieuwert/glp.
The preceding theorem and its proof suggest that one can construct many
logics that can be model checked in polynomial time. Should we not search for
a more expressive logic than efls? At the same time one can wonder whether
polynomial shrinking is indeed a necessary requirement for the construction of
a polynomially model checkable logic. In order to investigate these issues, two
conceivable extensions of efl and efls are defined. The first one, efln, is an
5.5. Extensions of EFL 89

extension of efl that allows one to form nested abilities. This logic allows one
to express interesting properties, but has a very high model checking complexity.
The next logic, eflns, is an extension of efls that is supposed to capture
both nesting and side effects. This logic seems intuitive, but it is hard to give a
proper semantics for this language.

5.5 Extensions of EFL

5.5.1 Model Checking efln
5.5.1. Definition. Suppose that Σ and P are finite sets (of agents and atomic
propositions). The language efln consists of formulas ψ generated by the fol-
lowing rules. In these rules p ∈ P and Γ ⊆ Σ.
φ ::=p | φ → φ | ⊥
ψ ::=[Γ]ψ | 2φ | ψ → ψ | ⊥
The usual connectives of this logic are interpreted as usual, and [Γ]φ holds if
there is a strategy ensuring φ.
F |= ⊥ never
F |= φ → ψ iff not F |= φ or F |= ψ
F |= 2φ iff ∀h ∈ Z(H) : π(h) |= φ where (Σ, H, turn, P, π) = F
F |= [Γ]φ iff ∃σΓ : U p(F, σΓ ) |= φ
The following example formulas illustrate how this logic can be used for the ‘Alice
and Bob eat cake’ example.
[B]([A]2a ∧ [A]2b)
This example expresses that B can let A choose who gets the biggest piece. This
formula does hold for the example: Bob can take the action of letting Alice choose.
[B](¬[A]2a ∧ ¬[A]2b)
The above formula expresses that Bob can make Alice unable to decide. One
might think that Bob can satisfy this goal by making a decision himself, but this
is not the case. Bob should use a nondeterministic strategy, such as selecting any
action, in order to ensure that A cannot determine anything.
[B](¬[B]2a ∧ ¬[B]2b)
This last example sounds strange, because it only makes sense if Bob does not
trust himself: It expresses that Bob can get rid of his own abilities. In a game
theory setting this can be useful: Bob would like to commit himself so that no-one
will try to put pressure on him. This formula does hold in our example: what
Bob has to do is to choose either always a, or commit to b. Both these pure
strategies do the trick.
90 Chapter 5. Politeness and Side Effects

A b A ¬b a↔bAa ∇ b A
a a ¬a ¬a

ab a b
Figure 5.5: A’s strategy for letting B decide

Independent Decision Problem

In chapter 4, the independent decision problem was introduced. In this problem
two agents A and B can each decide on a certain issue. An agent A can decide
whether a should hold or not, and agent B can decide whether b should hold or
not. A first protocol for this problem has been given in figure 4.2 on page 69.
Using efl one can conclude that B has additional powers to decide whether a
and b should have the same truth value, and we have constructed an equivalent
protocol in figure 4.3 on page 70. In efl these protocols are equivalent, and by
enumerating all formulas of the form [X : φ][Y : φ2 ]ψ one can check that these
are also equivalent under efls. Since these protocols still look quite different,
it would be good to have a logic that can distinguish these protocols. It can be
done in the logic efln.
Assume that A thinks very highly of agent B, and that A would prefer it if B
would decide on the value of both a and b. Whether A can transfer its decision
power is expressed by the following formula.
ψ = [A]([B](a ∧ b) ∧ [B](a ∧ ¬b) ∧ [B](¬a ∧ b) ∧ [B](¬a ∧ ¬b))
In the second protocol F2 this formula holds, since A can use the strategy that is
depicted in figure 5.5. On the other hand, this goal cannot be satisfied in game
form F1 in figure 4.2.

5.5.2. Theorem. Deciding whether an efln formula φ holds on an interpreted

game form F is PSPACE-complete, even in the case of one agent.

Proof. The definition of F |= φ can be converted into a naive algorithm. For

interpreting the construction [Γ]ψ one can try all strategies one after another.
This may take some time, but does not take much space: applying a strategy
gives a smaller model. Therefore, this can be done with an amount of memory
that is proportional to the size of the input. Hence the problem is in PSPACE.
It remains to be proven that this problem is PSPACE-hard. This can be done
by reducing the QBF decision problem of page 31 to the efln decision problem.
This reduction is explained in general in this proof, and then illustrated using an
example. The example is discussed on page 92 and displayed in figure 5.6.
5.5. Extensions of EFL 91

The objective of a QBF problem is to decide for a given formula of the form
∀x1 ∃x2 ∀x3 . . . ∃xn−1 ∀xn φq whether this formula holds. The formula φq is a propo-
sitional logic formula with only propositions from the set {xi |0 < i ≤ n}. Assume
that a QBF formula ∀x1 ∃x2 ∀x3 . . . ∃xn−1 ∀xn φq is given. We have to construct an
equivalent efln decision problem.
First we construct an interpreted game form F = ({X}, H, turn, P, π). Define
Pq to be the old set of atomic propositions: Pq = {xi |0 < i ≤ n}. The set
P = {q + , q − |q ∈ Pq } contains twice as many atomic proposition: for every old
proposition q there is a positive occurrence q + and a negative one q − . The set
H is defined as H = {, p|p ∈ P }. Each terminal run thus consists of one atomic
proposition. Naturally, turn() = X and π(p) = {p}.
We can thus construct the required formula φ in the following way. Suppose
that φq is in conjunctive normal form. Define a function f in the following way.
f (¬p) = 3p−
f (p) = 3p+
f (φ ∧ ψ) = f (φ) ∧ f (ψ)
f (φ ∨ ψ) = f (φ) ∨ f (ψ)
This function converts a propositional formula φ into an efln formula. It is
used in order to convert the propositional part φq into efln. The next definition
defines an efln formula ξi that expresses that all propositions xj with j < i have
been assigned a value, whereas the propositions xj with j > i do not have been
given a value yet.
^ ^
− −
ξi = ((3x+ j ) ∇ (3x j )) ∧ ((3x+
j ) ∧ (3xj ))
j≤i j>i

The idea used here is that a nondeterministic strategy for the constructed model
can be seen as a truth value assignment for the original QBF problem. If the
−
strategy includes x+i then xi is true in the corresponding assignment, and if xi is
included in the strategy then xi is false in the corresponding assignment. The part
−
(3x+ j ) ∇ (3xj ) expresses that exactly one of those two actions must be possible,
and thus ensures that the assignment is consistent. The formula ξi expresses that
−
for j > i, both actions must still be possible: (3x+ j ) ∧ (3xj ). This is necessary
because this choice has to be made later.
Next a function g is defined so that F |= g(∀x1 ∃x2 ∀x3 . . . ∃xn−1 ∀xn φq ) iff
the QBF statement ∀x1 ∃x2 ∀x3 . . . ∃xn−1 ∀xn φq holds. The function g is defined
recursively, so that agent X can at each step pick the truth value of exactly one
propositional variable xi .
g(φq ) = f (φq ) if φq ∈ Lp
g(∃xi φ) = [X](ξi ∧ g(φ))
g(∀xi φ) = ¬[X](ξi ∧ ¬g(φ))
92 Chapter 5. Politeness and Side Effects

X
p+ p− q + q −
p+ p− q+ q−
Figure 5.6: The model Fpq

The length of the formula g(φ) is quadratically bounded: kg(φ)k ≤ kφk2 . Thus,
for a given QBF problem, we have constructed an equivalent efln model check-
ing problem in polynomial time. 

To give an example of how the proof works, consider the QBF problem
∀p∃q(p ∨ ¬q) ∧ (¬p ∨ q). The model Fpq of the corresponding model checking
problem is pictured in figure 5.6. As explained at the end of the proof, the corre-
sponding formula φ is rather long, so it is broken down in parts, called ξ0 , ξ1 and
ψ.

ξ0 = (3p+ ∇ 3p− ) ∧ (3q + ∧ 3q − )

ξ1 = (3p+ ∇ 3p− ) ∧ (3q + ∇ 3q − )
ψ = (3p+ ∨ 3q − ) ∧ (3p− ∨ 3q + )
φ = ¬[X](ξ0 ∧ ¬[X](ξ1 ∧ ψ))

One can verify that Fpq |= φ, and thus the QBF problem ∀p∃q(p∨¬q)∧(¬p∨q)
has a positive answer. It is interesting to see in this example that one agent alone
makes things hard for itself, by denying itself certain rights. Since there is only
one agent, this is arguably not even game theory but decision theory. In this
framework the way single agents influence their own abilities is the cause of the
complexity. The extension to multiple agents comes for free.

5.5.2 Model Checking eflns

One advantage of efls over efl is that one can use efls to reason about side
effects in games. A formula [Γ : φ]ψ expresses that striving towards φ has ψ as
a side effect. It can be used to express that maximizing profit diminishes social
welfare or that knowing A’s actions is helpful for B. This feature is not present
in efln. On the other hand efln can be used to express goals for polite and
helpful agents. Within efln one can say that A wants to help B. A logical next
step is therefore to define an even richer language, called eflns, that combines
the features of efls and efln. This language is defined in this section, so that
we can look at possible interpretations of this language.
5.5. Extensions of EFL 93

5.5.3. Definition. Suppose that Σ and P are finite sets (of agents and atomic
propositions), with typical elements Γ ∈ Σ and p ∈ P . The language eflns
consists of formulas ψ generated by the following rules.

φ ::= p | φ → φ | ⊥
ψ ::= [Γ : ψ]ψ | 2φ | ψ → ψ | ⊥

The official reading of a formula [Γ : φ]ψ is that if Γ wants φ, then ψ holds. The
background assumption is again that if Γ forms a certain plan or adopts a certain
strategy, all agents know this strategy immediately (strategies are visible, one
might say).
This language is not literally an extension of all logics presented before, but
one can easily translate efl, efls and eflns formulas into this language. The
efl formula [Γ]φ translates to [Γ : 2φ]2φ. The efls formula [Γ : φ]ψ translates to
the eflns formula [Γ : 2φ]ψ, and the efln formula [Γ]φ translates into [Γ : φ]φ.
These translations preserve the intuitive meaning of each formula.
We would like to define a semantics for this logic on interpreted game forms
M , that is consistent with the semantics of efl, efls and efln. Thus, if for some
model M it holds that M |= [Γ]φ (using the efln semantics), then M |= [Γ : φ]ψ
under the eflns semantics. Another item on the wish list is that the semantics
is an update semantics.
This section contains two results related to possible interpretation of eflns.
First we give a possible interpretation that is consistent with the semantics of
efln, and show that under this semantics the new logic is not more expressive
than efln. Secondly, we show that there is no reasonable update semantics for
this language that is consistent with the semantics of efln. From these results
one can conclude that defining a logic for reasoning about politeness and side
effects at the same time is not trivial.

First Interpretation The following rules define an interpretation for eflns

over interpreted game forms F = (Σ, H, turn, P, π).
F |= ⊥ never
F |= φ → ψ iff not M |= φ or M |= ψ
F |= 2φ iff ∀h ∈ Z(H) : π(h) |= φ
F |= [Γ : φ]ψ iff ∃σΓ : U p(F, σΓ ) |= φ and U p(F, σΓ ) |= ψ
This is a reasonable definition, because it is consistent with efln. The update
operator [Γ : φ]ψ can be read as saying that it is possible that when Γ uses a
strategy for achieving φ, then ψ holds. This is similar in spirit to the interpre-
tation of similar formulas in efls, but not completely the same. For the logic
efls, we defined a unique rational strategy σΓe (φ) that is the most general strat-
egy for achieving φ. In this semantics we do not use a unique rational strategy,
but consider all strategies σΓ such that U p(F, σΓ ) |= φ.
94 Chapter 5. Politeness and Side Effects

The main drawback of this semantics is not that it is unreasonable, but that
it does not bring us more expressivity than we already had. Everything one can
say in eflns is under this semantics equivalent to something one could already
express using an efls formula.

5.5.4. Theorem. For every formula φ ∈ eflns one can find a formula ψ ∈
efln such that for any interpreted game form F the following holds

F |= φ ⇔ F |= ψ

Proof. This claim is proven using induction over the structure of the formula
φ ∈ eflns. The base case is provided by formulas φ = ⊥ and φ = 2φ1 . Both
these constructs appear in both languages and are interpreted in the same way,
so one can take ψ = φ.
For φ = φ1 → φ2 , one can take ψ = ψ1 → ψ2 , where ψ1 , ψ2 ∈ efln are
formulas that are equivalent to φ1 , φ2 respectively. The induction hypothesis
guarantees that these formulas exist.
The difficult case is thus the new construct [Γ : φ1 ]φ2 . This formula holds on a
model F if Γ has a strategy that satisfies both φ1 and φ2 . This can be expressed in
efln using the formula ψ = [Γ](ψ1 ∧ ψ2 ). Again ψ1 , ψ2 ∈ efls are formulas that
are equivalent to φ1 , φ2 respectively. The induction hypothesis again guarantees
that these formulas exist. 

Second Interpretation The previous result shows that if one does not use an
update semantics based on a unique ‘rational’ strategy, it becomes difficult to
express side effects. Saying that some strategy for φ has ψ as a side effect is not
the same as saying that the best or most rational strategy for φ has ψ as a side
effect.
In order to make this last statement, one would like to have is an update
semantics based on an update function f . This function f should return the
model f (F, Γ, φ) that one gets when Γ uses the rational or most obvious strategy
for obtaining φ.
The next theorem shows that one cannot easily find such an update semantics
for eflns. To be precise we show that there is no non-arbitrary update semantics
that is consistent with the interpretation of efls. Any update semantics would
have to make arbitrary choices about which strategies it considers rational.

5.5.5. Fact. There is no reasonable update semantics for the language eflns
that is consistent with the interpretation of efln.
5.5. Extensions of EFL 95

1 2

p q

Figure 5.7: A small model F6

Proof. An update semantics for eflns would have the following form. As a
model we again use an interpreted game form F = (Σ, H, turn, P, π). The symbol
f is used for the update function.

F |= ⊥ never
F |= φ → ψ iff not M |= φ or M |= ψ
F |= 2φ iff ∀h ∈ Z(H) : π(h) |= φ
F |= [Γ : φ]ψ iff f (F, Γ, φ) |= ψ

It is claimed that no suitable function f can be found. This is done by reasoning

which properties this function should have, and showing that these properties are
contradictory.
Specifically, we look at the behaviour of the function f on an example. In
figure 5.7 an interpreted game form F6 is displayed in which agent A can choose
for either p or q. Thus, this model satisfies the following efln formula.

F6 |= [A](2p ∇ 2q)

Under the given update semantics for the language eflns the following transla-
tion of this formula should hold.

F6 |= [A : 2p ∇ 2q](2p ∇ 2q)

Since this is an update semantics, one can apply the following validities for update
semantics, that were already stated in section 5.4.

|= [n](φ → ψ) → [n]φ → [n]ψ

|= ¬[n]ψ ↔ [n]¬ψ

Using these principles one can derive the following.

F6 |= [A : 2p ∇ 2q]2p ∇ [A : 2p ∇ 2q]2q
96 Chapter 5. Politeness and Side Effects

And thus exactly one of the following two formulas must hold.

F6 |= [A : 2p ∇ 2q]2p
F6 |= [A : 2p ∇ 2q]2q

In the model F6 , the propositions p and q play a symmetric role. All previous
logics have a semantics that does not depend on irrelevant properties such as the
ordering of propositions or actions. Therefore, one would expect a reasonable
semantics not to treat the propositions p or q differently. Any choice for one of
the two similar formulas would be arbitrary, and would thus be unreasonable. 

The word reasonable used in the previous fact is of course a vague word. The
word has been interpreted in the proof as meaning that any semantics should
behave similar in symmetric situations.
The proof is based upon the fact that there are two incompatible strategies
for bringing about A’s goal 2p ∇ 2q. In the interpretation of efln, the idea of
using the most general strategy was used to solve such dilemmas. Such a strategy
would leave the agent with the most freedom, and thus it would be rational for
the agent to use such a strategy. Unfortunately, for the example goal 2p ∇ 2q
neither effective strategy is more general than the other.

5.6 Conclusion
This chapter defines three new languages efls, efln and eflns that are richer
variants of the logic efl. Using these richer languages one can distinguish pro-
tocols that are equivalent for efl. These languages are thus useful for choosing
between protocols.
To illustrate this conclusion with some examples, consider again the following
two problems.

joint decision problem A decision p can be taken if either A or B think that

p should be the case. If both agents do not want p, it should be rejected.

independent decision problem An agent A can decide whether a should hold

or not, and agent B can decide whether b should hold or not.

Both problems can be described in efl, and different protocols for both problems
exist. From the viewpoint of efl, all these protocols are equivalent. In this
chapter, we have seen that the logic efls can be used to distinguish two protocols
for the joint decision problem.
The logic efln on the other hand can distinguish the two solution for the
independent decision problems. Thus, the added expressibility of both logics
allows us to answer more detailed questions about protocols.
5.6. Conclusion 97

For each of these logics one can determine the complexity of the model check-
ing problem, which indicates whether the language can be used for verification in
practice. The next table lists these results.

logic nesting side effects model checking

efl P
efls

P
efln

PSPACE-complete
eflns

PSPACE-complete

Verification of properties expressed in efl and efls is thus feasible, whereas

verification of properties expressed in efln can be very difficult. The definition
of eflns that we have given makes this language translatable into efln, so
it must have the same model checking complexity. In the previous section an
argument is given why a better semantics is hard to define.
Chapter 6
Preference Logics in Extensive Games

This chapter is based on joint research with Olivier Roy and Johan van Ben-
them.

6.1 Introduction
The logic efl presented in chapter 4 provides a high level view of protocols. It
can distinguish some protocols, but many protocols that somehow feel different
are equivalent according to the logic efl. One explanation for this result is that
agents, according to efl act without knowledge of the plans of other agents.
They search for strategies that lead to success no matter what the other agents
do. This approach is in stark contrast with the usual assumptions of game theory.
Under the assumption of complete information, agents know the preferences of
other agents. Therefore, agents can predict what other agents do, and use this
to their advantage. In order to provide a logical model that does recognize the
importance of agent preferences, we introduce in this chapter a logic based on
preference logic.
Consider the two protocols in figure 6.1. In these games, two agents A and
B are faced with the problem of a dirty shared desk. The value of a clean desk
is 2 for each agent, but the task of cleaning is valued at utility −1. The game

A B
2 B 2 A
1 1
1 2 1 2
1 2 0 2 1 0







2 1 0 1 2 0

Figure 6.1: Two extensive games

99
100 Chapter 6. Preference Logics in Extensive Games

on the left models the situation where Alice arrives first at the office. She can
decide to clean the desk, which gives her utility 1. She can also ignore the dirty
desk. When Bob arrives and the desk is clean, he experiences utility 2. If Alice
has not cleaned the desk, Bob can clean the desk. This gives him utility 1 and
Alice utility 2. Bob’s other option is to ignore the problem, in which case both
agents experience utility 0.
The game on the right is very similar, except that in this game Bob is the first
agent to arrive at the office. The roles of the two agents are thus reversed. The
central question about these two games is whether these games are equivalent. If
that is the case, then apparently either mechanism is a fair way for both agents
to jointly decide whether to clean. If however these games are not equivalent,
then one agent might have an advantage over the other agent.
The assumption of complete information that is common in game theory tells
us that agents are not only aware of their own preferences, but also of each other’s
preferences. Thus, both agents do not only know that they want a clean desk, but
they also know that the other agent wants exactly the same. This information
can be used by agents to their advantage. The agent that arrives first, Alice in
the left game, knows that the other agent prefers a clean desk. If she does not
clean the desk, then it is best for Bob to do the cleaning. Since she knows this,
she decides to ignore the problem. The rational outcome of the left game is thus
that Bob cleans the desk. The rational outcome of the right game is that Alice
does the cleaning. Both games are thus not equivalent for the agents, and each
agent is motivated to arrive first.
The outcomes of both games that are predicted above are subgame perfect
equilibria: The first agent reasons what will happen in the subgame where she
does not clean the desk, and uses this information to decide whether she should
clean. Such a subgame perfect equilibrium can be computed using a procedure
called backward induction, and these terms are considered synonyms here. In this
chapter, a logic is presented that can capture this reasoning. Since it seems that
this preference logic might be interesting in its own right, a completeness proof
for this logic is also given.
The structure of this chapter is the following. We define the language of
preference logic in section 6.2. In this section we define a semantics for this
language, a notion of bisimulation and a proof system. Since this language is
defined in what one can call a non-modal-logic style, the competeness proof of the
defined proof system is rather hands-on. Therefore, the next section, section 6.3,
is used to define what one can call a modern variant of preference logic. For this
logic, a completeness proof using standard modal logic techniques can be given.
Section 6.4 combines preference logic with a logic for reasoning about game trees.
This combined logic is used in section 6.5 to characterize the backward induction
solution concept.
6.2. Preference Logic 101

6.2 Preference Logic

To have a preference means that one puts “one thing before or above another”[80].
In the context of games, it is important to know the preferences that agents
have between the various outcomes. One can model such preferences by giving
binary relations between outcome states. A preference relation is thus a set
R = {(x, y)|outcome x is as good as or better than y}. In our logic, one can use
φhPref iψ to say that there is a φ state x and a ψ state y such that (x, y) ∈ R. If
there are multiple agents, the agent X whose preferences are being discussed is
indicated with a subscript: φhPref iX ψ. The use of preferences is an alternative
to the use of utilities. If one has a utility function U, one can define a preference
relation by stating that xhPref iy iff U(x) ≥ U(y). Thus, agents prefer outcomes
that have a higher utility over outcomes with a lower utility. On the other hand, if
one has a preference relation one can construct a utility function that represents
the same structure. This can even be done for probability distributions over
outcomes, which arise in mixed strategy games [75]. The utility function one gets
is of course not unique: one can apply any linear transformation to the utility
function, and still get the same preference relations. This can be seen as an
argument against utility functions and thus in favor of preference relations. In
mathematical definitions, such as those given in the chapter on game theory of
this dissertations, the use of utility functions is notationally more convenient.
For logical purposes the use of preference relations makes sense. After all, modal
languages are ‘languages for talking about relational structures’ [12, p. ix].

6.2.1. Definition. Suppose the finite sets Σ and P are given, and let X ∈ Σ and
p ∈ P be typical elements. Preference logic LP consists of formulas φ generated
by the following rule.

φ ::= p | φhPref iX φ | φ → φ | ⊥

A logic for reasoning about preferences with a very similar syntax was already
proposed in 1963 [119]. However our interpretation for this language is original.
One can introduce other operators by definition in terms of the given operators.
Besides the usual logical connectives, one can define the following.
def
EX φ = φhPref iX φ
def
AX φ = ¬EX ¬φ
def
φ[Pref ]X ψ = ¬(ψhPref iX φ)

One can think of φ[Pref ]X ψ as saying that φ is strictly preferred by X over ψ.

This is some kind of universal quantification: it refers to all states satisfying φ
and all states satisfying ψ. The operator φhPref iX ψ is the dual of this operator,
and says (in preference models) that it is possible that φ is as least as good as
102 Chapter 6. Preference Logics in Extensive Games

ψ. EX φ means that there exist circumstances in which φ holds, whereas AX φ

expresses that φ always holds.
As an example, the following formula expresses a reasonable property of pref-
erences. It expresses that if p is always better than q and q is always better than
r, then q is always better than r.

p[Pref ]X q ∧ q[Pref ]X r → p[Pref ]X r

Whether the principle expressed by this formula holds for a given relation,
depends of the constraints that we put on such relation. Three properties of
relations turn out to be important. A relation R is total if ∀xy : x 6= y ⇒ (xRy ∨
yRx). It is reflexive if ∀x : xRx, and anti-symmetric if ∀xy : (xRy ∧ yRx) →
x = y. The strict version Rs is the relation Rs = {(a, b)|aRb ∧ ¬bRa}. A relation
R is strict-transitive if ∀xyz : xRs y ∧ yRs z → xRs z. In normal circumstances
one would expect a preference relation to be total, strict-transitive and reflexive.

These properties reflect reasonable properties that one would expect from a
preference relation. If one assumes that preferences follow from an underlying
utility relation, these properties should hold. Indeed these three properties are
used in the definition of a preference model, defined below.

6.2.2. Definition. A reflexive frame F is a tuple F = (W, Σ, {}Σ ) such that

W is a set of outcomes, Σ is a finite set of agents and X ⊆ W × W is a reflexive
relation between worlds for each agent X of A.

6.2.3. Definition. A minimal preference model M is a tuple M = (W, Σ, {

}Σ , P, π) such (W, Σ, {}Σ ) is a reflexive frame, P is a finite set of atomic propo-
sitions and π : W → 2P assigns propositions to outcomes.

6.2.4. Definition. A preference model M is a minimal preference model M =

(W, Σ, {}Σ , P, π) such that each X ⊆ W ×W is a strict-transitive, total relation.

In the context of preference modes we refer to elements of W as either outcomes,

states or worlds. Intuitively the worlds w ∈ W are possible outcomes and w  w 0
means that w is as least as good as w 0 . An example of a preference model is
displayed in figure 6.2. In this figure the preferences of the single agent are
indicated by the vertical position of states: Higher states are preferred over lower
states. No lines between states are therefore necessary to indicate the preference
information.
As is common in modal logic we define a pointed model to be a pair M, w
where M is a model with a set of worlds W and w ∈ W . Formulas are interpreted
over pointed models M, w in the following way.
6.2. Preference Logic 103

q
r
p
s

Figure 6.2: A single agent preference model

M, w |= ⊥ never
M, w |= p iff p ∈ π(w)
M, w |= φhPref iX ψ iff ∃(w 0 , w 00 ) ∈X : M, w 0 |= φ and M, w 00 |= ψ
M, w |= φ → ψ iff not M, w |= φ or M, w |= ψ
Let M be the model displayed in figure 6.2 and let wp be the world where p holds.
Then we can show the following.
M, wp |= p ∧ (phPref iX s) ∧ (qhPref iX r)
In standard modal logic, an accessibility relation is used in the interpretation
of modal operators. The truth conditions of the modal operator only depend on
accessible worlds. The preference operator hPref iX looks at all worlds, so one
might say it uses the universal accessibility relation, in which any world is related
to any world. Such an operator is called a global operator. In the following
lemma it is shown that such an operator can be used to define the operator Eφ
that expresses that a world satisfying the formula φ exists.
6.2.5. Lemma. Let M = (W, Σ, {}Σ , P, π) be a minimal preference model
M, w 0 |= EX φ ⇔ ∃w ∈ W : M, w |= φ

Proof. Suppose that M, w 0 |= EX φ. By definition this means that M, w 0 |=

φhPref iX φ. This means that there are states w1 and w2 such that, among other
things, M, w1 |= φ. For the reverse direction, suppose that ∃w ∈ W : M, w |= φ.
Because the relation X is reflexive, we have w  w and therefore there are
worlds w1 = w, w2 = w such that w2 X w1 , M, w |= φ and M, w |= φ. We
conclude that M, w 0 |= φhPref iX φ and thus M, w 0 |= EX φ. 

A corollary of this theorem is that |= EX φ ↔ EY φ. It is therefore harmless to

omit the subscript X and simply write Eφ instead of EX φ.

6.2.1 Bisimulation
In this section we define a notion of bisimulation for preference models, and prove
that two models are bisimilar if and only if they satisfy the same preference logic
formulas.
104 Chapter 6. Preference Logics in Extensive Games

6.2.6. Definition. Let M = (W, Σ, {}Σ , P, π) and M 0 = (W 0 , Σ, {0 }Σ , P, π 0 )

be two minimal preference models. A relation R ⊆ W × W 0 is a bisimulation iff

• all pairs of related worlds (w, w 0 ) ∈ R satisfy the same atomic propositions:
π(w) = π 0 (w 0 ) and

• for all v, w ∈ H with v X w : ∃v 0 , w 0 ∈ W 0 : vRv 0 , wRw 0 ∧ v 0 0X w 0 and

• for all v 0 , w 0 ∈ H 0 with v 0 0X w 0 : ∃v, w ∈ H : vRv 0 , wRw 0 ∧ v X w.

We say that two pointed models M, w and M 0 , w 0 are bisimilar , iff there exists a
bisimulation R between M and M 0 such that (w, w 0 ) ∈ R. Two pointed models
M, w and M 0 , w 0 are equivalent iff they satisfy exactly the same formulas: ∀φ :
M, w |= φ ↔ M 0 , w 0 |= φ. The next theorem relates these two notions.

6.2.7. Theorem. Let P be finite and let M, w = (W, Σ, {}X∈Σ , P, π), w and
M 0 , w 0 = (W 0 , Σ, {0 }X∈Σ , P, π 0 ), w 0 be two pointed models. The models M, w and
M 0 , w 0 are bisimilar iff they are equivalent .

Proof. Let M = (W, Σ, , P, π) and M 0 = (W 0 , Σ, 0 , P, π 0 ). Suppose there is

a bisimulation R between M and M 0 such that wRw 0 . We show that that these
models are equivalent by induction on the structure of formulas φ. The case where
φ = ⊥ is easy. For any two worlds v, v 0 we have that M, v 6|= ⊥ and M 0 , v 0 6|= ⊥.
Consider now the case of φ = p ∈ P . Let v, v 0 again be arbitrary worlds in W, W 0
respectively. Suppose we have M, v |= p. The first condition of bisimulation tells
us that M 0 , v 0 |= p. Because our notion of bisimulation is symmetric, we can
repeat the argument with M and M 0 interchanged for the “only if” part. Assume
now the induction hypothesis that for all subformulas ψ of φ and all worlds v, v 0
we have that M, v |= ψ iff M 0 , v 0 |= ψ. It follows, using this hypothesis, that
if φ = ψ1 → ψ2 then M, v |= φ iff M 0 , v 0 |= φ. So as a last step we show that
M, v |= ψ1 hPref iψ2 iff M 0 , v 0 |= ψ1 hPref iψ2 . Suppose M, v |= ψ1 hPref iX ψ2 . This
means that there are worlds x, y ∈ W with M, x |= ψ1 , and M, y |= ψ2 and
x X y. Using the definition of bisimulation and the induction hypothesis, we
know that there are worlds x0 , y 0 ∈ W 0 such that M 0 , x0 |= φ, and M 0 , y 0 |= ψ and
x0 0X y 0 . Thus, we know that M 0 , w 0 |= ψ1 hPref iX ψ2 . The same argument with
M and M 0 interchanged can be used to show that M 0 , w |= ψ1 hPref iX ψ2 only if
M, w 0 |= ψ1 hPref iX ψ2 .
It remains to be proven that if ∀φ : M, v |= φ ↔ M 0 , v 0 |= φ, then
there is a bisimulation R between M and M 0 with vRv 0 . We assume that
∀φ : M, v |= φ ↔ M 0 , v 0 |= φ. The relation R that is needed is defined in
the following way: vRv 0 iff π(v) = π 0 (v 0 ). The first condition of bisimulation is
thus satisfied. In order to check the second condition, take two worlds v, u ∈ W
with v X u. One can find formulas φv and φu that describe exactly which atoms
are true in v and u respectively. This is possible because P is finite. Since v X u,
6.2. Preference Logic 105

p
q p q
p
Figure 6.3: Two bisimilar models

we have that M, w |= (φv hPref iX φu ), and thus M 0 , w 0 |= (φv hPref iX φu ). There

must be elements v 0 0 u0 so that M 0 , v 0 |= φv and M 0 , u0 |= φu . This proves the
second clause of the bisimulation definition. A symmetric argument gives us a
proof for the third condition. 

Using this notion, one can determine whether two models satisfy the same for-
mulas. In figure 6.3 two bisimilar models are displayed. In the right model, two
equally good states are displayed, one of them is labeled with proposition p, the
other one with proposition q. The left model has three states that are not equally
preferred by the single agent of this model. The left p state is better than the q
state, which is better than the right p state. (Again the preferences are indicated
by the vertical position of the states.)

6.2.2 Proof System

A proof system SP for preference logic can be defined in the following way. First
we list the axioms, then the reasoning rules. In AndConv1 and AndConv2 , it
does not matter whether X = Y or not. The symbol ± stands for a possible
negation: ±φ can be either ¬φ or φ. In each instance of the axioms AndConv1
and AndConv2 , one must make the same choice for this symbol.

P rop All propositional tautologies

E − intro φ → EX φ

Exist φhPref iX ψ → (EY φ ∧ EY ψ)

K1 (AY (φ → ψ) ∧ φhPref iX χ) → ψhPref iX χ
K2 (AY (φ → ψ) ∧ χhPref iX φ) → χhPref iX ψ
AndDist1 φhPref iX ψ → ((φ ∧ ξ)hPref iX ψ ∨ (φ ∧ ¬ξ)hPref iX ψ)
AndDist2 φhPref iX ψ → (φhPref iX (ψ ∧ ξ) ∨ φhPref iX (ψ ∧ ¬ξ))
AndConv1 (±(φhPref iY ψ) ∧ ξ)hPref iX χ ↔ ±(φhPref iY ψ) ∧ ξhPref iX χ
AndConv2 χhPref iX (±(φhPref iX ψ) ∧ ξ) ↔ ±(φhPref iY ψ) ∧ χhPref iX ξ
106 Chapter 6. Preference Logics in Extensive Games

The reasoning rules for this proof system are Modus Ponens and Necessitation
for AX . These two rules are listed below.
φ φφ→ψ
AX φ ψ
The following rules can be derived in this system.

OrDist1 (φ ∨ ψ)hPref iX χ → (φhPref iX χ) ∨ (ψhPref iX χ)

OrDist2 χhPref iX (φ ∨ ψ) → (χhPref iX φ) ∨ (χhPref iX ψ)

The first rule OrDist1 can be derived using the following instance of AndDist1 :

(φ ∨ ψ)hPref iX χ → (((φ ∨ ψ) ∧ φ)hPref iX χ ∨ ((φ ∨ ψ) ∧ ¬φ)hPref iX χ)

Since ((φ ∨ ψ) ∧ φ) is equivalent to φ and ((φ ∨ ψ) ∧ ¬φ) is equivalent to ψ in

propositional logic, one can use P rop and K1 to derive OrDist1 . Similarly one
can prove OrDist2 using an instance of AndDist2 .
It is interesting to consider the nesting of preferences. In order to measure
the level of nesting, define the function l

l(⊥) =0
l(p) =0
l(φ → ψ) = max(l(φ), l(ψ))
l(φhPref iX ψ) =1 + max(l(φ), l(ψ))

The language of preference logic allows for nested preference operators, but it is
not clear what such nested formulas express. Does it make sense to say that “to
prefer φ over ψ is at least as good as χ”? Someone who holds that such sentences
are meaningless may decide to exclude them from the logic by restricting the
language to L≤1P = {φ ∈ LP |l(φ) ≤ 1} where nesting of operators is not allowed.
We have decided to keep our approach as general as possible on this point, so we
did not use this restriction. But, interestingly enough, using axioms AndConv1
and AndConv2 one can show that we can always ‘unnest’ a nested formula: a
formula of nesting l > 1 is always equivalent to a formula of nesting (l − 1). This
is proven in the next lemma. This equivalence plays a role in the completeness
proof, and it shows that even if we can nest preference formulas, such nesting
does not add to the expressivity of the language.

6.2.8. Lemma. For all formulas φ ∈ LP with l(φ) > 1 there is a formula χ such
that SP `φ ↔ χ and l(χ) = l(φ) − 1

Proof. We prove that the theorem holds for φ = ψhPref iX ξ. For other formulas
φ it follows by an induction argument.
6.2. Preference Logic 107

Assume φ = ψhPref iX ξ. Any formula can be written in disjunctive normal

form. We use the notation dnf (φ) for the disjunctive normal form (see page
14) of any formula φ. The formula φ ↔ dnf (φ) is a propositional tautology
and thus we can derive SP `φ ↔ dnf (φ) and even SP `AX (φ ↔ dnf (φ)) for any
formula φ. For the language LP the disjunctive normal form has the following
appearance: dnf (φ) = ∨m ∧j ±(φ1mj hPref ihmj φ2mj ) ∧ φp where ± indicates the
possible appearance of a negation, and l(φp ) = 0.
In the next derivation j, k, l and m are indices over formulas (elements of a
conjunction or disjunction). The indices X and Y range over agents. The inner
indices Y are actually dependent on j, k, l and m, but these arguments have been
suppressed. Instead of writing Ymj on the left and Ykl on the right, and Yklmn in
the last formula, we have written Y . Also the symbol ±, which indicates a possible
negation, should be indexed so that corresponding occurences of this symbol are
interpreted correspondingly. These indices are also omitted for readability.

ψhPref iX ξ
⇔ P rop, N ecA , K1,2
[dnf (ψ)]hPref iX [dnf (ξ)]
⇔
1 2 1 2
[∨m ∧j ±(ψmj hPref iY ψmj )) ∧ ψ p ]hPref iX [∨k ∧l ±(ξkl hPref iY ξkl )) ∧ ξ p ]
⇔ OrDist1,2
1 2 1 2
∨m ∨k [∧j ± (ψmj hPref iY ψmj ) ∧ ψ p ]hPref iX [∧l ± (ξkl hPref iY ξkl ) ∧ ξ p]
⇔ AndConv1,2
1 2 1 2
∨m ∨k (∧j ± (ψmj hPref iY ψmj )) ∧ (∧l (ξkl hPref iY ξkl )) ∧ [ψ p hPref iX ξ p ]
def
= χ

One can see that l(χ) = l(φ) − 1 by noting that l(χ) = max(l(ψ), l(ξ)) and
l(φ) = 1 + max(l(ψ), l(ξ)). 

6.2.9. Theorem. The above proof system is sound: SP `φ implies |= φ

The validity proofs for each axiom are given below.

• (E − intro) Suppose that M, w |= φ. Since X is reflexive, we have that

w X w, and thus M, w |= φhPref iX φ. This is the same as M, w |= Eφ.

• (Exist) Suppose M, w |= φhPref iX ψ. This means that there are two worlds
w 0 and w 00 such that w 0  w 00 , and these worlds satisfy M, w 0 |= φ and
M, w 00 |= ψ. Since the relation Y is reflexive for all agents Y , we obtain
108 Chapter 6. Preference Logics in Extensive Games

M, w 0 |= EY φ, and M, w 0 |= EY ψ. These two conclusions can be combined

to derive M, w 0 |= EY φ ∧ EY ψ.

• (K1 - K2 ). It is not hard to see that AX φ is true iff φ holds in every world
of the model.
Suppose that M, w |= AY (φ → ψ) and M, w |= φhPref iX χ, for arbitrary
agents X and Y . This means that there are w1 , w2 ∈ W such that w1 X w2 ,
M, w1 |= φ and M, w2 |= χ. Furthermore for all worlds w 0 ∈ W we have
M, w 0 |= (φ → ψ). In particular, this last fact implies that M, w1 |= φ →
ψ, from which we get M, w1 |= ψ. Since w1 X w2 , we have M, w |=
ψhPref iX χ. The argument for K2 is similar.

• (AndDist1 − AndDist2 ) Suppose that M, w |= φhPref iX ψ. This means

that there are w 0 and w 00 such that M, w 0 |= φ, M, w 00 |= ψ and w 0 X w 00 .
For every formula ξ we know that either M, w 0 |= ξ or M, w 0 |= ¬ξ, and
hence M, w 0 |= (φ ∧ ξ) ∨ (φ ∧ ¬ξ) . Thus, M, w |= ((φ ∧ ξ)hPref iX ψ) ∨ ((φ ∧
¬ξ)hPref iX ψ). The argument for AndDist2 is similar.

• (AndConv1 − AndConv2 ) As for the other cases, we only prove soundness

for AndConv1 , the argument for the other axiom being entirely similar. We
also prove soundness for the axiom with a positive occurrence of φhPref iX ψ.
Suppose that M, w |= (φhPref iX ψ ∧ χ)hPref iX ξ. This means there are two
worlds w1 , w2 ∈ W such that M, w1 |= φhPref iX ψ ∧ χ and M, w2 |= ξ and
w1 X w2 . From these one can derive M, w1 |= φhPref iX ψ and M, w1 |=
χ. This new fact can then be used to show that M, w |= χhPref iX ξ and
M, w |= φhPref iX ψ. Finally M, w |= φhPref iX ψ ∧ χhPref iX ξ.
For the reverse implication, suppose that M, w |= φhPref iX ψ ∧ χhPref iX ξ.
This means firstly that there are two worlds w1 , w2 ∈ W such that M, w1 |=
φ, M, w2 |= ψ and w1 X w2 and secondly that there are two worlds
w3 , w4 ∈ W such that M, w3 |= χ, M, w4 |= ξ and w3 X w4 . From these
to facts one can derive that M, w3 |= φhPref iX ψ ∧ χ and this can be used
to conclude M, w |= (φhPref iX ψ ∧ χ)hPref iX ξ.

6.2.10. Theorem. The above proof system is complete: |= φ implies SP `φ

Suppose φ is given. Assume that ¬φ cannot be proven, in other words that

φ is consistent. We construct a model M with a world w such that M, w |= φ.
Let S = {φ} be a maximally consistent set containing φ. All we need to do is to
show that there is a model M, w such that ∀ψ ∈ S : M, w |= ψ. It then follows
that M, w |= φ.
Let P be the set of atoms occurring in φ. A maximal propositional conjunction
m
φ is an ordered conjunction of atoms or negated atoms such that every atom
in P is mentioned exactly once. Thus, if P = {a, b, c} then a ∧ ¬b ∧ c is a
maximal propositional conjunction, but b ∧ a is not. The ordering ensures that
6.2. Preference Logic 109

equivalent maximally consistent formulas are exactly equal. Let S prop consist of
all propositional logic formulas in S, and let S max = {φm hPref iX ψ m ∈ S|φm , ψ m
are maximal propositional conjunctions}. The proof now proceeds as follows. We
first construct a model M such that there is a state w in M so that M, w |= φ for
all φ ∈ S prop and such that every state in M satisfies all formulas in S max . Then
we show that M, w satisfies all formulas in S.
The model M = (W, Σ, , P, π) is defined in the following way. Let W =
{φm |(Eφm ) ∈ S max }. So W contains maximal propositional conjunctions. Σ is
the set of agents mentioned in S, and this set is finite because only a finite number
of agents can be mentioned in a finite formula. We define π(φm ) = {p| SP `φm →
p}. The preference relation is defined by φm X ψ m ⇔ (φm hPref iX ψ m ) ∈ S max .
This relation is reflexive: If (φm hPref iX ψ m ) ∈ S max then, because of Exist, we
know EX φm = φm hPref iX φm ∈ S max . Similarly for ψ m .
The world w that we need can be found in the following way. Suppose that
φ ∈ S prop for a maximal propositional conjunction φm . This implies, using
m

E − intro, that Eφm ∈ S and thus there is a world w such that M, w |= φm .

Since every formula φ ∈ S prop is a consequence of φm , we have that M, w |= φ for
all φ ∈ S prop .

6.2.11. Lemma. Let φ and ψ be propositional formulas. φhPref iX ψ ∈ S iff

M, w |= φhPref iX ψ

Proof. Let φ, ψ be propositional formulas and assume φhPref iX ψ ∈ S. We can

repeatedly apply axioms AndDist1 and AndDist2 for all the propositions, and use
the fact that χ∨ξ ∈ S implies χ ∈ S or ξ ∈ S, to obtain two maximal conjunction
φm and ψ m such that SP `φm → φ and SP `ψ m → ψ and φm hPref iX ψ m ∈ S.
It follows that φm hPref iX ψ m ∈ S max . Using axiom Exist we conclude that
Eφm ∈ S max and Eψ m ∈ S max . Therefore, φm ∈ W and ψ m ∈ W . From the
definition of X we obtain that φm X ψ m and this leads us to conclude that
M, w |= φm hPref iX ψ m . Using the soundness of K1 and K2 we can derive that
M, w |= φhPref iX ψ.
For the reverse part, assume that M, w |= φhPref iX ψ. From the soundness of
AndDist1 and AndDist2 it follows that there are maximal conjunctions φm and
ψ m such that M, w |= φm hPref iX ψ m . This implies that φm hPref iX ψ m ∈ S, and
using K1 and K2 we conclude that φhPref iX ψ ∈ S. 

An induction argument over the level of nesting in sets S can be given to show
that φ ∈ S iff M, w |= φ for any formula φ with l(φ) ≤ 1. To show that this is
also the case for higher level formulas, with nested preferences, we can use lemma
6.2.8 to find for any formula φ a formula ξ such that SP `φ ↔ ξ and l(ξ) ≤ 1.
If φ ∈ S then ξ ∈ S, and since l(ξ) ≤ 1 we can find a model M, w such that
M, w |= ξ. It now follows that M, w |= φ, and we have shown that the given
110 Chapter 6. Preference Logics in Extensive Games

proof system is complete for minimal preference models.

Now we can make the step from minimal preference models to preference
models. We say that an axiom A is sound on a set of reflexive frames S if
every instance φ ∈ A is true on every model M = (W, Σ, {X }X∈Σ , P, π) such
that (W, Σ, {X }X∈Σ ) ∈ S. An axiom scheme φ is complete for S if for every
reflexive frame (W, Σ, {X }X∈Σ ) ∈
/ S can be extended to a pointed model M, w =
(W, Σ, {X }X∈Σ , P, π), w such that there is an instance φ0 of φ with M, w |= ¬φ0 .
Define the following extra axioms.

T otal (EX φ ∧ EX ψ) → (φhPref iX ψ ∨ ψhPref iX φ)

T rans (EX ψ ∧ φhPref iX ξ) → (φhPref iX ψ ∨ ψhPref iX ξ)

6.2.12. Theorem. T otal is a sound and complete axiom scheme for the set of
reflexive frames with total preference relations.

Proof. Suppose that M = (W, Σ, {X }X∈Σ , P, π) is a model such that all
relations X are total and let w ∈ W . Assume that M, w |= EX φ ∧ EX ψ. This
means that there are two worlds x, y ∈ W such that M, x |= φ and M, y |= ψ.
From totality we know that either x X y or y  x. In the first case, we have
M, w |= φhPref iX ψ and in the second case M, w |= φhPref iX ψ. Either way this
leads to M, w |= (φhPref iX ψ ∨ ψhPref iX φ). Thus, T otal is sound for reflexive
frames with total preference relations.
For the second part assume that (W, Σ, {X }X∈Σ ) is a non-total reflexive
frame. This means that for some X and x, y ∈ W it is the case that neither
x X y nor y X x. Define P = {p, q} and π(x) = {p}, π(y) = {q} and for
all remaining worlds z : π(z) = ∅. Let M = (W, Σ, {X }X∈Σ , P, π). This model
satisfies M, x |= (EX p ∧ EX q). However, M, w |= (phPref iX q ∨ qhPref iX p) does
not hold. Thus, M, x |= (EX p ∧ EX q) → (phPref iX q ∨ qhPref iX p) does not hold.
Therefore, T otal is complete for the class of total reflexive frames. 

6.2.13. Theorem. T otal are sound and complete axioms for the set of reflexive
frames with strict-transitive, total preference relations.

Proof. Note that the transitivity axiom implies totality: Take φ = ξ. Thus, if
we add the axiom T rans then T otal is derivable.
First we prove the soundness of T rans on strict-transitive, total reflexive
frames. Suppose that M = (W, Σ, {X }X∈Σ , P, π) is a model such that all
relations X are total and strict-transitive, and let w ∈ W . Assume that
M, w |= (EX ψ ∧ φhPref iX ξ). This means that there are three worlds x, y, z ∈ W
such that M, x |= φ, M, y |= ψ, M, z |= ξ and x X z. In order to obtain a
contradiction, assume that M, w |= ¬(φhPref iX ψ) ∧ ¬(ψhPref iX ξ). Since the
6.3. An Alternative Preference Logic 111

model is total, it follows now that z X y and y X x. From strict-transitivity

we obtain that z X x, and this implies that not x X z. This is a contradiction,
so the axiom T rans does hold on this model. Since we have assumed M, w to be
an arbitrary total, strict-transitive model, we may conclude that any such model
satisfies the transitivity axiom.
For the second part, assume that (W, Σ, {X }X∈Σ ) is a reflexive frame. We
can assume that the preference relations are total, otherwise the derivable axiom
T otal would not hold on some model based on this reflexive frame. Thus, we
assume that for some X, the relation X is not a strict-transitive relation. This
means that for some states x, y, z ∈ W it is the case that x X y and y X z,
but not x X z. Using totality we can show that z X x must hold. Define
P = {p, q, r} and π(x) = {p}, π(y) = {q}, π(z) = {r} and for all remaining
worlds w : π(w) = ∅. Let M = (W, Σ, {X }X∈Σ , P, π). This model satisfies
M, x |= (EX q ∧ rhPref iX p), and M, x 6|= qhPref iX p and also M, x 6|= phPref iX q.
Hence the axiom T rans is not sound on this model.
Therefore, T rans characterizes the class of reflexive frames with strict-transitive,
total preference relations. 

Transitivity of a relation is a commonly assumed in modal logic, for instance

for reasoning about time [12], and a standard axiom for transitivity is 33φ → 3φ.
For preference logic we have used the more difficult notion of strict-transitivity.
A reasonable question is therefore whether a simpler axiom is not available. Con-
sider for instance the following axiom, which seems to capture transitivity.
((φhPref iX ψ) ∧ (ψhPref iX ξ)) → (φhPref iX ξ)
This axiom is unfortunately not valid on preference models. A counter-example
is the following instance, which does not hold on the model displayed in figure
6.2 on page 103.
((phPref iX (q ∨ s)) ∧ ((q ∨ s)hPref iX r)) → (phPref iX r)
Another reasonable question is whether there are strict-transitive models that
are not transitive. An example of such a model is given in figure 6.4. In this
figure the preference relation of a single agent has been indicated using arrows.
The reflexive arrows have been omitted.

6.3 An Alternative Preference Logic

The proof given above, although valid, lacks a certain elegance: it does not make
use of existing modal logic results. Therefore, in this paragraph we present a
different axiomatisation. In order to do so the language of preference logic is
slightly adapted.
112 Chapter 6. Preference Logics in Extensive Games

p r

Figure 6.4: A strict-transitive, intransitive preference model

6.3.1. Definition. Suppose the finite sets Σ and P are given, and let X ∈ Σ and
p ∈ P be typical elements. Alternative preference logic L2P consists of formulas φ
generated by the following rule.

φ ::= p | 3X φ | Eφ | φ → φ | ⊥

In this logic, the operator φhPref iX ψ can be defined as E(φ ∧ 3ψ). Intuitively
the meaning of 3X φ is that there is a state better for X than the current state,
in which φ holds. The construct Eφ means that somewhere in the model a state
exists in which φ holds. For this logic we define the operators 2X φ = ¬3X ¬φ
and Aφ = ¬E¬φ. These operators are thus duals for the two primitive operators.
This logic is interpreted in the following way. Let M = (W, Σ, , P, π) be a
preference model.
M, w |= ⊥ never
M, w |= p iff p ∈ π(w)
M, w |= Eφ iff ∃w 0 ∈ W : M, w 0 |= φ
M, w |= 3X φ iff ∃(w, w 0 ) ∈X : M, w 0 |= φ
M, w |= φ → ψ iff not M, w |= φ or M, w |= ψ
The operator 3X is interpreted in the standard modal logic fashion using the
relation X . The operator Eφ also has a standard interpretation, but based on
the universal relation. All worlds are accessible to this operator.

6.3.2. Fact. Alternative preference logic L2P is strictly more expressive than
preference logic LP .

Proof. The operator φhPref iX ψ can be defined in alternative preference logic,

but the operator 3X φ is not definable in the old preference logic. This can be
shown by looking again at the models in figure 6.3. The formula E(p ∧ ¬3q),
which says that there is a p world for which no equal or better q world exists, is
satisfied in one model but not in the other. Since these models satisfy the same
LP formulas, there cannot be a LP formula that is equivalent to E(p ∧ ¬3q). 
6.4. Finite Tree Logic 113

6.3.1 Proof System

One can define a proof system for this logic quite easily, because the interpreta-
tion of both operators is a standard modal logic interpretation. The proof system
SP2 for L2P has the following axioms:

Axioms for 3
K 2X (φ → ψ) → (2X φ → 2X ψ)
4 3 X 3X φ → 3 X φ
T φ → 3X φ
Axioms for E
KE A(φ → ψ) → (Aφ → Aψ)
4E EEφ → Eφ
TE φ → Eφ
BE φ → AEφ
Incl 3X φ → Eφ
Tot (Eφ ∧ Eψ) → (E(φ ∧ 3X ψ) ∨ E(ψ ∧ 3φ))
The reasoning rules for this logic are Modus Ponens, Necessitation for 2X φ and
necessity for Aφ. These three rules are listed below.
φ φ φ φ→ψ
2X φ Aφ ψ
6.3.3. Theorem. The proof system L2P is sound and complete for the language
L2P on minimal preference models with transitive preference relations.

Proof. It is clear that the axioms are sound and that the reasoning rules pre-
serve validity: they are all standard axioms. The logic is complete with respect to
the class of reflexive and transitive frames, see [12, p.417]. To see that the logic
is complete with respect to the class of reflexive, transitive and linear frames,
just notice that Lin, the axiom for linearity, is a Sahlqvist formula. Applying
the algorithm of the Sahlqvist Correspondence Theorem [12, p.165] one sees that
it corresponds to the first order expression of linearity: ∀x, y(x  y ∨ y  x).
Furthermore, by the canonicity of Sahlqvist formulas, [12, p.322], we know that
the canonical model for L2P is linear, and so that L2P is complete with respect to
the class of reflexive, transitive and linear frames. 

6.4 Finite Tree Logic

In this section a logic is presented that can be used for reasoning about finite
trees, such as game trees. Using this logic one can describe games trees. This
logic is adapted from Blackburn and Viol [13], as is the completeness proof.
114 Chapter 6. Preference Logics in Extensive Games

6.4.1. Definition. Suppose the finite sets Σ and P are given, and let X ∈ Σ and
p ∈ P be typical elements. Finite tree logic LT consists of formulas φ generated
by the following rule.

φ ::= p | φhPref iX φ | hXiφ | hΣiφ | hΣ+ iφ | φ → φ | ⊥

We define the following additional operators.

def
[X]φ = ¬hXi¬φ
def
[Σ]φ = ¬hΣi¬φ
def
[Σ+ ]φ = ¬hΣ+ i¬φ

6.4.2. Definition. Let {RX }X∈Σ be S an indexed set of relations RX , one for
each agent X ∈ Σ. We define R = {RX }X∈Σ as the union of all relations in
{RX }X∈Σ and R+ as the transitive closure of R.

6.4.3. Definition. A proto-model M is a tuple M = (W, Σ, {RX }X∈Σ , P, π, )

such that W is a set of worlds, Σ a finite set of agents, for each X ∈ Σ the relation
RX ⊆ W × W is a relation between worlds, P is a set of atomic propositions and
π : W → 2P assigns propositions to worlds.

6.4.4. Definition. A tree model M is a proto-model M = (W, Σ, {RX }X∈Σ , P, π)

such that R = ∪X RX defines a finite tree on some subset W T of W .

The worlds W \ W T are not related by the relation R to any other worlds. These
so-called ‘loose worlds’ are not reachable by actions, but are important in the
preferences of agents.
We define the set of terminal nodes Z(W, R) by Z(W, R) = {w ∈ W |¬∃w 0 :
wRw 0 }. The reach function reach(R, w) is the set of points reachable from w and
can be defined by stating that reach(R, w) is the smallest set S such that w ∈ S
and x ∈ S ∧ xRy ⇒ y ∈ S.

6.4.5. Definition. A preference tree model M is a tuple M = (W, Σ, {RX }X∈Σ , {X
}X∈Σ , P, π) such that (W, Σ, {RX }X∈Σ , P, π, ) is a tree model and for each agent
X ∈ Σ the relation (X ) ⊆ Z(W, R) × Z(W, R) is a reflexive, strict-transitive,
total relation.

M, w |= ⊥ never
M, w |= p iff p ∈ π(w)
M, w |= φ → ψ iff M, w |= φ implies M, w |= ψ
M, w |= hXiφ iff ∃w 0 : wRX w 0 such that M, w 0 |= φ
M, w |= hΣiφ iff ∃X, w 0 : wRX w 0 such that M, w 0 |= φ
M, w |= hΣ+ iφ iff ∃w 0 : wR+ w 0 such that M, w 0 |= φ
M, w |= φhPref iX ψ iff ∃(u, v) ∈ (X ) such that M, u |= φ and M, v |= ψ
6.4. Finite Tree Logic 115

As an example of how this logic can be used, consider the protocol displayed
in figure 4.1 on page 4.1. This protocol can be described as a preference tree
model M1 , if we indicate what the preferences of the agents are. One possible set
of preferences has therefore been indicated in the next table.
Agent Most to least preferred outcomes
A x, y, z
B y, x, z
C z, x, y
It is assumed here that the utility that an agent attaches to a certain outcome,
only depends on the atomic propositions that hold on a certain outcome. This is
a reasonable assumption, since these atomic propositions are supposed to encode
the relevant properties of each outcome. Thus, according to the table, A prefers
both x outcomes above both y outcomes, and both y outcomes over both z
outcomes. The first two formulas describe the structure of the game, and the
last formula refers to the preferences. Let wA be the root of model M1 , and let
φ⊥ = [Σ]⊥. The formula φ⊥ holds in terminal states.

M1 , w0 |= hAi(hBi(φ⊥ ∧ x) ∧ hBi(φ⊥ ∧ y) ∧ hBi(φ⊥ ∧ z))

M1 , w0 |= hAi(hCi(φ⊥ ∧ x) ∧ hCi(φ⊥ ∧ y) ∧ hCi(φ⊥ ∧ z))
M1 , w0 |= (yhPref iA z) ∧ (yhPref iB ¬y) ∧ (zhPref iC ¬z)

6.4.6. Definition. The proof system ST for finite tree logic consists of the rule
Modus Ponens and the following axioms.

prop =τ where τ is an instance of a propositional logic tautology

K =[X]φ → ([X](φ → ψ) → [X]ψ)
XY =hXi> → ¬hY i> where X 6= Y
all =hΣiφ ↔ ∨X hXiφ
trans =[Σ+ ]φ ↔ [Σ](φ ∧ [Σ+ ]φ)
L =hΣ+ iφ → hΣ+ i(φ ∧ ¬hΣ+ iφ)
AG =(([Σ]⊥ ∧ φ) ∨ hΣ+ i([Σ]⊥ ∧ φ)) → φhPref iX φ

The following formula is known in dynamic predicate logic as the induction prin-
ciple.
φI = ([Σ]φ ∧ [Σ+ ](φ → [Σ]φ)) → [Σ+ ]φ
This formula scheme is valid for finite tree logic. Here we show that this axiom
can be derived from the other principles.
6.4.7. Theorem. The induction principle can be derived in finite tree logic:

ST `φI
116 Chapter 6. Preference Logics in Extensive Games

Proof. Define α as the negation of φI : α = ([Σ]φ ∧ [Σ+ ](φ → hΣiφ)) ∧ ¬[Σ+ ]φ.
The Löb axiom L can be formulated as
ST `[Σ+ ]([Σ+ ]φ → φ) → [Σ+ ]φ
and axiom trans can be expressed equivalently as
ST `[Σ+ ]φ ↔ ([Σ]φ ∧ [Σ+ ]φ)
Using this formulation of the trans axiom, one can show that
ST `α ↔ ([Σ]φ ∧ [Σ+ ](φ → [Σ]φ)) ∧ ¬[Σ](φ ∨ [Σ+ ]φ)
This can be simplified, because [Σ]φ appears twice in this formula.
ST `α ↔ ([Σ]φ ∧ [Σ+ ](φ → [Σ]φ)) ∧ ¬[Σ][Σ+ ]φ
And one can bring the negation inside:
ST `α ↔ ([Σ]φ ∧ [Σ+ ](φ → [Σ]φ)) ∧ hΣi¬[Σ+ ]φ
Using trans again and Modus Ponens gives
ST `α ↔ ([Σ]φ ∧ [Σ][Σ]φ ∧ [Σ][Σ+ ](φ → [Σ]φ)) ∧ hΣi¬[Σ+ ]φ
One can now derive three implications from this formula.
ST `α → [Σ][Σ]φ
ST `α → [Σ][Σ+ ](φ → hΣiφ)
ST `α → hΣi¬[Σ+ ]φ
These three statements combine to the following conclusion
ST `α → hΣi([Σ]φ ∧ [Σ+ ](φ → [Σ]φ)) ∧ ¬[Σ+ ]φ)
This statement is the same as ST `α → hΣiα
Using contraposition this leads to ST `[Σ]φI → φI
From trans one can derive that ST `[Σ+ ]φI → [Σ]φI
And these can be combined into ST `[Σ+ ]φI → φI
The necessitation rule can be used to derive ST `[Σ+ ]([Σ+ ]φI → φI )
From L now follows ST `[Σ+ ]φI
and thus (with Modus Ponens) ST `φI 

This proof system is complete, and the proof for this fact is given below in
several steps. It is a straightforward adaptation from Blackburn and Viol [13],
except that we have translated notations where necessary. Blackburn and Viol
introduced this logic for the finite binary trees that are used in linguistics as parse
trees. Our adaptation shows that this logic can also be used for game trees.
6.4. Finite Tree Logic 117

6.4.8. Definition. The closure cl(Φ) of a set of formulas Φ is the smallest set
S such that the following hold.

• For all subformulas ψ of formulas φ ∈ Φ we have ψ ∈ S

• If hΣ+ iφ ∈ S then hΣiφ ∈ S

• If φ ∈ S and φ is not of the form ¬ψ then ¬φ ∈ S

If Φ is a finite set, then cl(Φ) is finite. From now on we assume that Φ is a finite
set.

6.4.9. Definition. A set A is a maximally consistent subset of some set S if

there is a maximally consistent set C such that A = C ∩ S. The atom set At(Φ)
of a set of formulas Φ consists of all the maximal consistent subsets A of cl(Φ).

If Φ is finite then also the set of

W atoms is finite. Furthermore, for every finite set
At(Φ) it is the case that SP ` A∈At(Φ) A. One can think of At(Φ) as the set of
subformulas whose truth we are interested in. A maximally consistent subset is
thus that part of a maximally consistent set that we are interested in.
These definitions are used to define a proto-model. This proto-model is not a
yet tree, but later on we order the elements of this model in such a way that a
tree is formed.

6.4.10. Definition. The proto-model C Φ is defined as (At(Φ), Σ, {RX }X∈Σ , P, π)

where Σ consists of all agents mentioned in Φ, P of all atomic propositions oc-
curring in Φ, and π(A) = {p ∈ P |p ∈ A}. The relations RX are defined by
^ ^
ARX B ⇔ SP 6 `¬( A ∧ hXi( B))

It is not hard to show that for the operator hΣi a similar

V condition
V holds. Using
axioms 3, 4 we can show that ARB ⇔ SS P 6 `¬( A ∧ hΣi( B)). We define
L0 (Φ) = {A ∈ At(Φ)|[Σ]⊥ ∈ A}. Let Si = j≤i Li . If At(Φ) \ Si 6= ∅ then Li+1
exists and we define
^ _ ^
Li+1 = {A ∈ At(Φ)|A ∈ / Si and A ∧ [Σ+ ]( B)is consistent}
B∈Si

6.4.11. Lemma. Suppose A ∈ Li and hXiφ ∈ A. There is a B ∈ Lj such that

ARX B and j < i.

Proof. Suppose that ST `¬(φ ∧ hΣ+ i¬φ). Using necessitation one can derive
that ST `¬hΣ+ i(φ ∧ hΣ+ i¬φ). The Löb axiom can now be used in contra positive
form (thus ¬b → ¬a instead of a → b). This gives us ST `¬φ. Another way to put
118 Chapter 6. Preference Logics in Extensive Games

this is to say that if φ is consistent (ST 6 `¬φ), then (φ ∧ hΣ+ i¬φ) is consistent:
ST 6 `¬(φ ∧ hΣ+ i¬φ).
Suppose now that A and B together form a partition of At(Φ): All atoms
appear in A or B but not in both. One can V show using W the result
V of the previous
+
paragraph that there is an Ai ∈ A such that Ai ∧[Σ ]( Bj ∈B Bj ) is consistent.
The proof for this observation is the following.
W V W V
TheWdisjunction Ai ∈A Ai is consistent and therefore (observation 1) ( Ai ∈A Ai )∧
[Σ+ ]¬ Ai ∈A Ai is consistent. Since A and B form a partition of a set At(Φ) of
V
all maximally W V subsets, ifWno setVin A is satisfied then a set of B must
consistent
be satisfied: ¬ Ai ∈A Ai implies Bj ∈B Bj . This leads to the conclusion that
( Ai ∈A Ai ) ∧ [Σ+ ]( Bj ∈B Bj ) is consistent and thus for some Ai ∈ A we have
W V W V

that Ai ∧ [Σ+ ]( Bj ∈B Bj ) is consistent. In the construction of the proto-model,

W V

the set Si and At(Φ) \ Si form a partition of At(Φ). This proves the observation.
We can use the observation that we have just proven to see that if Li+1 exists,
then it is non-empty. Every level i the set Si thus gets bigger. Since At(Φ) is a
finite set, eventually Si = At(Φ) and every atom has been assigned to a certain
level. For the proof of the lemma, suppose A ∈ Li and hXiφ ∈ A. It is clear that
i > 0 and thus Si−1 exists. To obtain a contradiction, suppose that V there isVno
atom B ∈ Si−1 such that ARX B, thusV that for all B
V the formula
V A ∧ hXi B
is inconsistent. This means that ST ` A ∧ [X]¬( B∈Si−1 B). As a shortcut,
define B = B∈Si−1 B. Since A ∈ Li , we have that A ∧ [Σ+ ]B is consistent.
V V V
V V V
Since we canV rewrite [Σ] as Y [Y ], we know that A ∧ Y [Y ]B is consistent and
thus that A ∧ [X]B is consistent. This yields a contradiction. Thus, there must
be an atom B ∈ Si−1 such that ARX B. 

This concludes the preparations for the completeness proof, and we can prove the
completeness theorem.

6.4.12. Theorem. The proof system ST is complete.

Proof. We can construct a model for any formula φ. Take Φ = {φ}. Choose
A ∈ At(Φ) such that φ ∈ A. Let W0 = {A}. We call a pair (w, hXiφ) of w ∈ Wn
an unsatisfied demand if hXiφ ∈ w but there is no w 0 ∈ Wn such that wRk w 0 and
φ ∈ w 0 . As long as there are unsatisfied demands, pick one unsatisfied demand
(w, hXiφ) and take a world w 0 ∈ Lm in a set Lm such that wRX w 0 and φ ∈ W .
The world w 0 must have a lower level than w: if w ∈ Li then i < m. Lemma 6.4.11
guarantees that such a w 0 exists. Define Wn+1 = Wn ∪ {w 0 }. The construction
process must terminate after a certain number of steps, because we add worlds
with strictly lower levels every time. Thus, for some sufficiently large m, there is
a world Wm that has no unsatisfied demands.
Let the proto-model C Φ = (At(Φ), Σ, {RX }X∈Σ , P, π) be defined as before.
0
Define the following model M = (Wm , Σ, {RX }X∈Σ , P, π 0 ) where {RX
0
}X∈Σ , π 0 are
6.5. Backward Induction: An Application 119

the restrictions of {RX }X∈Σ , π to Wm . One can use induction on the structure of
the formula φ to prove that M, w0 |= φ.
It remains for us to define a preference relation over Wm . The axiom AG
establishes that all worlds that are involved in the actions, are used in the pref-
erence relation. All other axioms only deal with preferences or only with actions.
Since we have proven the preference logic to be complete, any consistent set of
preference formulas can be satisfied by a model. This model contains exactly
one state for each maximally consistent propositional formula, but we can mul-
tiply these states to obtain a bisimilar model in which we have a correspondence
between the outcomes of the action relation and the preference worlds. If some
world of the preference model cannot be mapped on the action model, a ‘loose
world’ can be created that does not form part of the game tree: Our definition of
a tree model allows for worlds that only play a role in the preference model. 

6.5 Backward Induction: An Application

One of the reasons to study preference logics is that they can be used for logical
investigations into game theoretic solution concepts. Since the subgame perfect
equilibrium that is computed by the backward induction procedure is one of the
best known solution concepts, it is a good guinea pig for testing the expressivity of
game-related logics. For instance it is used for demonstrating the use of branching
time temporal logic by Bonanno [14], and is also modelled by Harrenstein [45].
The subgame perfect equilibrium has been defined in defition 3.3.10 on page
45. Below we rephrase this definition in terms more suitable for logical purposes.
We define a solution concept as a relation Rsol between the states of an extensive
game. The relation contains the moves that are rational: These moves are the
recommendations to the players made by the solution concept. Below we give
the definition of the relation BI that describes the backward induction solution
concept. Let M = (W, Σ, {RX }X∈Σ , {X }X∈Σ , P, π) be a preference tree model.
BI is a subset of the relation R. The outcomes of this relation are Z(W, BI),
and the reach of this relation from x is reach(BI, x). Intuitively, reach(BI, x)
contains the nodes that one can reach using only moves that appear in BI. We
define the reachable outcomes as rz(W, R, w) = Z(W, R) ∩ reach(R, w). The
relation BI is defined in such a way that for all w 0 ∈ W \ Z(W ) it is the case that
rz(W, BI, w 0 ) ⊆ Z(W, R) and krz(W, BI, w)k = 1. Thus, BI only recommends
possible moves, and it recommends exactly one move in every node that is part
of the tree. Finally, if for some state w it is the case that wBIw 0 and wRX w 00 ,
then there is an x ∈ rz(W, BI, w 0 ) such that there is a y ∈ rz(W, BI, w 00 ) such
that x X y. That is, w 0 is possibly at least as good as w 00 . This property makes
it rational for agent X to choose the move w 0 above w 00 .
120 Chapter 6. Preference Logics in Extensive Games

Not every game has a unique backward induction relation. Take for instance
any game in which all agents value all outcomes equally. In such a game, any
relation that chooses one move for each node is a backward induction relation.
If the preference relation is anti-symmetric however, then there is exactly
one backward induction relation. Otherwise there might be more. Thus, if no
agents values a pair of nodes equally, then there exists a unique subgame perfect
equilibrium.
Below we extend finite tree logic with operators 3sol φ and 3∗sol φ that refer
to recommended moves. This new logic is called solved game logic because it is
interpreted over solved models.

6.5.1. Definition. Suppose the finite sets Σ and P are given, and let X ∈ Σ
and p ∈ P be typical elements. Solved game logic Lsol consists of formulas φ
generated by the rule

φ ::= p | φhPref iX φ | hXiφ | hΣiφ | hΣ+ iφ | φ → φ | ⊥ | 3sol φ | 3∗sol φ

Again one can assume the following derived operators.

def
2sol φ = ¬3sol ¬φ
2∗sol φ = ¬3∗sol ¬φ
def

The construction 3sol φ means that one can use a recommended moved to reach a
state where φ holds. Similarly, 3∗sol φ means that one can use only recommended
moves to reach an outcome where φ holds.
This logic can be interpreted over solved models. These models are similar to
preference tree models, but contain an extra relation that describes a solution to
the game.

6.5.2. Definition. A solved model M is a tuple M = (W, Σ, {RX }X∈Σ , {X

}X∈Σ , P, π, Rsol ) so that (W, Σ, {RX }X∈Σ , {X }X∈Σ , P, π) is a preference tree model
and Rsol ⊆ R is a function: for each nonterminal state s there is a unique next
state t such that (s, t) ∈ Rsol .

The relation Rsol indicates a game-theoretic solution to the game, because it

contains recommended moves. A solved model is thus a structure that contains
a game and recommendations to all players. Such a solved model can be tested
for rationality, by checking whether the recommendations are consistent with the
preferences of the agents. The interpretation of all operators is the same as for
preference tree logic. The new operators 3sol and 3∗sol are interpreted in the
following way.

M, w |= 3sol φ iff ∃w 0 : wRsol w 0 such that M, w 0 |= φ

M, w |= 3∗sol φ iff ∃w 0 ∈ rz(W, Rsol , w) such that M, w 0 |= φ
6.5. Backward Induction: An Application 121

Note that the operator 3∗sol is interpreted such that it refers to outcome states
only. This is done because only outcome states have preferences.
In order to be able to characterize backward induction, we have to put an
extra constraint on the models that we consider. In the following definition we
define when we call a preference relation functional (with respect to a given
interpretation function π).
6.5.3. Definition. Consider a preference relation X in the context of a model
with some interpretation π. The relation X is functional if there is a function
fX : 2P → R such that for all worlds v, w we have that v X w implies fX (π(v)) ≥
fX (π(w)).
We call preference models, tree models and solved models functional if all pref-
erence relations that occur in these models are functional (with respect to the
interpretation function π of the model). If a model is function, it means that
the atomic propositions encode all properties that agents use in their preferences.
Thus, in functional models the atomic propositions are all that agents care about.
The right model in figure 6.3 is functional, but the left model is not.
The following four formulas characterize the backward induction solution con-
cept, at least for functional models. This means that all functional models on
which all instances of the following four formulas hold, have the backward in-
duction solution concept as their solution. One can thus say that the concept of
backward induction is described by the logical formulas.
In fact the first four properties hold on any solved model, because of the
constraints we have put on the solution relation. Therefore, one could say that
the fourth property characterizes backward induction. The formula B5 expresses
what kind of reasoning can be used to motivate the backward induction solution
concept, and can thus be used to explain this concept. This way of using logic
to characterize solutions concepts gives more insight in the ideas or assumptions
behind these concepts [30].

B1 =hΣi> → 3sol >

If one can do a move, one can do a recommended move
B2 =3sol φ → hΣiφ
If moving to a φ state is recommended, it is possible
B3 =3∗sol φ ↔ (([Σ]⊥ ∧ φ) ∨ 3sol 3∗sol φ)
A recommended outcome can be reached in zero or more steps
B4 =3sol φ → 2sol φ
Only one move is recommended
B5 =(3sol 2∗sol φ ∧ hXi2∗sol ψ) → (φhPref iX ψ)
X should not move against its preferences
122 Chapter 6. Preference Logics in Extensive Games

6.5.4. Theorem. If a pointed solved model M, w has the backward induction

relation as its solution then it satisfies the formulas B1 to B5 .

Proof. Suppose that M, w is a solved model, and that the relation Rsol is indeed
the backward induction relation BI. For each instance of formula Bi we show
that it holds in this model.
• Suppose that M, w |= hΣi>. This means that w is an internal node. From
krz(W, BI, w)k = 1 it follows that there must be a next BI move, and
therefore M, w |= 3sol >.
• Suppose that M, w |= 3sol φ, which means there is a state w 0 such that
(w, w 0 ) ∈ BI and M, w 0 |= φ. Since BI ⊆ R, it holds that (w, w 0 ) ∈ R and
thus M, w |= hΣiφ.
From krz(W, BI, v)k = 1 for all nonterminal nodes v, it follows that there
is only one state w 0 with (w, w 0 ) ∈ BI. Therefore, for any state w 0 with
(w, w 0 ) ∈ BI it is the case that M, w 0 |= φ and thus M, w |= 2sol φ.
• Suppose that M, w |= 3∗sol φ, which means there is an outcome w 0 ∈
rz(W, BI, w) so that M, w 0 |= φ. If w is an outcome itself, then w 0 = w
and M, w |= ([Σ]⊥ ∧ φ). Otherwise there is a w 00 with (w, w 00) ∈ BI and
M, w 00 |= 3∗sol φ and thus M, w |= 3sol 3∗sol φ.

• Suppose that M, w |= 3sol φ, which means that there is a state w 0 such that
(w, w 0 ) ∈ BI and M, w 0 |= φ. In the definition of a solved model it is stated
that this world w 0 is unique, and hence there are no other worlds w 00 such
that (w, w 00 ) ∈ BI. Therefore, M, w |= 2sol φ.
• Finally, assume M, w |= (3sol 2∗sol φ ∧ hXi2∗sol ψ). Thus, there is a state
w1 with (w, w1 ) ∈ BI and M, w1 |= 2∗sol φ. There is also a state w2 such
that (w, w2 ) ∈ RX with M, w2 |= 2∗sol ψ. Hence ∃v1 ∈ rz(W, BI, w 0 ) with
M, v1 |= φ and ∃v2 ∈ rz(W, BI, w2 ) with M, v2 |= ψ. The definition of the
BI relation now tells us that v1 is possibly at least as good as v2 , and thus
v1 X v2 , and therefore M, w |= φhPref iX ψ.


6.5.5. Theorem. If a pointed solved model M, w with a functional interpretation

satisfies the formula B4 , then it has the backward induction relation as its solution.

Proof. Suppose that a solved model M, w has preferences that are functional:
Outcomes w, w 0 with π(w) = π(w 0 ) are equally preferred. Assume that all in-
stances of B4 hold on M, w.
6.5. Backward Induction: An Application 123

1:p 2:q 3:p

Figure 6.5: A non-functional tree model M

We have to show that the relation Rsol in M is a backward induction relation.

Not surprisingly, this has to be shown inductively, starting with the final nodes.
The base case is formed by nodes w that are outcomes, and for these nodes
nothing has to be proven: The Rsol relation of an outcome is always the empty
relation, because there are no moves to recommend.
Assume now that w is a nonterminal node such that (w, w1 ) ∈ Rsol , and let
(w, w2 ) ∈ RX , and that the recommended move (w, w1 ) is in violation of the
backward induction properties. This assumption is made to derive a contradic-
tion. Thus, we assume that all outcomes in rz(W, BI, w1 ) are strictly worse
than all outcomes in rz(W, BI, w2 ), according to agent X. Since the prefer-
ences are functional, it holds that π(o1 ) 6= π(o2 ) for all o1 ∈ rz(W, BI, w1 ) and
o2 ∈ rz(W, BI, w2 ). Take a formula φ1 that describes an outcome o1 exactly,
and a formula φ2 that describes an outcome o2 exactly. Naturally M, o1 |= ¬φ2
and vice versa. Since the preferences are functional, all φ1 states are strictly less
preferred than all φ2 states, and thus M, w |= ¬(φ1 hPref iX φ2 ). Since W, w |=
(3sol 2∗sol φ1 ∧ hXi2∗sol φ2 ) and B4 is supposed to hold, this is a contradiction.
Thus, it is not possible that the move (w, w1 ) does not satisfy the backward
induction properties. Using induction we can now conclude that all moves rec-
ommended by Rsol are moves recommended by backward induction, and hence
that Rsol = BI. 

It is necessary to restrict the relations in the model to be functional. There

are non-functional solved models that satisfy the given formulas, but whose so-
lution relation is not a backward induction solution. An example of such a non-
functional model M is displayed in figure 6.5. Assume that M is a solved model
where outcome 3 is preferred over the other two outcomes, and that the solu-
tion relation in M recommends outcome 1. This solution is not the backward
induction relation, since outcome 3 is clearly better. However it satisfies B1 to
B5 .
In order to show that the logical approach of this chapter is more expressive
than the use of the logic efl, consider again the first solution in figure 4.1 on
124 Chapter 6. Preference Logics in Extensive Games

B C

x y z x y z

Figure 6.6: Solved model M2

page 58 to the example voting problem of chapter 4. On page 115 we defined a

preference tree model by combining the protocol of figure 4.1 with a given set of
preferences. We extend this model to a solved model M2 by adding suggested
moves. The solution relation added is that A chooses wB , B chooses y and C
chooses z. This solution relation is indicated in figure 6.6.
Call the root node wA , B’s decision node wB and C’s decision node wC .
Assume that the following preference formulas hold.

M, wA |= x[Pref ]A y ∧ y[Pref ]A z
M, wA |= y[Pref ]B (x ∨ z)
M, wA |= z[Pref ]C (x ∨ y)

If the solution of this solved model is the backward induction relation, then it
follows from y[Pref ]B (x ∨ z) that M, wB |= 3sol y. Similarly, M, wC |= 3sol z.
Since one can derive from the first line that M, wA |= ¬(zhPref iA y), it follows
from B5 that not M, wA |= 3sol 2∗sol z and hence M, wA |= 2∗sol y. Therefore, the
indicated solution relation is indeed the backward induction solution.
After considering more different preference assumptions, one can show that
agents B and C have the best ‘chance’(assuming all different preference relations
are equally likely) to get the options they prefer most. Agent A on the other
hand is least likely to get the option it prefers least. Thus, using preference logic
one can give advice to agents which role they should take in this voting problem,
which was not possible based on efl. Agents that strongly prefer one option
should take the role of agent B or C, whereas agents that strongly dislike one of
the options are better of as agent A.

6.6 Conclusion
In this chapter, the focus has not been on model checking but on proof theory.
A language LP for reasoning has been presented and a complete proof system
for this language is given. We have also developed a more modern language for
6.6. Conclusion 125

preferences L2P , and shown that this language is more expressive than LP , while
having a proof system with a simpler completeness proof.
In section 6.4, we have used preference logic in combination with a logic for
reasoning about finite trees. This allows one to reason about extensive games.
Again a proof system for this logic has been defined, using techniques that are
standard in modal logic. The logic can be used for characterising the backward
induction solution concept. An analysis of the example voting problem of chapter
4 shows that this logic is more expressive than efl.
The most important conclusion that can be drawn from this chapter is that
modal logic, because so much is known about it, is a suitable tool for model-
ing game-theoretic reasoning. Possible future work would be to analyse other
solution concepts as well, such as iteration of dominant strategies, and the (not
subgame-perfect) Nash equilibrium. A more challenging project would be to
analyse imperfect information games. One such attempt already exists in the
form of ATEL [102], also discussed in section 7.5. Recent ATEL research has
shown that reasoning about imperfect information games is a challenging prob-
lem [3, 54, 55, 109]. It would be interesting to compare the ATEL approach, with
the more direct modal logic approach used in this chapter.
Chapter 7
Knowledge Condition Games

7.1 Introduction
In the previous chapters, we have looked at protocols that can be modelled as per-
fect information game forms. In such protocols all agents are aware of all previous
events, and therefore no aspects of the current situation are unknown. In this
chapter the focus is on protocols that can be modelled as imperfect information
game forms. Such protocols are interesting for at least two reasons:

• The imperfect information of agents has consequences of what strategies

they can use. Finding optimal strategies for imperfect information games
is therefore a more complex problem than for perfect information games.

• The knowledge that agents do and do not have of the current situation can
be used in the definition of the game. Having certain knowledge can be the
goal of an agent or a coalition of agents.

In order to study these two aspects of multi-agent protocols, we define a new class
of games, called knowledge condition games. In a knowledge condition game, two
coalitions of agents enact a protocol. One coalition strives to reach a certain
knowledge situation, and the other coalition tries to prevent the first coalition
from reaching its goal. In other words, one coalition “wins” if it is able to force a
certain condition to hold in the world, where this condition relates to the know-
ledge (and absence of knowledge) of the agents in the game. Formally, we specify
the goal situation (i.e., the condition that the agents strive to achieve) using epis-
temic logic, and protocols are modeled as interpreted game forms with imperfect
information.
After defining these games and illustrate using various examples, we focus
on the computational complexity of determining who wins a knowledge condi-
tion game under various assumptions. Specifically the following questions are
answered.

127
128 Chapter 7. Knowledge Condition Games

• Whether the presence of opponents make it harder to determine the exis-

tence of a winning strategy in a knowledge condition game

• Whether winner determination is harder if one assumes that strategies are

known to agents

• Whether one can identify variants of knowledge condition games in which

winner determination is tractable

The complexity results also allow one to see whether reasoning about knowledge in
strategic situations is indeed a complex problem. The fact that we have collected
in this chapter many different complexity results shows that this is indeed the
case.
The structure of this chapter is as follows. In the next section, section 7.2
we have collected all necessary definitions. Section 7.3 provides four examples of
knowledge condition games. The first example shows how knowledge properties
are important in a voting protocol. The second example involves a more playful
quiz problem. It shows how signaling can enter into reasoning about knowledge.
The third example, the Russian Cards problem, is larger than the previous two
and hence the corresponding knowledge condition game is not easily solved by
hand. In the last example the use of a multi-step strategy for a coalition of
three agents is demonstrated. Section 7.4 presents four results relating to the
complexity of knowledge condition games. We prove the complexity of deciding
a knowledge condition game in which strategies are known, first for the restricted
case without opponents, then with opponents. We then do the same for knowledge
condition games in which strategies are unknown. Section 7.5 discusses some
related work, and section 7.6 presents some conclusions.

7.2 Defining Knowledge Condition Games

In this section we define how one can create a knowledge condition game G from
an interpreted game form F . This is done in two definitions at the end of the
section. Before these definitions, we define epistemic logic, game forms, strategies
and updates, which are all needed in order to define knowledge condition games.
The notion of an interpreted game form has been introduced in chapter 3.
In that chapter, only interpreted game forms with perfect information have been
defined. Definitions for imperfect information game forms are given below.

7.2.1. Definition. An interpreted game form F is a tuple

F = (Σ, H, turn, ∼, P, π),

where:
7.2. Defining Knowledge Condition Games 129

A
1 2
p B

Figure 7.1: Interpreted game form F0

• Σ is a finite set of agents;

• H is a non-empty, prefix-closed set of finite sequences;
• turn is a function turn : H \ Z(H) → Σ;
• for each X ∈ Σ the relation ∼X ⊆ H × H is an equivalence relation between
sequences;
• P is a finite set of atomic propositions; and
• π : Z(H) → 2P returns the true atomic propositions of any terminal history.
These components must satisfy the following condition:
if turn(h) = X and h0 ∼X h then also turn(h0 ) = X and A(H, h) = A(H, h0 ).
(This definition is adapted from Osborne and Rubinstein [79, p.200]). We have
extended their notion of information sets such that agents also have information
when they are not in charge, which is a not uncommon for logical purposes [15, 98].
Atomic propositions can be used to refer to certain terminal histories, for in-
stance to histories where an agent achieves a certain goal. The idea of annotating
end states or terminal histories with logical propositions has been used before
by Harrenstein et al [45] and the author [113]. Approaches based on temporal
logic [101, 102] often annotate all nodes of the model with propositions, so that
formulas can be interpreted anywhere in the model.
An example interpreted game form F0 is depicted in figure 7.1. In this exam-
ple, agent A can make a choice from two alternatives (numbered 1 and 2), one
of which satisfies p. After this choice, A can distinguish these situations, but B
cannot.
For every interpreted game form F we can calculate an epistemic model M =
m(F ) representing the knowledge in the end states of F . We do this by taking all
the terminal histories of F as the set of states of M . The states of the model M
are all outcomes of the interpreted game form F , and two outcomes are related
in M iff they are related in F .
7.2.2. Definition. Let F = (Σ, H, turn, ∼, P, π) be an interpreted game form.
The end situation model m(F ) is defined as m(F ) = (Σ, Z(H), ∼0 , P, π) where for
each agent X, ∼0X is the restriction of ∼X to Z(H) × Z(H).
130 Chapter 7. Knowledge Condition Games

The transformation m is used to express when an interpreted game form F

makes a formula φ true. The function m only uses the epistemic relation between
end states. The relations between other states are however used in the definition
of uniform strategies.

7.2.1 Strategies
Strategies are an important part of every game. Informally a strategy σΓ is a
function that tells all agents in coalition Γ what to do next in the histories they
control. We use nondeterministic strategies for our agents. These strategies have
been defined in definition 3.3.6 on page 43. Such a strategy does not return a
unique option that the agent should take, but it returns a set of options, with
the intention that the agent should randomly select an element of this set. Our
strategies are thus akin to the randomized or ‘mixed’ strategies, or more correctly
the behavioural strategies, of game theory [79, p.212], except that we do not
consider probabilities of making particular choices. Since we deal with imperfect
information games, only uniform strategies, as defined in definition 3.3.13, are
considered.
For the example interpreted game form F0 there are three different strategies
σ{A} for agent A. The strategy can either tell the agent to take the first option,
or it can prescribe the second option, or the strategy can express that the agent
should randomly choose between both options. Formally, these possibilities are
1 2 3
defined by respectively σ{A} () = {1}, σ{A} () = {2} and σ{A} () = {1, 2}.
For any strategy σΓ for an interpreted game form F we can consider a re-
stricted interpreted game form F 0 in which the agents X ∈ Γ only choose options
that are part of the strategy. The agents Y ∈ / Γ can still do whatever they want in
0
F . Such a restricted interpreted game form models the situation in which coali-
tion Γ is committed to the given strategy. The restricted model F 0 is computed
by an update function F 0 = u(F, σΓ ).

7.2.3. Definition. Let F = (Σ, H, turn, ∼, P, π) be an interpreted game form.

The update function u is defined by

u(F, σΓ ) = (Σ, H 0 , turn 0 , ∼0 , P, π 0 ),

where:

• H 0 is the smallest subset of H such that  ∈ H 0 and for each h ∈ H 0 and

a ∈ A(H, h): if turn(h) ∈/ Γ or a ∈ σΓ (h) then ha ∈ H 0 ;

• ∼0 is such that for all X: ∼0X =∼X ∩(H 0 × H 0 ); and

• turn 0 and π 0 are the same as turn and π, but with their domain restricted
to H 0 .
7.2. Defining Knowledge Condition Games 131

3
An update of the example F0 with strategy σ{A} does not change anything:
3 1
u(F0 , σ{A} ) = F0 . An update with σ{A} returns a model F1 with only two histories:
 and 1. This means that the epistemic model of F1 only has one state in which
p holds. Thus, using the interpretation of epistemic logic (defined on page 16), it
1
holds that m(u(F0 , σ{A} )), 1 |= KB p.

7.2.2 Strategic Games

The function G = kcg(F, Γ, Ξ, φ) defines a knowledge condition game in which Γ
wishes to achieve φ, while Ξ hopes to prevent it. The game G is not an extensive
game, but a game in normal or strategic form. It is not possible to consider G as
an extensive game, because whether the knowledge condition holds is not a local
property of each end state.
We are only interested in two-player, constant-sum, win-loss games, and in
these games only two payoff vectors are possible: (1, 0) which is best for the first
player, and (0, 1) which is best for the second player. In these games one can say
that an agent can win if it has a strategy that guarantees that the agents gets
utility 1. If the first player can win we write w(G) = 1.

7.2.4. Definition. Let G = ({A, B}, {SA , SB }, U) be a two player constant-sum

win-loss game. The winner function w is defined by

w(G) = 1 ⇔ ∃σA ∈ SA ∀σB ∈ SB : U(σA , σB ) = (1, 0)

7.2.3 Knowledge Condition Games

A knowledge condition game is a two-player, constant-sum, win-loss strategic
game. It is played between two coalitions Γ and Ξ of agents. These sets must be
disjoint, but not every agent has to be in one of those sets. If an agent X ∈ Σ
is not in Γ ∪ Ξ then this agent is said to be neutral. The agents in Γ are called
proponents, and the agents in Ξ opponents. To define a knowledge condition
game, we must give an interpreted game form F and an epistemic logic formula
φ: The proponents try to make this formula true on F , and the opponents try to
make it false on F . Formally:

7.2.5. Definition. Let F = (Σ, H, turn, ∼, P, π) be an interpreted game form,

Γ, Ξ ⊆ Σ disjoint sets of agents and φ ∈ LK a knowledge formula. Define
kcg(F, Γ, Ξ, φ) = ({Γ, Ξ}, {SnΓ, SnΞ }, U) where SnΓ , SnΞ contain all nondeterminis-
tic strategies of Γ and Ξ in F respectively, and

(1, 0) iff ∀w ∈ W : (Σ, W, ∼, P, π 0 ), w |= φ


U(σΓ , σΞ ) =
(0, 1) otherwise

where (Σ, W, ∼, P, π 0 ) = m(u(u(F, σΓ ), σΞ )).

132 Chapter 7. Knowledge Condition Games

Take the example game form F0 and take φ0 = KB p. For the game G0 =
kcg(F0 , {A}, ∅, φ0 ) we can compute a payoff matrix. As calculated before, {A}
has three strategies. The empty coalition only has the unique empty function f∅
as a strategy.

1 2 3
σ{A} σ{A} σ{A}
f∅ (1,0) (0,1) (0,1)

1
We see that for this game, {A} has a winning strategy (namely σ{A} ). Therefore,
w(kcg(F0 , {A}, ∅, φ0 )) = 1. In the above definition, we use the updated model
m(u(u(F, σΓ ), σΞ )) as a model for what all agents know. We have thus implicitly
assumed that everybody commonly knows which strategies are used by Γ and Ξ,
r if one assumes that strategies are somehow visible to other agents. As we have
argued on page 81 in the case of perfect information games, this is a reasonable
assumption. It makes sense if one considers strategies as well-known conventions.
Also if a game is played by computer programs that are open for inspection, this
is a reasonable assumption. Finally, one can argue that assuming that no details
can be kept secret is a very conservative and thus sound assumption if one tries
to prove the correctness of security protocols. In some circumstances, however,
one might not want to make this assumption. Therefore, we present below a
variant kcg 0 of knowledge condition games in which the knowledge formulas φ
is evaluated the original model m(F ). The strategies are used to determine the
reachable states w and the proponents win if in all these states w, it holds that
m(F ), w |= φ.

7.2.6. Definition. Let F = (Σ, H, turn, ∼, P, π) be an interpreted game form,

Γ, Ξ ⊆ Σ disjoint sets of agents and φ ∈ LK a knowledge formula. Define
kcg 0 (F, Γ, Ξ, φ) = ({Γ, Ξ}, {SΓ , SΞ }, U0 ) where SΓ and SΞ contain all strategies
of Γ, Ξ in F respectively, and

0 (1, 0) iff ∀w ∈ W : m(F ), w |= φ
U (σΓ , σΞ ) =
(0, 1) otherwise

where W is defined by (Σ, W, ∼, P, π) = m(u(u(F, σΓ ), σΞ )).

The difference between kcg and kcg 0 lies in their respective utility function.
The function U evaluates the formula φ in the model m(u(u(F, σΓ ), σΞ )), in all
states. The function U0 evaluates the formula φ in the model m(F ), thus in the
model before updates. This difference reflects the idea that in kcg, strategies are
commonly known, whereas in kcg 0 they are not known. The function U0 only
evaluates the formula φ in states w that occur in m(u(u(F, σΓ ), σΞ )). The idea
here is that the truth of φ only matters in states that are actually reached, and
which states are reachable depends on the strategies chosen.
7.3. Examples 133

7.3 Examples
7.3.1 Anonymous Voting
A voting protocol can be used when a group of agents has to make a joint decision
on a certain issue. A common protocol is majority voting: Each agent can vote for
an option and the option that gets the most votes is the outcome of the protocol.
In the example interpreted game form FV = (Σ, H, turn, ∼, P, π), three agents A,
B and C use majority voting to decide whether a plan P should be accepted or
not. Thus Σ = {A, B, c} and P = {a, b, c, p}. Each agent has to choose from two
actions: support the plan (s), or reject it (r). They vote in alphabetical order,
so first A chooses between action s and r, then B (without knowing A’s choice)
chooses either s or r and finally C does the same, unaware of what A and B
did. This protocol thus has eight terminal histories. The proposition p indicates
whether P is accepted and p holds if at least two agents choose s. Furthermore the
proposition a holds if A chooses s, b if B chooses s and the same for C with c. The
interpretation function is thus π(sss) = {a, b, c, p}, π(ssr) = {a, b, p} . . . π(rrr) =
∅. We assume that s 6∼X s0 if s and s0 differ in the evaluation of the outcome p,
or if the vote of X differs in s from that in s0 .
The following game results hold.
w(kcg(FV , {A, B}, {C}, p)) = 1
w(kcg(FV , {A, B}, {C}, KB c ∨ KB ¬c)) = 1
w(kcg(FV , {B}, {C}, KB c ∨ KB ¬c)) = 0
A and B together can ensure that p is true, by both voting s. They can also vote
differently, so that a and ¬b result. In this case the outcome will solely depend
on C’s choice. They thus learn what C voted. Agent B cannot learn what C did
on its own.
One example, described by Schneier [89, p. 133], is a voting protocol where
B would have the option of copying A’s (encrypted) vote. In that case one might
get
w(kcg(FV0 , {B}, {A, C}, KB a ∨ KB ¬a)) = 1
This is an unwanted property and thus a ‘bug’ in the protocol. It is necessary to
reason about knowledge to express this bug, so a standard game-theoretic analysis
might not have revealed this shortcoming.

7.3.2 Fifty-Fifty Problem

Consider the following scenario:

In a TV quiz show the quiz master asks a candidate the following

question: Which day of the week comes directly after Tuesday? Is it
a) Monday, b) Wednesday, c) Friday or d) Saturday. The candidate
134 Chapter 7. Knowledge Condition Games

N
1 2 3 4

13 14 12 14 12 13
23 24 34 34 24 23

Figure 7.2: The fifty-fifty problem FQ

has no clue whatsoever about the days of the week, and replies: ‘I am
not sure. Can I do fifty-fifty?’. The quiz master has to remove two
options that are not the answer, so he says: ‘The answer is not Monday
and neither Friday’. Does the candidate now know the answer?

This situation frequently occurs on television in several European countries in the

‘Millionaire show’. One can also consider this situation to be a metaphor for a
multi-agent information exchange situation. One can model this in an interpreted
game form FQ = (Σ, H, turn, ∼, P, π). The set of agents is Σ = {N, Q, C},
involving an agent N (Nature) that determines what the right answer is, a quiz
master Q that eliminates two answers, and a candidate C. This interpreted game
form is depicted in figure 7.2. First Nature selects one of the answers to be the
right answer: It can choose from the actions 1, 2, 3 and 4. The quiz master,
who knows the right answer, can then select an action ij that indicates that
the two options i and j are eliminated; i and j must be different from the right
answer. The terminal histories are thus all histories (k, ij). For such histories,
(k, ij) ∼C (k 0 , i0 j 0 ) if the same options are eliminated: ij = i0 j 0 . The set of atomic
propositions is P = {ai | 1 ≤ i ≤ 4} ∪ {ei | 1 ≤ i ≤ 4}, and each terminal
history is interpreted in the following way: π((k, ij)) = {ak , ei , ej }. The question
is whether the candidate knows the answer at the end of the protocol. This is
expressed by ψ = KC a1 ∨ KC a2 ∨ KC a3 ∨ KC a4 . The following table lists several
properties of this situation.

Nature may favour the candidate: w(kcg(FQ, {N }, ∅, ψ)) = 1

Nature may not favour the candidate: w(kcg(FQ, {N }, ∅, ¬ψ)) = 1
The quiz master can help the candidate: w(kcg(FQ, {Q}, ∅, ψ)) = 1

We thus see that whether the candidate knows the answer depends on Nature
and on the quiz master Q. If Nature uses a deterministic strategy, in which for
instance a1 always holds, then the candidate knows that this is the right answer.
However, if Nature uses the nondeterministic strategy in which each answer could
be the right answer, the candidate will not know the answer.
7.3. Examples 135

N
1 2 3 4

23 34 14 12

Figure 7.3: The updated interpreted game form u(FQ , σ{Q} )

The situation becomes more interesting if the quiz master gets involved. In
this game the quiz master has the ability to signal the right answer to the candi-
date. Consider, for example, strategy σ{Q} , defined as follows.
σ{Q} (1) = {23}
σ{Q} (2) = {34}
σ{Q} (3) = {14}
σ{Q} (4) = {12}
This strategy tells the candidate exactly what the right answer is: The answer
directly before the two eliminated options (assuming 4 comes before 1). The
updated model u(FQ , σ{Q} ) is given in figure 7.3. This strategy acts as a code
between the candidate and the quiz master. It is the strategy that proves that
w(kcg(FQ, {Q}, ∅, q)) = 1. A practical conclusion one can draw is that one should
not bet on this quiz if one does not know what the interests of the quiz master
are.
This example also demonstrates why we prefer to assume that strategies are
commonly known. If one would have used the alternative definition kcg 0 , in which
agents do not know what strategies are used, then one can obtain the following
results.
Nature cannot favour the candidate: w(kcg(FQ , {N }, ∅, ψ)) = 0
The quiz master cannot help the candidate: w(kcg(FQ , {Q}, ∅, ψ)) = 0
These results are counter-intuitive, since signaling in games is a phenomenon
that does occur in practice. When proving the security of a protocol, it is a
good principle to make the weakest assumptions possible. At first sight, it seems
that assuming that strategies are not known is the weakest possible assumption.
However, in the case of proving ignorance, first sight can be misleading. It is
harder to prove that the candidate does not know the answer when he or she
knows all strategies that are used, than it is to prove ignorance when he or she
does not know the strategies. Therefore, the weakest and safest assumption is to
assume that he does know the strategies. This shows that it is best to use the
definition of kcg rather than the alternative kcg 0 for these ignorance proofs. This
motivates the choice to make kcg the default and call kcg 0 the alternative.
136 Chapter 7. Knowledge Condition Games

7.3.3 Russian Cards Problem

The Russian cards problem first appeared in the Russian mathematics olympiad
in 2000 [106]. It has subsequently been picked up by logicians as an example of
a information based security. An informal description, taken from [106], is the
following.

From a pack of seven known cards two players each draw three cards
and a third player gets the remaining card. How can the players with
three cards openly (publicly) inform each other about their cards,
without the third player learning from any of their cards who holds
it?

Following the analysis by Van Ditmarsch [106] we call the agents A, B and C
and the cards 0, 1, 2, 3, 4, 5 and 6. The interesting thing about this problem
is that certain solutions to this problem appear sound, but are not sound. A
solution to this problem is a joint strategy for A and B that prescribes what they
should communicate to each other. We are mostly interested in direct exchanges:
statements by A such that B directly learns all of A’s cards. Agent B can respond
by telling A which card C has. Assume for the moment that the actual deal of
cards is that A holds 0, 1 and 2, that B holds 3, 4 and 5 and that C holds 6. Instead
of reasoning about complete strategies for A and B,we settle for identifying which
statements by A for this situation van can be part of a strategy. Here are some
solution attempts. Imagine that the next public statements are made by agent A
so that all agents can hear it.

I have 012 or 345: This statement is true and when taken literally it does not
tell C anything about a single card. Unfortunately A can imagine that C
holds card 5. in which case this statement would reveal A’s cards to C. So
A cannot make this statement safely.

I have 012 or I have none of these cards: A knows that C cannot pinpoint
any card after learning this statement. Unfortunately C can reason like
this: Suppose A has 345. In that case she cannot exclude that I hold card
2. If I had card 2 I would know that B holds card 0 and 1. Alice would
never allow me to learn that. Contradiction. By this line of reasoning she
can eliminate all possibilities except 012.

I have 012, 034, 056, 135, 146 or 236 This is an example of a statement A
can make.

In this section we define a knowledge condition game corresponding to the Russian

Cards problem. This approach can be compared to previous attempts using
epistemic model checking and dynamic epistemic logic.
7.3. Examples 137

We introduce a set of agents Σ = {N, A, B, C} and a set of cards D =

{0, 1, 2, 3, 4, 5, 6}. A nice way to use propositions for this problem is to use ai
to indicate that agent A holds card i, and similarly for bi and ci . The set of all
deals of cards allowed in this problem is

∆ = {abc|def |g |{a, b, c, d, e, f, g} = {0, 1, 2, 3, 4, 5, 6} ∧ a < b < c ∧ d < e < f }

For a deal abc|def |g, the cards of A are a, b, c, agent B owns d, e, f and C holds
card f . We can thus say that in a situation with deal 012|345|6, the following
formula holds: a0 ∧ a1 ∧ a2 ∧ b3 ∧ b4 ∧ b5 ∧ c6 .
An interpreted game form

FRC = (Σ, H, turn, ∼, P, π)

can now be defined as follows. As indicated above, we take Σ = {N, A, B, C}.

The set H is described by

H = {, δ, (δ, x)|δ ∈ ∆, x ∈ {0, 1, 2, 3, 4, 5}}

The variable x is used to indicate a symbol that A communicates publicly to

B and C. In the examples above, this symbol was a sentence such as “I have 012
or 345”. In a knowledge condition game, it is sufficient to use abstract symbols.
The agents B and C know when A uses a signal x, and thus the meaning of x.
We have chosen to allow only six different signals. It is not obvious beforehand
whether six signals is enough, but we will answer this question later.
The function turn is defined such that turn() = N and turn(δ) = A for all
δ ∈ ∆. The equivalence relations ∼ are defined such that agents know their on
cards, and the action selected by A. Thus the following definition applies, where
X can stand for any agent, h for any element of H, and x for any signal.

h 6∼X h0 if khk 6= kh0 k

abc|def |g 6∼A a0 b0 c0 |d0 e0 f 0 |g 0 iff(a, b, c) 6= (a0 , b0 , c0 )
abc|def |g 6∼B a0 b0 c0 |d0 e0 f 0 |g 0 iff(d, e, f ) 6= (d0 , e0 , f 0 )
abc|def |g 6∼A a0 b0 c0 |d0 e0 f 0 |g 0 iffg 6= g 0
((abc|def |g), x) 6∼A ((a0 b0 c0 |d0 e0 f 0 |g 0 ), x0 ) iff(a, b, c, x) 6= (a0 , b0 , c0 , x0 )
((abc|def |g), x) 6∼B ((a0 b0 c0 |d0 e0 f 0 |g 0 ), x0 ) iff(d, e, f, x) 6= (d0 , e0 , f 0 , x0 )
((abc|def |g), x) 6∼C ((a0 b0 c0 |d0 e0 f 0 |g 0 ), x0 ) iff(g, x) 6= (g 0 , x0 )
h ∼ X h0 otherwise

For the set P of propositions, we take P = {ai , bi , ci |i ∈ {0, 1, 2, 3, 4, 5, 6}}.

The function π is defined as π(((tuv|wxy|z), s)) = {at , au , av , bw , bx , by , cz }. Thus
in the situation (012|345|6, 1) the propositions a0 , a1 , a2 , b3 , b4 , b5 , c6 hold.
138 Chapter 7. Knowledge Condition Games

The knowledge goal of this problem can then be expressed by conjunction

bknows ∧ cig. The positive part bknows expresses that at B knows the deal of
cards. ^
bknows = (bi → KB bi)
i∈D

The second part cig expresses that C does not know for any card who holds it.
We call this a negative knowledge requirement, or an ignorance requirement.
^
cig = (¬KC ai ∧ ¬KC bi )
i∈D

We can now solve the Russian cards problem by finding a strategy that fulfills,
for a suitable model F , the following question.

w(kcg(F, {A, B}, ∅, bknows ∧ cig)) = 1

We can verify that there are strategies σ = σ{A} such that

w(kcg(F, {A, B}, ∅, bknows ∧ cig)) = 1

One such strategy is the following

0 ∈ σ(abc|def |g) iff abc ∈ {012, 034, 056, 135, 146, 236, 245}
1 ∈ σ(abc|def |g) iff abc ∈ {013, 026, 045, 125, 146, 234, 356}
2 ∈ σ(abc|def |g) iff abc ∈ {014, 025, 036, 123, 156, 246, 345}
3 ∈ σ(abc|def |g) iff abc ∈ {015, 024, 036, 126, 134, 235, 456}
4 ∈ σ(abc|def |g) iff abc ∈ {016, 023, 045, 124, 135, 256, 346}
5 ∈ σ(abc|def |g) iff abc ∈ {012, 035, 046, 136, 145, 234, 256}

It follows that “I have 012,034,056,135,146,236 or 245” is a safe statement for

A to make when she has one of these sets of cards. Another conclusion one can
draw is that in the Russian Cards problem, a vocabulary of six different signals
is sufficient for A to communicate its hand safely to B.
Note that the strategy described above is nondeterministic. For certain hands,
agent A has a choice of two actions. For instance σ(012|def |g) = {0, 5} and
σ(146|def |g) = {0, 1}. It is necessary for A to make truly random choices, and
not simplify its strategy by always going for one action in these cases. To see this,
suppose that agent A would decide that it does not use action 0 if another action
is available, and suppose that the card deal is 056|124|3. In that case agent A
would use action 0, and C would know that A can only have card deals 034, 056,
135, 236, or 245. Since C has card 3 itself, it can deduce that the only card deals
that A can have are 056 and 245. Therefore C knows that A must have card
7.3. Examples 139

5. When using this strategy, A should therefore make a genuine random choice
between the two available actions, for instance using a coin flip.
Other strategies for A are deterministic. One deterministic strategy, that re-
quires at least seven different signals, is the strategy σ2 , described by the following
formula.
σ2 (abc|def |g) = {(a + b + c) mod 7}
The idea is that agent A announces the sum of its cards modula 7. For instance
if A has cards 024 it should use action 6. When C hears that A has chosen action
6, it can deduce that A has cards 024, 015, 123, 256 or 346. In case that C has
card 6, it would know that A has either cards 024, 015 or 123, but does not know
for any specific card that A has it. Thus we know that there are deterministic as
well as nondeterministic strategies for agent A.
In the existing literature it was already proven that the statement “I have
012,034,056,135,146,236 or 245” can be made. In Van Ditmarsch’s paper [106] all
statements that can be used when A holds 012 are given. However Van Ditmarsch
does not present a complete strategy, and indeed it is not trivial to come up with
a set of six statements that cover all possible card combinations of agent A.
Thus knowledge condition games is a suitable framework for searching detailed
strategies for situations such as the Russian Cards problem.

7.3.4 Communication Example

In this example four agents are communicating to each other. The agents take
turns in sending out a message, a single bit in this example. Not all agents can
see all messages: agent A can only see what signal D sends, B what D sends, C
can see what signal A sends, and D can see what B and C send. The problem is
that B would like to know what message C sends.
In order to model a situation with four communicating agents, we define an
interpreted game form FC = (Σ, H, turn, ∼, P, π) with the following components.
• Σ = {A, B, C, D}

• H = {, a, ab, abc, abcd|a, b, c, d ∈ {0, 1}}

• P = {p}

• The function turn is defined by

turn() = A
turn(a) = B
turn(ab) = C
turn(abc) = D

where a, b, c ∈ {0, 1}.

140 Chapter 7. Knowledge Condition Games

• The equivalence relations ∼ are defined by the following equations. In these

equations, X can be any agent, and a, b, c, d and their primed counterprats
can be either 0 or 1.

h 6∼X h0 if khk 6= kh0 k

abcd ∼A a0 b0 c0 d0 iff ad = a0 d0
abcd ∼B a0 b0 c0 d0 iff bd = b0 d0
abcd ∼C a0 b0 c0 d0 iff ac = a0 c0
abcd ∼D a0 b0 c0 d0 iff bcd = b0 c0 d0
abc ∼X a0 b0 c0 iff abc0 ∼X a0 b0 c0 0
ab ∼X a0 b0 iff ab00 ∼X a0 b0 00
a ∼X a0 iff a000 ∼X a0 000

• π(ab1d) = {p} and π(ab0d) = ∅

¿From the definition of the equivalence relations one sees that A can see what D
does, B can see what D does, C can see what A does, and D can see what B
and C does. Also, agents can remember their own action, and they can see all
messages that are sent.
In order to come up with a knowledge condition game, we assume that the
three agents B, C and D act as a team. The goal is to make B know what action
C selects. It is not hard to see that the three agents can do this: D can copy
the message of A. Agent B can see this copied signal, and therefore knows what
C has done. In order to make this example more interesting, we add another
requirement: We assume that the coalition of agents does not want A to know
what message C sends. Since agent A can also see what agent D does, the copying
strategy sketched in this paragraph no longer works. The three agents must use
a more complicated coalition strategy.
The knowledge goal described above consists of a positive and a negative part:
agent B must gain some knowledge about C’s action, and agent A must not gain
knowledge about C. This complex goal is described by the following formula φ.

φ = (KB p ∨ KB ¬p) ∧ ¬(KC p ∨ KC ¬p)

Note that the proposition p corresponds to C’s signal.

The question is whether the three-agent coalition has a strategy that makes
this goal true. In terms of knowledge condition games, we thus would like to have
an answer to the following question.

w(kcg(FC , {B, C, D}, {A}, φ)) = 1

7.4. Computational Complexity 141

The answer to this question is “yes”. The following strategy σBCD is a successful
strategy in this knowledge condition game.

σBCD (a) ={0, 1}

σBCD (ab) ={0, 1}
σBCD (abc) ={0|b = c} ∪ {1|b 6= c}

This strategy is nondeterministic in the first two steps, but deterministic in the
last step. Agent B, who knows its own action, can deduce from D’s action
whether the proposition p holds. Agent A cannot do this, since it does not know
what B has done. It is necessary for B and C that they make a nondeterministic
choice, otherwise A could deduce whether p holds from knowing the strategy that
is used.

7.4 Computational Complexity

Looking at computational complexity is interesting for two reasons. First of
all it tells whether a certain problem is ‘tractable’, i.e. whether the problem
can be solved in practice. Secondly, it can tell you more about the problem.
It can tell you for instance whether something is a very general problem (i.e.,
whether the problem format can be used to formulate questions about many
different situations, such as logic), or what features makes a problem difficult. In
this section we look at the complexity of the kcg decision problem, which is the
problem of deciding for a game kcg(F, Γ, Ξ, φ) whether the first coalition Γ has a
winning strategy. We look at this problem under various assumptions, and report
four theorems, as follows:

• The first theorem is concerned with the problem of deciding whether a coali-
tion Γ can win a knowledge condition game with an empty set of opponents.
This is called the no-opponents knowledge condition game decision prob-
lem. It turns out that this problem is already NP-complete, and thus not
tractable.

• The second theorem states that the general kcg decision problem is even
harder: with opponents the problem is Σ2 P-complete.

For the other theorems we use the alternative version of knowledge condition
games kcg 0 .
• In the third theorem we claim that the no-opponents kcg 0 decision problem
is as hard as the general problem.
• Both problems are NP-complete, which is the fourth theorem.
142 Chapter 7. Knowledge Condition Games

In this chapter, we encode interpreted game forms in an explicit way, by listing

all histories. In practice protocols are often specified in an implicit way (for in-
stance in some form of source code) and such representations can be exponentially
more efficient.
7.4.1. Theorem. The problem to decide for given coalitions F, Γ and formula φ
whether w(kcg(F, Γ, ∅, φ)) = 1 is NP-complete.

Proof. Assume that F, Γ, φ are given. The empty coalition has only one strategy
σ∅ . This strategy is such that u(F, σ∅ ) = F . Therefore
w(kcg(F, Γ, ∅, φ)) = 1 ⇔ ∃σΓ m(u(F, σΓ )) |= φ
A nondeterministic polynomial algorithm for this problem exists. Find or guess
nondeterministically a strategy σΓ . Since a strategy encodes a subset of actions
available in F , the size of σΓ is smaller than the size of F and thus polynomial
in the input size. Now calculate M = m(u(F, σΓ )), and verify for each state w
of M that M, w |= φ. The number of states in M is at most the number of
terminal histories of F , so kM k ≤ kF k. All of this can be done in polynomial
time. Therefore, this problem can be solved using a nondeterministic polynomial
algorithm and this problem is in NP.
In order to show that the restricted kcg problem of the theorem is as hard
as any NP problem, we show that any instance of the 3SAT problem described
on page V 30 can be transformed into an equivalent restricted kcg instance. Let
φ3 = i (ai ∨ bi ∨ ci ) be a propositional logic formula in conjunctive normal
form with three literals per clause. The literal formulas ai , bi , ci must be either
atomic propositions or negated atomic propositions. We can construct an inter-
preted game form F with a single agent Σ = {A} and a formula φ such that
w(kcg(F, {A}, ∅, φ)) = 1 if and only if ∃S : S |= φ3 .
The model F = ({A}, H, turn, ∼, P, π) is constructed in the following way.
Let P 3 be the set of atomic propositions occurring in φ3 . The new set of atomic
propositions P contains two propositions for any old proposition: P = {x+ |x ∈
P 3 } ∪ {x− |x ∈ P 3 }. For each new proposition a history is created: H = {} ∪
{ep |p ∈ P }. The interpretation function is such that only the corresponding
atomic proposition is true: π(ep ) = {p}. Furthermore turn() = A. Agent A
cannot distinguish any end state: ep ∼A eq for all terminal histories ep and eq .
The formula φ = φ1 ∧ φ2 is a conjunction of two parts. The part φ1 expresses
that for each original atomic proposition p ∈ P 3 , either the positive proposition
p+ is considered possible or the negative p− , but not both:
^
φ1 = (MA p+ ∨ MA p− ) ∧ ¬(MA p+ ∧ MA p− )
p∈P 3

The idea is that the strategy that A uses is actually an assignment of values to
all atomic propositions in P 3 . The condition φ1 expresses that such assignment
must assign either the truth value true (p+ ) or false (p− ) to each proposition p.
7.4. Computational Complexity 143

p+ p− q+ q− r+ r−
Figure 7.4: The model of 3SAT formula ψ

The φ2 part encodes the original formula φ3 = i (ai ∨ bi ∨ ci ). In the next

V
definition we use a helper function f defined such that f (¬p) = p− and f (p) = p+ .
Using this function we define B as follows.
^
φ2 = MA (f (ai ) ∨ f (bi ) ∨ f (ci ))
i

It is not hard to see that any strategy σ{A} such that m(u(F, σ{A} )) |= φ1 ∧ φ2
corresponds to an assignment S such that p ∈ S if and only if p+ ∈ σ{X} (), and
that this assignment satisfies S |= φ3 . Since the formula and model constructed
have sizes that are linear with respect to the size of φ3 , this is a polynomial re-
duction. Therefore, the restricted kcg problem is NP-hard. Since we have also
shown that the problem is in NP, we conclude that the restricted kcg problem is
NP-complete. 

As an example, consider the satisfiability of the 3SAT formula ψ = (p ∨ ¬q ∨

r) ∧ (¬q ∨ ¬p ∨ r). This formula contains three propositions, so the corresponding
interpreted game form, depicted in figure 7.4, contains six terminal histories. The
corresponding knowledge formula is ψK .

ψK = (MA p+ ∨ MA p− ) ∧ ¬(MA p+ ∧ MA p− )∧
(MA q + ∨ MA q − ) ∧ ¬(MA q + ∧ MA q − )∧
(MA r + ∨ MA r − ) ∧ ¬(MA r + ∧ MA r − )∧
MA (p+ ∨ q − ∨ r + ) ∧ MA (q − ∨ p− ∨ r + )

A typical NP-complete problem is to determine whether a propositional logic

formula is satisfiable. Suppose φ is a formula with atomic propositions x1 , x2 , . . . xn .
We can thus write φ = φ(~x) where the vector ~x consists of all the xi . The sat-
isfaction problem can now be phrased as deciding whether ∃~x : φ(~x). In the
same way we can formulate more difficult problems, by allowing more quantifiers:
∃y∀x : φ(~x, ~y) is the problem where one has to decide whether there is an ~x such
that φ(~x, ~y) is true for all ~y. This problem, called SAT2 , is a typical Σ2 P complete
problem [81, ch. 17]. It is widely believed, but not proven, that these problems
are strictly more difficult than NP-complete problems.
144 Chapter 7. Knowledge Condition Games

B B B B

p+ r + p+ r − p− r + p− r − q + r + q + r − q − r + q − r −
Figure 7.5: The construction of the Σ2 P proof

7.4.2. Theorem. Deciding for given F, Γ, Ξ and φ whether w(kcg(F, Γ, Ξ, φ)) =

1, is Σ2 P-complete problem.

Proof. First we have to prove that this problem is indeed in Σ2 P. In order to

do this, consider the winning condition of a knowledge condition in more detail.

w(kcg(F, Γ, Ξ, φ)) = 1 ⇔ ∃σΓ ∀σΞ m(u(u(F, σΓ ), σΞ )) |= φ

Suppose that F, Γ, Ξ and φ are given. It is possible to encode strategies of

Ξ as assignments to a vector of propositional variables ~y, and the strategy of
Γ as assignments to ~x. One can then find a formula ψ(~x, ~y) that is true if
m(u(u(F, σΓ ), σΞ )) |= φ. The size of this formula is polynomial in |F | + |φ|.
The kcg decision problem is equivalent to a SAT2 problem:

w(kcg(F, Γ, Ξ, φ)) = 1 ⇔ ∃~x∀~y : ψ(~x, ~y)

Deciding whether ∃~x∀~y : ψ(~x, ~y), is a SAT2 problem, and is thus in Σ2 P.

The second part of the proof is to show that the kcg decision problem is indeed
complete for this class, and this can be done by reducing SAT2 to a knowledge
condition game. The proof is similar to the previous NP-completeness proof,
but now involves two agents. Assume that a SAT2 problem V ∃~y∀~x : ψ(~x, ~y) is
given. We can assume that ψ is in 3SAT form: ψ = i (ai ∨ bi ∨ ci ). First
we define an interpreted game form F = (Σ, H, turn, ∼, P, π). Let Σ = {A, B},
− −
and Z(H) = {(a, b)|∃i, j : a = x+ +
i or a = xi , b = yj or b = yj }. The set H
contains all histories of Z(H) and all prefixes of these histories. The function
turn is defined such that A moves first, and then B moves: turn() = A and
turn((x±i )) = B. The relations ∼A and ∼B are equal, and defined such that each
agent only knows the length of each history: s ∼A s0 ⇔ |s| = |s0 |. The set of
propositions P of the kcg problem is {z + |z ∈ (~x ∪ ~y)} ∪ {z − |z ∈ (~x ∪ ~y )}. The
function π is defined by π(a, b) = {a, b}. This completes the definition of the
interpreted game form F . The number of terminal histories of F is 2|~x| · 2|~y|, and
thus the size of F is polynomial in the size of the input problem.
We define Γ = {A} and Ξ = {B}. Next, we define an epistemic logic formula
φ such that Γ can win the game kcg(F, Γ, Ξ, φ) iff ∃~x∀~y : ψ(~x, ~y). Let φ =
7.4. Computational Complexity 145

¬φB ∨ (φA ∧ f (ψ(~x, ~y)). The part φB expresses that the strategy of B corresponds
to an assignment to ~y. The part φA expresses that the strategy of A corresponds
to a strategy for ~x. Finally, f (ψ(~x, ~y)) is a translation of the input formula ψ(~x, ~y).
^
φB = ((MB yj+ ∨ MB yj− ) ∧ ¬(MB (yj+ ∧ yj− ))
j
^
− −
φA = ((MA x+ +
i ∨ MA xi ) ∧ ¬(MA (xi ∧ xi ))
i
^ ^
f (ψ(~x, ~y)) = f ( (ai ∨ bi ∨ ci )) = (f (ai ) ∨ f (bi ) ∨ f (ci ))
i i

The function f is defined such that f (¬p) = p− and f (p) = p+ . The size
of φ is linear in the size of ψ. Therefore, this is a polynomial reduction. This
completes the proof that the knowledge condition game decision problem is Σ2 P-
hard. Since it is also in Σ2 P, we conclude that the problem is Σ2 P-complete. 

The construction of a model F is illustrated in figure 7.5. This is the model that
you would get in the reduction of ψ(~x, ~y) where ~x contains p and q and ~y consists
of r. The model is again relatively small: only two actions happen in each play
of this interpreted game form. The first one is decided by agent A, the second
one by B.
In the two previous proofs, it is essential that the agents are aware of the
strategies they choose. Both constructions would not work with the alternative
definition kcg 0 . One can hope that the computational complexity of the kcg 0
decision problem would be lower. Indeed one can prove that in this case it does
not matter whether there are opponents.

7.4.3. Theorem. Assume that F, Γ, Ξ and φ are given. w(kcg 0 (F, Γ, Ξ, φ)) = 1
iff w(kcg 0 (F, Γ, ∅, φ)) = 1.

Proof. Let G = kcg 0 (F, Γ, Ξ, φ) be a kcg 0 decision problem. Notice that the
goal of coalition Ξ is to choose a strategy σΞ such that U0 (σΓ , σΞ ) = (0, 1), where
U0 is the utility function of the game G. Since U0 is defined using universal
quantification over the set of terminal histories of u(u(G, σΓ ), σΞ ), the best thing
to do for coalition Ξ is to make sure that this set is as large as possible. In order
to achieve this, σΞ should choose the neutral strategy that allows any action: The
strategy σ with σ(h) = A(H, h). Since we have assumed that neutral agents can
do any action, we might as well assume that the agents X ∈ Ξ are neutral, and
determine the value of the game w(kcg 0 (F, Γ, ∅, φ)) = 1. 

We see thus that the presence of opponents is not relevant, and indeed in ATEL
no distinction between opponents and neutral agents is made. The question is
146 Chapter 7. Knowledge Condition Games

now whether solving the kcg 0 decision problem is still as hard as the original no-
opponents kcg problem. The answer is yes. The no-opponents kcg 0 problem is
also NP-complete. However, the proof is different in an interesting way.

7.4.4. Theorem. Deciding for given F, Γ and φ whether w(kcg 0 (F, Γ, ∅, φ)) = 1,
is an NP-complete problem.

Proof. We can prove that this problem is in NP by a similar argument as given

for theorem 7.4.1. V For the hardness result we again show a reduction from 3SAT.
Assume that φ = ni=1 (ai ∨bi ∨ci ) is a propositional formula in conjunctive normal
3

form with three literals per clause. Let P 3 be the set of atomic propositions
occurring in φ3 . We define an interpreted game form between two agents: an
agent Q that asks questions, and an agent A that answers them. The proponent
coalition is Γ = {A} and Q is assumed to be neutral. Every terminal history is of
the form (p, b, i, x), where p ∈ P 3 , b ∈ {0, 1}, i ∈ {1, 2, . . . , n} and x ∈ {ai , bi , ci }.
The first action p is chosen by agent Q and must be one atomic proposition of φ3 .
The agent A must then reply by giving a boolean value b. This indicates what
truth value A has in mind for p. Then agent Q choses one triplet (ai ∨ bi ∨ ci ) that
appears in φ3 . Agent A then has to choose which of these three parts it thinks
should be true: either ai or bi or ci . The trick however is that ∼A is defined in
such a way that for all histories h and h0 , agent A only knows the length of the
histories: h ∼A h0 iff |h| = |h0 |.
A does not know, when making its final decision, which answer it has given on
its first turn. The agent thus risks giving inconsistent information. For instance
in the history (p, 1, (¬p ∨ q ∨ r), ¬p) agent A first says that p is true, and then
says that it thinks that ¬p holds. The goal of agent A in the game is to avoid
these inconsistent histories. We let P = {e} consist of one proposition and
define for all p ∈ P 3 the interpretation function such that π((p, 1, i, ¬p)) = {e},
π((pj , 0, i, pj )) = {e} and π((p, b, i, x)) = ∅ otherwise. One can now consider the
knowledge condition game G = kcg 0 (F, {A}, ∅, ¬e). Agent A can win the game
G iff there is a satisfying assignment for φ3 . 

The proof given above is very similar to a proof given by Schobbens [91] for
the NP-completeness of ATL with imperfect information. This corroborates our
claim that this variant of knowledge condition games is closely related to ATL
and thus to ATEL. The proof exploits the fact that in games where coalitions do
not have perfect recall, it is very difficult for agents and coalitions to coordinate
their own actions.

7.4.1 Tractable Variants

In the previous section we proved that, in general, the kcg decision problem is
not tractable. In this section we identify some easier cases.
7.4. Computational Complexity 147

7.4.5. Theorem. Let F be an interpreted game form with perfect recall, Γ any
coalition of agents and φ an epistemic formula. Deciding whether w(kcg 0 (F, Γ, ∅, φ)) =
1 can be done in polynomial time.

Proof. Let M = (Σ, W, R, P, π) = m(F ) be the end state model of F . We can

compute the set S = {w ∈ W | M, w |= φ} in polynomial time. Define a utility
function U such that U(w) = 1 iff w ∈ S and U(w) = 0 otherwise. The pair F, U is
now an extensive game with perfect recall. The optimal solution σ for this game
can be computed in polynomial time [58]. If the expected payoff of σ is exactly
one, then w(kcg 0 (F, Γ, ∅, φ)) = 1, otherwise w(kcg 0 (F, Γ, ∅, φ)) 6= 1. 

For perfect recall frameworks and the variant kcg 0 the decision problem is thus
tractable. One might wonder whether the same claim can be made for kcg. The
answer is no, because one can modify the NP-completeness proof for kcg in such a
way that it uses a perfect recall interpreted game form. The modification is that
one has two agents, A and A0 , so that A is the agent that chooses a strategy, and
A0 is the agent that cannot distinguish end states and occurs in the knowledge
condition. In general one can always find a perfect recall interpreted game form
that is equivalent for the kcg decision problem by choosing a fresh agent for each
decision node, and use fresh agents in the knowledge condition.
Instead of asking whether there are interpreted game forms F that make
decision problems easy, one can also ask whether there are easy formulas φ. The
answer to this question is yes. To see how this works, we first formulate the
notion of positive formulas and negative formulas formulas.

7.4.6. Definition. For any p ∈ P , the formula p is both positive and negative.
Falsum ⊥ is also both positive and negative. If φ is positive and ψ is negative,
then φ → ψ is negative. Vice versa, if ψ is positive and φ is negative, then φ → ψ
is positive. If φ is positive then KX φ is positive.

Positive and negative formulas are both called monotone formulas, because one
can prove that they preserve truth in the following way. Suppose that M =
(Σ, W, ∼, P, π) and M 0 = (Σ, W 0 , ∼0 , P, π 0 ) are models such that W 0 ⊆ W and ∼0
and π 0 are the restrictions of ∼ and π to W 0 . In this case we say that M 0 is a
submodel of M . Suppose φ+ is a positive formula, and φ− is a negative formula.
Then the following statements can be proven.

M, w |= φ+ implies M 0 , w |= φ+
M 0 , w |= φ− implies M, w |= φ−
The proof of these statements is done by induction over the formula structure.
The interesting step involves the knowledge operator. Suppose that M, w |=
KX φ+ . By definition this means that ∀v ∈ W : w ∼X v =⇒ M, v |= φ+ . Since
W 0 is a subset of W , this means that ∀v ∈ W 0 : w ∼0X v =⇒ M, v |= φ+ . Using
148 Chapter 7. Knowledge Condition Games

the induction hypothesis we obtain ∀v ∈ W 0 : w ∼0X v =⇒ M 0 , v |= φ+ and thus

M 0 , w |= KX φ+ .
Knowledge condition games with monotone formulas are easier to solve than
general knowledge condition games.
7.4.7. Theorem. The problem to decide for given F, Γ, Ξ and a monotone for-
mula φ whether w(kcg(F, Γ, Ξ, φ)) = 1 can be solved in polynomial time.

Proof. We prove the case where φ is a positive formula. The argument for
negative formulas is similar. Recall that by definition, w(kcg(F, Γ, Ξ, φ)) = 1
iff ∃σΓ ∀σΞ ∀w ∈ W it holds that m(u(u(F, σΓ ), σΞ )), w |= φ where W is the set
of worlds in the model m(u(u(F, σΓ ), σΞ )). Since φ is a positive formula, we
know that m(u(F, σΓ )), w |= φ implies m(u(u(F, σΓ ), σΞ )), w |= φ. The best thing
for coalition Ξ to do is to use a strategy that does not eliminate any action.
They should use a neutral strategy σ 0 such that u(F, σ 0 ) = F . This strategy is
described by σ 0 (h) = A(H, h).
For coalition Γ things are exactly opposite. Suppose that σ 1 and σ 2 are
strategies so that σ 1 is more specific than σ 2 . Formally, this means that ∀h :
σ 1 (h) ⊆ σ 2 (h). The monotonicity of φ implies that m(u(F, σ 2 )), w |= φ implies
m(u(F, σ 1 )), w |= φ. Coalition Γ thus does best be choosing the more specific
strategy σ 1 . For coalition Γ we thus only have to consider the most specific
strategies. These most specific strategies are what one can call pure, because they
select exactly one action at each decision point. A backward induction argument
can be used to show that there are as many pure strategies for F as there are
terminal histories in F . We can try all pure strategies σ p to see if one satisfies
∀w : m(u(F, σ p )), w |= φ. This gives an algorithm that needs time O(kF k2 · kφk).
The first kF k is caused by the fact that we need to consider all pure strategies.
The remaining term kF k · kφk is the time needed to determine whether for all w
it is the case that u(M, σ p ), w |= φ. The decision problem can thus be done in
polynomial time.
For negative formulas the roles of Γ and Ξ are interchanged. For a negative
formula φ, coalition Γ can use the neutral strategy σ 0 . The opponent coalition Ξ
should now try all pure strategies σ p . 

7.5 Related Work

Knowledge conditions games are games based on epistemic logic, that can be used
for modelling games about knowledge. This makes them very similar to the logic
ATEL [102, 103]. This logic is an extension of epistemic logic with operators to
talk about group abilities and time. It is based upon ATL, discussed in chapter
2.
7.5. Related Work 149

The language of ATEL contains temporal operators similar to CTL and know-
ledge operators. The temporal operators are always preceded by an agent oper-
ator.

7.5.1. Definition. Let Σ be a set of agents, and P a set of atomic propositions.

The logic ATEL contains formulas φ generated by the following rule. In this rule,
p is a typical element of P , X ∈ Σ and Γ ⊆ Σ

φ ::= p | φ → φ | ⊥ | hhΓiiψ | KX φ
ψ ::= 2φ | φ U φ

This logic is interpreted over alternating epistemic transition systems. These are
defined as tuples (P, Σ, Q, ∼, π, δ). As usual P is a set of atomic propositions and
Σ a set of agents. The set Q is a set of states the system can be in, and π : Q → P
adds propositions to these states. For any agent X the relation ∼X ⊆ Q × Q is
Q
an equivalence relation, and δ : Q × Σ → 22 assigns to each agent in each state
a set of sets of states. Each agent can choose one set of states, and the next state
of the system will be from that set.
An example would be a system where Q = {0, 1, 2, 3, 4}. Suppose that
δ(0, X) = {{1, 2}, {3, 4}} and δ(0, Y ) = {{1, 3}, {2, 4}}. Agent X can now choose
{1, 2} and Y can choose {2, 4}. They make these choices simultaneously. The
next state of the system will be 2, because that is the only common state in their
chosen sets. It is necessary to put some constraints on δ so that a next state can
always be chosen.
The interpretation of this logic uses the notion of strategy to interpret the
coalition operator hhΓii. A strategy for Γ is any function that makes a choice
σΓ (X, q) ∈ δ(q, X) for any agent X ∈ Γ in any state q ∈ Q. Based on a strategy
σΓ , one can define the set of possible walks W(σΓ ) through Q so that all choices
for agents X ∈ Γ are made as recommended by the strategy. This set of walks is
used in the following interpretation of ATEL.

M, q |= ⊥ never
M, q |= p where p ∈ P iff p ∈ π(v)
M, q |= φ → ψ iff M, q |= φ implies M, q |= ψ
M, q |= KX φ iff ∀(q, q 0 ) ∈∼X : M, q 0 |= φ
M, q |= hhΓiiφ iff ∃σΓ : ∀w = v... ∈ W(σΓ ) : M, w |= φ

M, w |= 2φ iff ∀n > 0 : Q, w(n) |= φ

M, w |= φ U ψ iff ∃m > 0 : M, w(m) |= ψ and
∀m > k > 0 : M, w(k) |= φ
150 Chapter 7. Knowledge Condition Games

A main advantage of ATEL over kcg is that ATEL extends temporal logic, and
can thus be used to express different kinds of goals such as eventually achieving
something, or avoiding some state forever. When this logic was presented it was
reported that the logic has a low model checking complexity [102]. Unfortunately
this only holds if one allows strategies that are not uniform (see definition 3.3.13).
If one demands uniform strategies, model checking becomes NP-complete, even
without using the knowledge operator [91]. Another point of discussion for this
logic is the fact that the existence of a strategy, used in the interpretation of
hhΓiiφ, is a very weak condition. One can come up with situations were hhXiiφ
holds but one would not expect X to achieve φ [54, 55, 109]. Thus, it seems that
the interpretation of this logic still needs some sorting out, and indeed ATEL
currently receives a lot of research attention [3, 87].
Knowledge condition games is a less versatile verification framework than
ATEL, because kcg does not allow complicated temporal reasoning. Only the
special case of knowledge at the outcome stage of the protocol is studied. Know-
ledge condition games also do not allow for concurrent moves. This has the
advantage that knowledge condition games are easier to understand, and that
the complications that arise in the interpretation of ATEL do not arise in the
context of knowledge condition games. An interesting difference between ATEL
and kcg is that in kcg nondeterministic (and hence arguably “richer”) strategies
are used, whereas ATEL uses deterministic strategies.
One can also compare knowledge condition games to variants of dynamic
epistemic logic, described on page 54, since dynamic epistemic logic allows rea-
soning about the effect of actions on the knowledge of agents. Indeed the quiz
master problem is inspired by Van Ditmarsch’ analysis of the Russian Cards
problem [106].

7.6 Conclusion
By combining protocols and knowledge conditions into games, one can express
properties of multi-agent protocols relating to security and secrecy. In a know-
ledge condition game, one can make fine distinctions between for instance neutral
and opponent agents, and one can give examples where this distinction is signif-
icant. Therefore, these games are a promising direction for future research into
the interaction between knowledge and strategies.
The complexity results reported in this chapter draw an interesting picture.
There seems to be a computational cost for assuming that agents know strate-
gies. The single agent decision problem is already intractable. The presence of
opponents makes it even harder to compute whether a coalition can guarantee
a property. If we drop this assumption and reformulate the notion of winning a
knowledge condition game, then the extra complexity of adding opponents dis-
appears. However, the problem without opponents is still NP-complete and thus
7.6. Conclusion 151

intractable, but for different reasons. The complexity proof is no longer based
upon formulating a difficult knowledge formula, but on the hardness of coordi-
nating in an interpreted game form without perfect recall.
Future research could focus on comparing decision problems for knowledge
condition games to other game-theoretic decision problems, in order to establish
what exactly the complexity cost is of considering knowledge goals. It would
also be interesting to find out under which assumptions knowledge condition
games can be solved in polynomial time. Other directions include looking at
knowledge condition games from a logical viewpoint by searching for axioms, and
to consider the mechanism design problem to find an interpreted game form with
given properties.
Chapter 8
Entropy and Privacy

8.1 Introduction
Information is valuable, and thus agents do not always want to give it away. Both
organisations and individuals often want to keep certain information private. At
the same time they might want to act upon it. Does this reveal the information?
In this chapter we study how agents should act if they want to maximize their
utility, while at the same time not giving away too much information. Unlike
the previous chapter, in this chapter we do this based on explicit probabilities.
We define two classes of games in which the utility for each agent does not only
depend on the payoff of the chosen action, but also on the information properties
of the strategy used. These games are called minimal information games and
most normal games and might be applied to the following situations.

• Supermarkets and e-commerce shops register what is bought by each of their

customers. Customers know this and even assist in this process by using
so-called ‘bonus cards’ (Albert Heyn) or ‘club cards’(Tesco). Nevertheless,
many customers are worried about their privacy. They would prefer it if
the shop knew less about them. Customers can do something to minimize
the knowledge of the shop. First of all they can make their shopping less
regular (i.e. randomly buy items so that the shop is not sure which products
the customer actually uses). Secondly, they can sign up for more than one
card(account) or swap cards between each other. On the Internet, deleting
cookies at random intervals and using a different IP number can have the
same effect.

• In a second price auction it is optimal to bid exactly as much as you think

the item is worth [63]. However, you might have spent a lot of time to
estimate the value of the item, so you do not want to reveal your estimate.
Since your bid has to be public, it seems that you might do better by bidding
slightly random. By modeling this as a minimal information game, one can

153
154 Chapter 8. Entropy and Privacy

compute how one should randomise. A similar argument applies when you
send out an artificial agent to do your shopping. If the agent is sent over
an insecure network, everyone can inspect the source code and thus the
bidding strategy of the agent. You might not want to send an agent that is
exactly optimal for your preferences, in order to hide your preferences.
• Many public places are now monitored by closed circuit television systems.
If you come to one such place regularly, the camera attendants learn a lot
about your habits and thus about you. You feel that this is a breach of your
personal privacy, and decide to hide your habits by changing your behaviour
often, for instance by going to different shops in a different order every time.
This situation can also be modeled as a minimal information game. Again
one can translate this example to the domain of artificial agents and the
Internet.
• Consider now the case of a criminal who wants to steal from a shop guarded
by a closed circuit television system. He wants to look like a regular shopper,
but has different goals. He thus wants to behave so that he can steal the
most, while at the same time appear to be a normal shopper. This can be
modeled as a most normal game.

As the similar setting of the last two examples suggest, minimal information
games and most normal games are related to each other. From these examples
it should also be clear that we assume that the strategies that agents use are
publicly known. This assumption makes our results stronger (if you have privacy
while your strategy is public, you will have even more privacy when you can keep
your strategy secret).
Privacy has received a lot of attention from economists and in legal settings.
Some key sources have been collected on a website [1]. The work in this chapter
differs from these economic papers for two reasons. First of all we only deal with
personal information privacy, whereas the word ‘privacy’ also has other meanings.
The second difference is that these papers try to explain the need for personal
privacy in terms of economic utility. Odlyzko for instance relates privacy and
price discrimination [77]. It is assumed here that privacy is a fundamental value,
that is not instrumental to any gain. Privacy itself is a good cause that can be
enjoyed directly.
Distributed constraint optimization techniques can be used by agents to solve
coordination problems such as scheduling a meeting at the most suitable time
and place. In these applications agents have to reveal information on their pref-
erences for the meeting, but this information is also privacy-sensitive [33, 67].
In this application domain there is also a trade-off between solution quality and
privacy, and this can also be modeled using entropy [33]. Thus, privacy-related
research certainly has practical applications and it would be interesting to study
these further. Therefore, we agree with Maheswaran and others’ [67] ‘call to arms
8.1. Introduction 155

to improve privacy protection algorithms and further research on privacy’. The

results of this chapter can be seen as a response to this call, since the strategies
that are developed in this chapter can be used within privacy protection algo-
rithms. For example the agent in the examples in section 8.7 use a randomised
strategy, computed using the results of this chapter, in order to make it as hard
as possible for observers to learn their preferences.
The games defined in this chapter use a soft (probabilistic, quantitative) ap-
proach towards information. They deal with probabilities explicitly, and can
make subtle distinctions between possible, likely and almost certain events. This
soft approach can be contrasted to the hard approach (discrete, qualitative) of
logic and model checking. When taking a hard approach in protocol analysis, one
is only interested in what is possible and what not, with a complete disregard
for the relative likelihoods of different outcomes. Both the soft and the hard
approach have been used for multi-agent systems. The use of epistemic logic
to understand the game of Cluedo [106] is an example of the hard approach, as
well as other logical approaches to reasoning about knowledge and knowledge
change [6, 9, 32, 102, 115]. Recent work on privacy preserving auctions [18]
and work on the Dining Cryptographer problem [19] or the Russian Cards prob-
lem [106, 112] can also be classified as ‘hard’. At the same time there is some work
on reasoning about uncertainty [43, 60] that combines logic and a soft approach
to information. The soft approach is more detailed than the hard approach, be-
cause it gives exact probabilities. In certain circumstances this is an advantage.
The hard approach can tell us that agents do best by randomising their strat-
egy, but does not indicate the exact probabilities of an optimal strategy. On the
other hand the higher level of abstraction of the hard approach makes it easier
to interpret the results.
A quantitative approach, based on information theory, can also be used to
look at natural language pragmatics. See for instance the ongoing work by Van
Rooij [116]. Another way to use entropy in a game-related setting is in a searching
game such as Mastermind [59].
In this chapter, the focus is on strategic games, whereas in most previous
chapters of this dissertation we use extensive games. The reason is that it is
quite complicated, starting with the notation, to do a similar exercise for extensive
games. It is also not necessary: An extensive game is a more detailed description
of a strategic game, so the results of this chapter can be applied to extensive
games.
The layout of this chapter is as follows. Section 8.2 describes a detailed
example problem. The next section, section 8.3, introduces basic information
theory notions such as entropy. In section 8.4 we define minimal information
games, and calculate the best strategies in these games. In section 8.5 we do
the same for most normal games. Section 8.6 shows that these concepts can be
used for defining new solution concepts. As application is discussed in section
8.7. Finally, the conclusions are presented in section 8.8.
156 Chapter 8. Entropy and Privacy

8.2 Example
The following problem serves as an example. Alice (agent 1) needs to buy one
box of breakfast cereals every week. Every week she is faced with the following
choice: whether to buy Allgrain (A), Barley (B) or Cornflakes (C). Alice is not
indifferent to which brand she eats. In fact she likes A better than B and B
better than C, as is indicated by the following matrix of utilities.

action A B C
utility 3.0 2.0 1.0

If Alice is solely interested in maximising her expected utility, she should buy
A every week. However, Alice knows that the shop is watching her shopping
behaviour closely, and she is concerned about her privacy. She decides that the
decision that she makes should be private, and she can achieve this by flipping a
coin and letting her decision depend on this coin flip. This way the shop cannot
predict her decision.
Alice first attempts to use the following random strategy.

action A B C
probability 0.98 0.01 0.01

If Alice uses this strategy, then the shop does not know anything about her
decision: All three actions may occur with positive probability. At the same time
her expected payoff is still very high, because the suboptimal actions occur with a
very low probability. Problem solved, so it seems. But this is not the whole story.
Even though the shop does not gain any knowledge, it does gain information from
this strategy. If the shop learns, from repeated observation, that Alice uses this
strategy, then it is quite certain that she will buy A. The shop has gained quite
a lot of information. Therefore, the indicated strategy is not the right strategy if
one analyses the situation using information theory.
One can argue that no new types of games are needed, because one can capture
Alice’s wish for privacy in the utility function of some modified pure or mixed
strategy game. This is not the case because in these games the utility of strategies
is determined solely by the utility of single actions: The utility function must be
of the form U = Σa p(a)u(a), where p(a) is the probability of action a, and u(a)
the payoff of this individual action. Privacy and uncertainty are not reducible to
certain individual actions, and therefore no suitable pure or mixed strategy game
can be found.
A more sophisticated idea is to model privacy by adding an extra player G
that tries to guess Alice’s actions. In such a game, Alice would gain a high payoff
by randomising her actions, and thus optimal strategies for this game would also
be privacy-preserving strategies. The following payoff matrix gives such a game.
The parameters 1 , 2 , 3 , and δ1 , δ2 , δ3 are all positive.
8.3. Information Theory 157

G\ Alice A B C
A 1 , 3.0 − δ1 0, 2.0 0, 1.0
B 0, 3.0 2 , 2.0 − δ2 0, 1.0
C 0, 3.0 0, 2.0 3 , 1.0 − δ3
This strategic game, in which both agent choose their strategy independently at
the same time, has been designed such that agent G has incentives i to choose the
same action as Alice, while Alice receives penalties δi if G has ‘guessed’ her next
action correctly. This game is thus arguably a good model for a situation in which
A wants privacy. It is however not clear how one should estimate all the variables
that one needs for this larger game. These considerations have convinced us that
it is easier to treat privacy as an independent aspect of an agent’s utility.

8.3 Information Theory

Information theory is the field of science that deals with the measurement of in-
formation [28]. It has applications in signal processing, communication networks,
cryptography and error correction codes. In this chapter we use information the-
ory, and its central notion entropy, to estimate the amount of information in
strategies. Strategies will be modeled as stochastic variables ranging over a finite
set of actions, so we define entropy over stochastic variables. The entropy of a
stochastic variable is the amount of randomness in, the disorder of, or uncertainty
about the value that the variable will take. The concept of entropy was intro-
duced by Shannon [95], and it is widely seen as the most natural measure for
information [28]. We define the following function f (x, y), that is helpful for the
definition of entropy. Let lg be the base 2 logarithm.

 0 if x = 0 and y = 0
f (x, y) = ∞ if x > 0 and y = 0
−x lg y if x ≥ 0 and y > 0


For a discrete random variable X we define the entropy E(X), which is measured
in bits, in the following way.
X
E(X) = f (p(X = k), p(X = k))
k

This definition of entropy does not work for continuous random variables. A
different definition for continuous variables also exists [95, p. 35], based on in-
tegration rather than summation. Since this is slightly more complicated and
continuous random variables are not used in this chapter, the details are not
discussed here.
A random variable X with values in the domain {1, 2, . . . , m} can be speci-
fied by giving a vector of length m with the probabilities of each value: (p(X =
158 Chapter 8. Entropy and Privacy

1
entropy

0.9

0.8

0.7

0.6
entropy

0.5

0.4

0.3

0.2

0.1

0
0 0.2 0.4 0.6 0.8 1
probability

Figure 8.1: The function E((x, 1 − x))

1), p(X = 2), . . . , p(X = m)). For a mixed strategy, the numbers {1, 2, . . . , m}
represent the available actions. A requirement for probability measures on stochas-
tic variables is that the probabilities should add up to 1. We can thus only use
vectors x that indeed add up to 1, so it is convenient to define the set of all these
vectors. The set definition of the set Pm from page 40 is repeated here, and we
also define Qm as the set of nonzero vectors.
X
Pm ={x ∈ [0, 1]m | xi = 1}
i
X
m m
Q ={x ∈ (0, 1] | xi = 1}
i

The set Pm contains all vectors of length m that add up to 1, and Qm contains
all vectors that add up to 1 and do not take the value 0. The set Qm is important
in some of the proofs, but often we work with the more general set Pm . We can
apply the notion of entropy to probability vectors x ∈ Pm .
X
E(x) = f (xk , xk )
k

In figure 8.1 the function E((x, 1 − x)) is displayed (here we apply the function
E to a probabilty vector (x, 1 − x) that depends on a variable x ∈ [0, 1]). Thus
the figure shows the entropy of a two-valued random variable (y1 , y2 ) = (x, 1 − x)
, where x is the probability of the first action, and 1 − x the probability of the
second action. As you can see the entropy in the two pure strategies, namely
(1, 0) and 0, 1 is zero. The entropy is maximal if both actions are equally likely,
8.3. Information Theory 159

at (0.5, 0.5). In the context of strategies, a strategy with a higher entropy leaves
observers with more uncertainty, and thus gives the agent that uses that strategy
more privacy. Below we give five examples of entropy. The example strategy
vectors can all be seen as strategies over three basic actions. A strategy (a, b, c)
contains the probability a of selection the first action, b for the second action and
c for the third.

E((1/3, 1/3, 1/3)) = 1.585 bits

E((0.5, 0.25, 0.25)) = 1.5 bits
E((0.5, 0.5, 0)) = 1 bit
E((0.98, 0.01, 0.01)) = 0.161 bits
E((1.0, 0, 0)) = 0 bits

Pure strategies, in which only one action gets a positive probability, have an
entropy of zero bits. The entropy function is bounded. It cannot be negative,
and a vector x of length m can have at most an entropy of lg m. It has this entropy
if all the entries xi are equal to 1/m, thus if the vector represents a stochastic
variable with a uniform distribution.
The second idea that we use from information theory is relative entropy [28].
The function E rel (x, y) can be used to compare two probability vectors x, y ∈ Pn .
The underlying idea is that E rel (x, y) measures how much difference one would
notice if probability vector x is used instead of y for selecting actions. In order
to compute this difference, we add up the differences for each action k. The
probability xk corresponds to the probability that action k is chosen, given that
strategy x is used: xk = P (k|x). Similarly yk = P (k|y). Using Bayes’ law one can
calculate the relative likelihood of strategy x instead of strategy y when observing
that action k is chosen: P (x|k)/P (y|k). Assuming that the a priori probabilities
P (x) and P (y) are equal, one can derive that this is xk /yk .

P (x|k) P (x ∩ k)P (k) P (k|x)P (x) P (k|x)

= = =
P (y|k) P (y ∩ k)P (k) P (k|y)P (y) P (k|y)

This observation is the motive behind the following definition.

X
E rel (x, y) = f (xk , yk /xk )
k

The function E rel almost behaves as a distance function or metric. It is never

negative and only zero if x = y. It also satisfies the triangle inequality. It is infinite
if for some k it is the case that xk > 0 and yk = 0. The only difference between
this function and a distance function or metric is that E rel is not symmetric. In
160 Chapter 8. Entropy and Privacy

many cases E rel (x, y) 6= E rel (y, x).

E rel ((0.5, 0.5), (0.75, 0.25)) = 0.2075 bits

E rel ((0.75, 0.25), (0.5, 0.5)) = 0.1887 bits
E rel ((0.9, 0.1), (0.75, 0.25)) = 0.1045 bits
E rel ((0.75, 0.25), (0.9, 0.1)) = 0.1332 bits

If x has a higher entropy than x0 , then on average for a random vector y it is the
case that E rel (y, x) < E rel (y, x0 ). It is harder to notice a difference between y and
a high entropy vector x than to notice a difference between y and a low entropy
vector x0 .

8.4 Minimal Information Games

The next definition of a minimal information game aims to capture the following
situation. Agents choose a mixed strategy with two goals in mind. First of all,
they want a high payoff. Secondly, they want privacy. They feel that they have
more privacy if others are more uncertain about the action they will choose, and
thus they prefer strategies with a high entropy. These games thus model the
situation where agents have a fundamental desire for privacy.
We have to specify how the agent would like to trade privacy against payoff.
This is governed by a parameter α > 0 that indicates the value of privacy. It
expresses how much expected payoff the agent is willing to trade against a bit of
privacy. The higher α, the more the agent values privacy.

8.4.1. Definition. Let A be a m1 × m2 . . . × mn multi-matrix and α > 0. The

minimal information game Miα (A) is a tuple (Σ,
P{S}Xσ , U) where Σ = {1, 2, . . . , n},
mX
the strategy sets are SX = P and U (~s) = i si Ai (~s) + αE(sX )
X X

The parameter α regulates how much all the agents value the fact that there
is uncertainty over their next action. If we would allow α = 0, then the game
becomes a mixed strategy game: Mi0 (A) = Mx(A). As α approaches infinity, the
actual payoff becomes less and less important. It would have been possible to
choose α differently for each agent, but this would have made the definition less
clear.
As an example, we consider the shopping game from the introduction. This
game has only one agent, that has three options A, B, C with respective payoffs
3, 2, 1. The optimal strategies for the minimal information game with different
values of α is given in the next table. It also lists the utility of s that the agent
would get in the mixed strategy game Mx(A) for the given strategy s and the
utility that the agent would get in the minimal information game Miα (A).
8.4. Minimal Information Games 161

α p1 p2 p3 Mx(A) Miα (A)

0.1 0.999 4 · 10−5 2 · 10−9 3.0 3.0
0.5 0.876 0.117 0.015 2.852 3.168
1.0 0.665 0.244 0.090 2.575 3.775

The best payoff that the agent can get is 3.0 by only choosing the first action.
However this would result in no privacy, because if everybody knows that the
agent uses this strategy, then any observer knows beforehand what the agent will
do every week. For a low value of α the utility of s in Miα (A) is very close to
this optimal value of 3. For higher values, the average payoff without entropy
becomes lower. We could call this the cost of privacy. From the table we can
see that if the agent values privacy at one unit per bit (α is expressed in units
per bit) then the agent does best by paying 0.425 in order to obtain 0.775 bits of
privacy.
The question is of course how we can calculate the strategies that maximize
the utility in minimal information games. For the linear functions of the mixed
strategy games this is a solved problem, but for more complicated functions, such
as the utility function of a minimal information game, this can be difficult. In
the next theorem the solution for this optimisation problem is shown.

8.4.2. Theorem. Let Miα (A) be a minimal information game and ~s a strategy
profile. The set bX (~s) is a singleton {b} such that
−1 X
2α Ai (~s)
bi = P α−1 AX (~s)
k2
k

Proof. Let Miα (A) = (Σ, {SX }X∈Σ , U) be a minimal information game. We
have to prove that the set bX (~s) contains one element, and that that element is
described by the given formula. We first show that all points in bX (~s) are interior
points. Then we derive an equation that any best response must satisfy, and show
that this equation has a unique solution, namely the one given in the theorem.
Let n be the number of actions that agent X can choose from. Take any
vector ~x ∈ SX and assume that ~x ∈ Pn \ Qn . We are going to show that
there is a better vector ~y, and thus ~x is not a best response. There is some
i such that xi = 0 and some j such that xj 6= 0. We will show that there
is some  such that ~y = [[x−i , ]−j , xj − ] is a better vector: UX ([~s−X , ~y]) >
UX ([~s−X , ~x]). To show this, note that the utility function UX is continuous and
differentiable. Note further that δxδ i UX ([~s−X , ~x]) = +∞ and δxδ j UX ([~s−X , ~x]) <
+∞. Therefore, for sufficiently small , the gain from raising xi outweighs the
potential loss from lowering xj . Therefore, for sufficiently small  we have that
UX ([~s−X , ~y]) > UX ([~s−X , ~x]) and thus ~x ∈
/ bX (~s).
Now suppose that b ∈ b (~s). We know that b ∈ Qn . Take i, j ∈ {1, 2, . . . , m}
X

as two different indices. Since b is optimal, it should not be possible to increase UX

162 Chapter 8. Entropy and Privacy

by increasing bi while decreasing bj , and therefore for any optimal point it holds
that δbδ i UX ([~s−X , b]) = δbδj UX ([~s−X , b]). We can use this as a starting point for the
following link of equations. First we compute the derivative δbδ i UX ([~s−X , b]).

δ X
U ([~s−X , b]) =
δbi
δ X
( bj A X s−X , b]) + αE(~b)) =
j ([~
δbi j
δ
AX
i (~
s) + α (E(~b)) =
δbi
AX s) + α(− lg bi − lg e) =
i (~
AX s) − α lg bi − α lg e
i (~

Using this derivative one can reduce the equation given above in the following
way.
δ X δ X
U ([~s−X , b]) = U ([~s−X , b]) ⇔
δbi δbj
AX s) − α lg bi = AX
i (~ s) − α lg bj
j (~ ⇔
AX
i (~
s) − AX
j (~
s) = α lg bi − α lg bj ⇔
AX
i (~
s)
2 bαi
=
2Aj
X (~
s) bαj

Since b ∈ Pn it holds that b sums up to i bi = 1. For any b ∈ b(~s) one can

P
−1 X
find some positive constant c such that bi = c · 2α Ai (~s) . It now follows from the
−1 X
above equation that for any bj it is the case that bj = c2α Aj (~s) . We can now
X −1 X
calculate k bk = 1 = c k 2−αAk (~s) and thus we know that 1c = k 2α Ak (~s) .
P P P
Thus, we have proven that there is a unique point b ∈ bX (~s) which satisfies the
equation in theorem 8.4.2 

8.4.3. Theorem. Every minimal information game Miα (A) has a Nash equilib-
rium.

Proof. Let f be the function from S1 × . . . × Sn to S1 × . . . × Sn that returns the

strategy vector with the best responses for each agent. Thus, f is the function
that for each x returns the unique point f (x) such that f (x) ∈ b(x). The previous
theorem shows that this is a continuous function. The set S1 ×. . . Sn is topological
isomorphic to some closed sphere Bm . We can now use Brouwer’s fixed point
theorem, which tells us that every continuous function f : Bm → Bm must have
8.4. Minimal Information Games 163

a point x with f (x) = x [4]. We thus obtain a strategy vector x with f (x) = x,
and thus a point x such that x ∈ b(x). This point is a Nash equilibrium. 

This proof is related to Nash’s original proof that Nash equilibria exist in mixed
strategy games by the fact that both theorems can be proven using Brouwer’s
fixed point theorem. The difference however is that the mixed strategy games
have linear payoff functions. Minimal information games do not have linear payoff
functions, so in this proof the fixed point theorem is used in a different way.
The two theorems of this section, theorm 8.4.3 differ in their constructiveness.
Theorem 8.4.2 gives a concrete way to compute optimal responses in minimal
information games. This theorem can therefore be applied immediately. Indeed
we have used the result formula of this theorem to compute the optimal strategies
in the table on page 160. Thus one can immediately apply this theorem in order
to decide how to act, or to predict how others will act, in situations that can
be modelled as minimal information games. Indeed in section 8.7 we apply the
theorem again to find strategies for agents.

Bach or Stravinsky
Theorem 8.4.3 is not immediately applicable, because it does not tell one how one
should find a Nash Equilibrium. It is thus not constructive in a practical sense.
However it is important to know that a Nash equilibrium exists, since this can
be a strong motivation for finding one. In the next example we use the following
bi-matrix A for defining a two-person minimal information game.
2,1 0,0
0,0 1,2

This matrix is often used in a game called Bach or Stravinsky[79, p. 16]. The
story behind these payoffs is that both agents can decide where they want to
go tonight, either to a Bach concert or a Stravinsky concert. Both agents enjoy
each others company, and hence they receive zero payoff if they do not go to the
same concert. The first agent prefers Bach and thus experiences 2 units of value
when both agent choose the first option. The second agent values Bach at 1 and
Stravinsky as 2.
Since we are interested in privacy, we assume that both agents value their
privacy. Hence we define a minimal information game Miα (A), where α = 0.5. As
we have seen in theorem 8.4.2 it is optimal for agents to randomize their behaviour
somehow. Theorem 8.4.3 tells us there is at least one Nash equilibrium. We have
used computer search to find one for the stated value of α.

Agent prob. action 1 prob action 2

1 0.148 0.851
2 0.042 0.957
164 Chapter 8. Entropy and Privacy

One can see that in this Nash equilibrium Stravinsky is the most likely outcome.
Both agents choose action 2 most often. However they do not do this with
absolute certainty, in order to leave some uncertainty for observers. The exact
probabilities are different for both agents since they have slightly different payoffs.

8.5 Most Normal Games

So far we have discussed the situation in which the agents try to protect their
privacy against an opponent interested in their next action. In this section we
look at another situation, in which agents try to hide their preferences. It is
assumed that an average strategy for ‘normal’ users is given. One agent however
has different preferences from the normal users, but does not want to be identified
as not normal. Therefore, the agent is searching for a strategy that appears as
normal as possible and maximizes its payoff at the same time.
We approach the problem in exactly the same way as we have approached the
first problem. We define most normal games Mnα (A) that depend on a parameter
α expressing how important normal behaviour for the agent is.

8.5.1. Definition. Let A be a m1 × m2 . . . × mn multi-matrix, let α > 0, and

let ~t be a strategy vector for the game Mx(A). The most normal game Mnα (A, ~t)
is a tuple (Σ, P{SX }, U) where Σ = {1, 2, . . . , n}, the strategy sets are SX = PmX
and UX (~s) = i sX X
s) − αE rel (sX , tX )
i Ai (~

The parameter α again determines the trade-off between selecting actions with a
high payoff and acting normal.
8.5.2. Theorem. Let Mnα (A, ~t) be a most normal game and ~s a strategy profile
for this game. The set bX (~s) is a singleton {b} such that
−1 X
tX 2α Ai (~s)
bi = P i X α−1 AX (~s)
k tk 2
k

Proof. Let Mnα (A, ~t) be a most normal game, ~s a strategy profile and X ∈ Σ
an agent. Suppose that b ∈ bX (~s) is the best response for agent X and let i be one
of B’s actions. If ti = 0 and bi 6= 0, then the relative entropy becomes infinite,
and the utility thus infinitely low. This cannot be optimal, thus if b maximizes
the utility, then ti = 0 implies bi = 0. Thus, in this case the optimal point is not
an interior point. It follows that if ti = 1, then for any optimal strategy b we
must have bi = 1.
Consider now the case where ti > 0. We calculate the derivative of the relative
entropy function.
δ rel δ X
E (b, tX ) = −bi (lg tX X
i − lg bi ) = lg bi + lg e − lg ti
δbi δbi i
8.5. Most Normal Games 165

We see that if bi > 0 approaches zero, then this derivative becomes negative
infinity. If bi is sufficiently small, then we would lower the utility UX ([~s−X , b]) by
decreasing bi further. Therefore, for any optimal value of b, it cannot be the case
that ti > 0 and bi = 0.
Since we have shown that ti = 0 implies bi P = 0, it remains for us to find the
m
optimal vector in the space S = {b ∈ [0, 1] | i bi = 1 ∧ (ti = 0 → bi = 0)}.
The previous argument has shown that b is an interior point of this set S. Such
points can only be optimal if δbδ i UX ([~s−X , b]) = δbδj UX ([~s−X , b]) for any pair i, j
with ti , tj > 0. The next computation will show that there is a unique point
satisfying this condition. Since any continuous function on a closed domain must
have a maximum, this point b will maximize agent X’s utility in the normal form
game.
First we calculate the derivative.

δ X
U ([~s−X , b]) =
δbi
δ
AX s) − α E rel (b, tX ) =
i (~
δbi
Ai (~s) − α(lg bi + lg e − lg tX
X
i ) =
AX s) − α lg bi − α lg e + α lg tX
i (~ i

δ δ
Now find the points b where the derivatives δbi
UX and δbj
UX are equal.

δ X δ X
U ([~s−X , b]) = U ([~s−X , b]) ⇔
δbi δbj
AX s) − α lg bi + α lg tX
i (~
X
i = Aj (~ s) − α lg bj + α lg tX
j ⇔
α lg(bi /bj ) − α lg(tX X
i /tj ) = AX
i (~
s) − AX
j (~
s) ⇔
α−1 AX
i (~
s)
bi tX
i 2
= α−1 AX
bj tX
j 2
j (~
s)

−1 AX (~ 1 X α−1 AX
Again we can choose c such that bi = ctX α i s) k (~
s)
P
i 2 and show that c
= k tk 2 .
This leads to the next formula.
−1 X
tX 2α Ai (~s)
bi = P i X α−1 AX (~s)
k tk 2
k

This formula gives us bi = 1 if ti = 1, and bi = 0 if ti = 0. Therefore, this formula

gives us the optimal strategy for any normal form game. 
166 Chapter 8. Entropy and Privacy

Discussion
One consequence of the theorem is the following observation. If a certain action
i is not considered by normal agents (tX i = 0) then the non-normal agent should
not consider action i either (bi = 0). If one had used a hard, logical approach
one could have reached the same conclusion. In the most extreme case one can
consider the case where normal agents use a pure strategy. In that case the non-
normal agent has to use the same pure strategy. If the non-normal agent values
all actions equally, he also does best by copying the normal strategy. In all other
cases the best strategy for the non-normal agent is different. Apparently the
agent does best by always taking some risk and getting a higher utility.

8.6 Equilibrium Refinements

By introducing minimal information games we have introduced a game with a
new kind of utility function. For small values of α the game Miα (A) is very
similar to the mixed strategy game Mx(A). One can, with some imagination, see
a Nash equilibrium x of Miα (A) as a solution of Mx(A). In that case, one has a
new solution concept for mixed strategy games Mx(A). Such a solution x of some
game Miα (A) is not a Nash equilibrium of Mx(A), but an approximation of it.
How good this approximation is depends on the parameter α. We can define a
Nash equilibrium by letting α approach zero. This way, we can define a ‘minimal
information’ equilibrium.

8.6.1. Definition. The strategy profile x is a minimal information equilib-

rium of Mx(A) iff there is a sequence α1 , α2 , . . . of positive numbers such that
limi→∞ αi = 0, a sequence x1 , x2 , . . . such that xi is a Nash equilibrium of Miαi (A)
and limi→∞ xi = x.

8.6.2. Theorem. Every mixed strategy game Mx(A) has a minimal information
equilibrium.

Proof. Define the sequence β1 , β2 , . . . by βi = 1/i. This sequence converges

to zero. By theorem 8.4.3 each game Miβi (A) has some Nash equilibrium yi .
The strategy space S1 × . . . × Sn is a closed and bounded subset of Rm for some
m. Therefore, since any closed and bounded subset of Rm is compact [121] we
derive that every sequence in S1 × . . . × Sn has some converging subsequence.
Let x1 , x2 , . . . be a converging subsequence of y1 , y2 , . . . and let x be the limit
of limi→∞ xi . Let α1 , α2 , . . . be the corresponding subsequence of β1 , β2 , . . ., so
that xi is a Nash equilibrium of Miαi (A). When α approaches infinity, the utility
function of Miαi (A) converges uniformly to the utility function of Mx(A). Since
xi is always maximizing each agents utility in Miαi (A), it must be the case that x
maximizes the utility of Mx(A) for each agent. Therefore, x is a Nash equilibrium
8.7. Telecom Network Example 167

of Mx(A). 

Every minimal information equilibrium is a special case of a proper equilib-

rium as defined by Myerson, and therefore it is also a trembling hand perfect
equilibrium [72]. These refinements can thus be motivated (if one wants to) by
an appeal to privacy minded agents. Perhaps there are other applications where
one needs a response concept that selects interior solution points, for instance to
avoid division by zero. In that case the minimal information best responses seem
suitable.

8.7 Telecom Network Example

Modern technology allows governments and other large institution to closely ob-
serve the movement of individuals. In the introduction we mentioned closed
circuit television systems, but it is also possible using mobile telephone networks
and in the near future RFID tags. In this section we therefore assume that an
observer can monitor the behaviour of agents in a small part of a city. Three
different scenarios have been implemented in a visual computer simulation, that
allows the user to take the role of the observer. The user can try to identify
what group agents belong to based on their behaviour. The agents have been
programmed to optimize their behaviour using the optimal strategies of theo-
rems 8.4.2 and 8.5.2. Explicit strategies that are based on these theorems are
for instance given in table 8.7 on page 169. Different agents value their privacy
differently and thus use a different value for α. It takes more time to spot agents
that use a higher value of α, so the use of optimal strategies for privacy protection
is effective in making life harder for an observer. However if the observer has no
time constraints, it can ultimately identify all agents.
The simulation is available as a Java applet on the world wide web, at the ad-
dress www.bluering.nl/sieuwert/programs/privacysim/simprivacy.html. One can see
several agents walking between their home and several shops. The simulator cur-
rently contains three levels. Each level is a new puzzle or challenge to the user.
The user can see all agents, and monitor which places they visit. The user also
knows what groups of agents exist, and what the preferences of each group are.
The goal is to guess the group of each agent.
All agents use optimal strategies for hiding their preferences. One might say
that the agents know that they are being watched, and act in order to make it
difficult to identify to which group they belong. In other words, the agents act as if
they are playing a minimal information game or a most normal game. The agents
are however not in competition with each other, but act independently. Below
we quickly describe the settings of the first two levels. For the third level a longer
description is given, in which the strategies used by the agents are described in
168 Chapter 8. Entropy and Privacy

Figure 8.2: Sim Privacy

details.

Level 1: Rich or Poor

The first level shows part of a city with two shops. One shop is a cheap shop,
the other one is an expensive shop. Two groups of agents live in this city, namely
poor agents and rich agents. The poor agents prefer to go to the cheap shop,
and the rich agents prefer the expensive shop. However, all agents do not want
anybody to know whether they are rich or poor. Therefore, all agents randomize
their shopping behaviour, and visit both shops with some probability. The goal
of this level is to determine for each agent whether it is a rich or a poor agent.
The main learning point from studying this level in the simulation is that these
puzzles can be solved. Since the agents adapt their strategy towards their payoffs
(we have shown that it is optimal for them to do so in theorem 8.4.2), one gains
some information from observing the agents behavior. If one is allowed to observe
the agents long enough, one will gain enough information to determine the type
of each agent with any level of probability. The simulation therefore shows that
in the long run it is impossible to protect ones privacy against observers who have
this much detailed information about ones daily behaviour. If one believes in the
universal human right to privacy, it is therefore necessary to prevent organisations
from collecting arbitrary large amounts of data, or to store such data for indefinite
amounts of time.

Level 2: Citizens and Criminals

In this level there are again two groups of agents. The citizens shop in any of
the four shops, and occasionally have to go to the bank to withdraw money. The
8.7. Telecom Network Example 169

citizens go to each shop with equal probability. The criminals have other sources
of income, and thus have no need to visit the bank. However the criminals do
not want others to know that they are criminal, so sometimes they do walk to
the bank to keep up appearances, but less often than they go to the shop. How
often they go to the shop depends on their level of paranoia: Normal agents go
less than paranoid agents.
This level demonstrates the influence of the parameter α on the behaviour of
agents. The four different types of agents, from normal to paranoid, have the
same preferences but value privacy differently. Anyone who has solved this level
has experienced that paranoid agents are harder to identify. One can thus protect
ones privacy better by acting more randomly.

Level 3: Crooks and Spooks

In the third level there are four shops and three groups of agents. The four
shops are the walmart, drugstore, spy shop and the bank. The three groups
are citizens, who are by all considered to be normal, the crooks, who are the
unorganised criminals, and the spooks, who are the organised criminals. The
utility values of each type of agent is given in the table below.

group walmart drugstore spy shop bank

citizens 0 1 1 2
spooks 1 1 2 1
crooks 1 1 2 1

The spooks and the crooks have the same preferences. The difference between
those two groups is that the spooks know what the citizens do, whereas the
crooks have no idea what normal is. Therefore, the crooks use a strategy that is
as random as possible, whereas the spooks use a strategy that is as similar as the
citizens as possible. The crooks can be said to be playing a most normal game,
and the spooks a minimal information game.
In the next table one sees the strategies that the agents use in this simulation.
The first column lists the type of an agent. The second lists the value of α that
that agent uses. For each agent type, there is a strategy for a not-so paranoid
agents (α = 1) and for more paranoid agents (α = 1.5). The remaining columns
list the probability that each agent visits a location.

type α walmart drugstore spy shop bank

citizens 1 0.072 0.196 0.196 0.534
citizens 1.5 0.115 0.224 0.224 0.436
spooks 1 0.054 0.146 0.399 0.399
spooks 1.5 0.06 0.165 0.322 0.45
crooks 1 0.174 0.174 0.475 0.174
crooks 1.5 0.202 0.202 0.393 0.202
170 Chapter 8. Entropy and Privacy

For computing the strategy of agent types citizens and crooks we have used
theorem 8.4.2 to compute the optimal strategies. For the spooks agents we have
used theorem 8.5.2, where the strategy of the non-paranoid citizens has been used
as the normal strategy. One can see in the table that for all three types of agents,
the more paranoid agents choose a strategy with a higher entropy. They act more
random. It is also clear that the spooks use a strategy that is more similar to
the citizens strategy, and hence they are harder to distinguish from the citizens.
For instance the crooks go often to walmart, but the other two types of agents do
not. By determining the frequency of walmart visits, an observer can determine
whether an agent is a crook or not.
In general, animated simulations such as this one can be used to demonstrate
certain phenomena in a more convincing and entertaining way than calculations
can. One can simulate much larger systems than one can solve by analytical
means, and thus simulations can be of more realistic size than examples can. On
the other hand, a proof-by-simulation lacks rigour. One can argue that simula-
tions do not lead to scientific knowledge in a way that proof does.
This simulation has been programmed in Java, a language very suitable for
interactive graphical programs. No specific agent systems library has been used.
The source code is available on request.

8.8 Conclusion
Two new kinds of games have been defined. First of all, minimal information
games, in which agents want to maximize the uncertainty that observers have
over their next move. Secondly, most normal games, in which agents want to
behave as similar as possible to an existing ‘normal’ agent, while maximizing
their payoff. The definitions use the concepts entropy and relative entropy which
are borrowed from information theory. In two theorems it is shown what the
optimal best responses are in these games. These turn out to be unique in each
situation, and to depend continuously on the payoff matrix and the opponent
strategies. From this continuity one can derive that Nash equilibria exist in these
games.
Minimal information games can be used to analyse situations with privacy-
minded agents. If agents attach some value to privacy, the best strategy always
gives them some privacy.
In most normal games, the situation is slightly more complicated. How well
the non-normal agent X can do depends very much on the strategy that normal
agents use. If the normal agents use a pure strategy, then X has no choice but to
adopt the same strategy. The situation however becomes a lot better if the normal
agents are privacy-minded. In this case they choose a high-entropy strategy, and
this leaves the wanting-to-be-normal agent a lot of room to pursue its own agenda.
One can extend the work in these games in several ways. It would be in-
8.8. Conclusion 171

teresting to look at experimental data, to see whether most-normal or minimal-

information strategies are used in the real world. Secondly, one could implement
these strategies in order to obtain privacy. The question is then whether the soft
approach to privacy is what users want.
On a theoretical side, it seems that these games give approximations to the
Nash equilibrium with useful technical properties. Two of these properties are
continuity of the best response function and the fact that best responses are
always interior.
Chapter 9
Conclusion

For many people, verification of software sounds like watching paint dry: Appar-
ently necessary, but quite dull compared to the creative process that came before
it, and the creative uses that come after it. The average user of the verified
software hardly learns anything from watching the process: either the program is
fine, or a bug is found and fixed, after which the program is also fine.
This dissertation is intented to convince the reader that verification of multi-
agent protocols is in fact very interesting. First of all because multi-agent pro-
tocols are widely used, sometimes at unexpected places. The debate about the
proposed constitution for the European Union which took place in May 2005, is
essentially a multi-agent protocol problem: what voting procedure should be used
so that every country and person is represented fairly? Often one can capture
requirements such as fairness in different ways, and deciding what is the best way
is not a mere technical matter.
The second reason why multi-agent protocols are so interesting is that rea-
soning about multi-agent systems is complicated and can have surprising results.
In software verification, the state-space explosion problem is often cited as the
biggest obstacle: the systems to be verified often have a huge number of different
states. Multi-agent protocols can have a small number of states, especially when
these protocols have to be explained to and used by humans. On the other hand
the requirements for these protocols can be subtle and difficult to interpret: in
many cases, properties such as fairness can be hard to define and verify. Different
logics based on extensive games have been presented in the previous chapters.
Using three examples, a voting problem, the joint decision problem and the inde-
pendent decision problem, we have shown that more complex logics can be used
to identify subtle differences in protocols. These more expressive logics can have
less favourable computational properties, making verification intractable. Thus,
besides social arguments, there are also technical arguments in favour or against
certain approaches.
An important distinction, that has been borrowed from game theory, is the

173
174 Chapter 9. Conclusion

one between perfect information protocols and imperfect information protocols.

In the first class of protocols all actions and all facts about the current state
are public. Knowledge about these aspects thus does not play a role in these
protocols. In imperfect information protocols knowledge is vitally important.
Chapters 4 to 5 are focused on perfect information protocols, the final chapters
7 and 8 on imperfect information protocols.

9.1 Perfect Information Protocols

Modal logic is a very useful tool for studying perfect information protocols. It
is easy to define logics that deal specifically with these protocols. The first logic
presented, efl, can be used to reason about which coalitions can enforce what
kind of outcome. It can also be used in practice for verification of existing proto-
cols. Unfortunately it is not very expressive: Many protocols that feel different
satisfy the same efl properties.
One can extend the language efl in order to make more interesting properties.
This leads to two meaningful extensions: efls and efln. The first language
can express more complex reasoning involving side effects of adopting certain
strategies: “Suppose I know that you want this, can I then do that?”. The
second logic, efln, can be used for expressing nested properties such as “I want
to allow you to allow me to do this”. For these three logics we have determined
the computational complexity of model checking. A fourth language eflns that
combines features from efln and efls has also been defined. This language is
however is hard to interpret in a conservative way.
In chapter 6 a more explicit logic is used for reasoning about preferences. The
language has been extended with operators reasoning about game trees, which
makes it possible to use this logic for analysing game-theoretic reasoning in detail.
As an example the concept of backward induction has been analysed in this logic.
These different logics illustrate that in order to understand a multi-agent
protocol, one has to understand the background assumptions: what do agents
know about each other and the situation. One always has a choice how to analyse
a protocol. Even protocols with perfect information, that are often seen as the
easiest case, can be difficult to compare.

9.2 Imperfect Information Protocols

It is well-known that knowledge and information are very important for agents,
and it is also common to use game theory for analysing the interaction between
agents with different interests. It is therefore a ‘logical’ next step to consider
games about information . A knowledge condition game is a game between two
groups of agents: one group wants to reach a certain knowledge situation, the
9.2. Imperfect Information Protocols 175

other group wants to stop the first group from reaching this situation. This situ-
ation that the groups want to reach or avoid is specified using ordinary epistemic
logic. The fact that a well known logic is used makes knowledge condition games
easier to understand, compared to logical languages with new operators. The fact
that knowledge condition games only model the knowledge of the agents in the
final situation is also an advantage: no temporal reasoning is necessary. Research
in temporal logic has shown that reasoning about time is complex in itself, so it
is not wise to make things even more difficult by mixing the aspects of time and
strategies.
The complexity results for knowledge condition games indicate that games
about knowledge can be intractable. They become tractable when monotone for-
mulas are used. The complexity is thus caused by the fact that in epistemic logic,
one can mix knowledge demands (Somebody knows something) and ignorance
demands (Somebody does not know something).
It often makes sense to assume that agents are aware of the strategies that are
used, for instance of strategies that are so often used that they become conven-
tions, or when dealing with security protocols. One can also assume that agents
do not know strategies. This has been defined as kcg 0 . This alternative definition
makes decision problems slightly easier, and is thus a convenient assumption.
In a knowledge condition game where ignorance is demanded, the optimal
strategy is often a random one. The coalition of agents that wants somebody to
be ignorant should choose their actions in a random, unpredictable way. The fact
that making random choices can be optimal has been known to game theorists
before [11], but sometimes surprises people: flipping a coin is not often recom-
mended for important decisions. The chapter on knowledge condition games does
not tell what kind of coin one should use. It does not tell what exact probabilities
one should use to choose between actions, because the logical approach does not
work with explicit probabilities.
In order to be able to say something about those probabilities, chapter 8
introduces minimal information games. In these games agents have two goals:
getting an optimal payoff by choosing the best actions, and randomizing their
behaviour in such a way that an observer is kept ignorant about what the agent
might do in the future. In order to measure ‘ignorance’, information theory is
used. I have computed optimal strategies for these games, and these strategies
give detailed information how one should randomize. The same is done for the
related notion of most normal games. In those games, agents want to behave as
similar to ‘normal’ as possible, but also getting the highest payoff. Thus, in this
chapter the question about what coin one should use is solved.
Comparing those two approaches, one based on logic and one based on infor-
mation theory, one can make two observations. On the one hand one can say that
the logical approach is more general. Using epistemic logic one can express goals
that mix knowledge and ignorance. The games based on information theory only
deal with ignorance. In general one needs a logical approach in order to form
176 Chapter 9. Conclusion

complex goals. On the other hand, a logical approach has the disadvantage that
it is more abstract: important details, such as the exact probabilities, are often
omitted.

9.3 Results
The next table shows the complexity results stated in this dissertation. The
problems in the class PSPACE are definitely not tractable: no efficient algorithms
for these problems exist. The same is probably true, in practical terms, of the
problems in the class Σ2 P: Even though Σ2 P problems are theoretically easier to
solve, all problems in both classes are too hard to be solved in practice. The class
NP contains problems that are also believed to be hard. No efficient algorithms
for these problems are known, but sometimes one can find heuristics for those
problems. The problems in the final class, P, are called tractable. They can be
solved in reasonable time.
number class members
1 PSPACE efl model checking with linear representation
2 efln model checking
3 Σ2 P kcg decision problem with opponents
4 NP kcg without opponents
5 kcg 0
6 P eflmodel checking
7 efls model checking
8 kcg for monotone formulas

It is clear that analysing games is a complex affair: many of these problems are
intractable. The intractability has different causes. Sometimes, in cases 3 and
4 for instance, the presence of opponents can make a problem much harder. In
other cases, namely 1 and 2, the situation with one agent is already complex.
This is a bit surprising.
In the chapter on logic it has been explained that theorem proving and model
checking are both important techniques for multi-agent protocol verification. In
this dissertation the following complete proof systems are presented.

description logic proof system

effectivity logic efl SEF L
preference logic LP SP
alternative preference logic L2P SP2
finite tree logic LT ST

The first result, that there is a complete proof system for efl, supports the hope
that logical mechanism design is possible. The completeness proof sketches an
9.3. Results 177

algorithm for constructing protocols. It would be interesting to implement and

test this procedure in the future.
Preference logic LP is a more natural language to express preferences than
propositional logic. One can use preference logic to say things such as “coffee is
better than tea”, instead of the less informative “coffee is good” or “tea is bad”.
A proof system for this logic exists. This logic has been used for constructing a
logic Lsol , that can be used for characterising game-theoretic solution concepts.
Interesting future work would be to use this preference logic in an update frame-
work.
 Bibliography

[1] A. Acquisti. The economics of privacy.

http://www.heinz.cmu.edu/ acquisti/economics-privacy.htm, 2004. Inter-
net page.

[2] D. Adams. Mostly Harmless. Harmony Books, NY, 1992.

[3] T. Agotness. A note on syntactic characterization of incomplete infor-

mation in ATEL. In S. van Otterloo, P. McBurney, W. van der Hoek,
and M. Wooldridge, editors, Proceedings of the first Knowledge and Games
Workshop, pages 34–42. University of Liverpool, 2004.

[4] M. Aigner and G. Ziegler. Proofs from the Book. Springer-Verlag: Berlin,
Germany, 1999.

[5] R. Alur, L. de Alfaro, T. A. Henzinger, S. C. Krishnan, F. Y. C. Mang,

S. Qadeer, S. K. Rajamani, and S. Taşiran. mocha user manual. University
of Berkeley Report, 2000.

[6] R. Alur, T. A. Henzinger, and O. Kupferman. Alternating-time temporal

logic. Journal of the ACM, 49(5):672–713, September 2002.

[7] R. Aumann and A Brandenburger. Epistemic conditions for a Nash equi-

librium. Econometrica, 63:1161–1180, 1995.

[8] A. Baltag. A logic for suspicious players: Epistemic actions and belief-
updates in games. Bulletin of Economic Research, 54:1–45, 2002.

[9] A. Baltag, L.S. Moss, and S. Solecki. The logic of public announcements,
common knowledge and private suspicions. Originally presented at TARK
98, accepted for publication in Annals of Pure and Applied Logic, 2002.

179
180 Bibliography

[10] M. Benerecetti and A. Cimatti. Symbolic model checking for multi agent
systems. In Proceedings of the ICLP’01 workshop on Computational Logic
in Multi-Agent Systems, 2001.
[11] K. Binmore. Fun and Games: A Text on Game Theory. D. C. Heath and
Company: Lexington, MA, 1992.
[12] P. Blackburn, M. de Rijke, and Y. Venema. Modal Logic. Cambridge
University Press: Cambridge, England, 2001.
[13] P. Blackburn and W. Meyer-Viol. Linguistics, logic, and finite trees. Logic
Journal of the IGPL, pages 3–29, 1994.
[14] G. Bonanno. Branching time, perfect information games and backward
induction. Games and Economic Behavior, 36:57–73, 2001.
[15] G. Bonanno. Memory implies von neumann-morgenstern games. Knowledge
Rationality and Action, to appear, 2004.
[16] G. Boolos. The Logic of Provability. Cambridge University Press, 1993.
[17] S. Brahms and D. Taylor. Fair division: from cake cutting to dispute reso-
lution. Cambridge University Press, 1996.
[18] F. Brandt and T. Sandholm. (Im)possibility of unconditionally privacy-
preserving auctions. In Proceedings of the International Joint Conference
on Autonomous Agents and Multi-Agent Systems (AAMAS), pages 810–
817, New York, 2004.
[19] D. Chaum. The dining cryptographers problem: Unconditional sender and
recipient untraceability. Journal of Cryptology, 1:65–75, 1988.
[20] R. Chisholm and E. Rosa. On the logic of intrinsically better. American
Philosophical Quarterly, 3:244–249, 1966.
[21] M. Chwe. Rational Ritual: Culture, Coordination and Common Knowledge.
Princeton University Press: Princeton, NJ, 2001.
[22] E. M. Clarke, O. Grumberg, and D. A. Peled. Model Checking. The MIT
Press: Cambridge, MA, 2000.
[23] E.M. Clarke and E.A. Emerson. Design and synthesis of synchronisation
skeletons using branching time temporal logic. Lecture Notes in Computer
Science, 131:52–71, 1981.
[24] V. Conitzer and T. Sandholm. Complexity of manipulating elections with
few candidates. In Proceedings of the 18th National Conference on Artificial
Intelligence (AAAI-02), pages 262–273, 2002.
Bibliography 181

[25] V. Conitzer and T. Sandholm. Complexity of mechanism design. In Pro-

ceedings of the Uncertainty in Artificial Intelligence Conference (UAI), Ed-
monton, Canada., 2002.

[26] S. Cook. The P versus NP problem. Manuscript prepared for the Clay
Mathematics Institute for the Millennium Prize Problems, 2000.

[27] T. Cormen, C. Leiserson, and R. Rivest. Introduction to Algorithms. The

MIT Press: Cambridge, MA, 1990.

[28] T. Cover and J. Thomas. Elements of Information Theory. John Wiley &
Sons, New York, 1991.

[29] R. Dash, N. Jennings, and D. Parkes. Computational-mechanism design:

A call to arms. IEEE Intelligent Systems, pages 40–47, November 2003.

[30] B. de Bruin. Explaining games: On the Logic of Game Theoretic Expla-

nations. PhD thesis, University of Amsterdam, 2004. ILLC Dissertation
Series 2004-03.

[31] F. Dechesne. Game, set, maths: formal investigations into logic with im-
perfect information. PhD thesis, Tilburg University and Technische Uni-
versiteit Eindhoven, 2005.

[32] R. Fagin, J. Halpern, Y. Moses, and M. Vardi. Reasoning about knowledge.

The MIT Press: Cambridge, MA, 1995.

[33] M. Franzin, F. Rossi, E. Freuder, and R. Wallace. Multi-agent constraint

scheduling with preferences: efficiency, solution quality and privacy loss.
Computational Intelligence, 20:264–286, 2004.

[34] L.T.F. Gamut. Logic, Language and Meaning: Volume 2. University of

Chicago Press, Chicago, 1991. L.T.F. Gamut is a collective pseudonym for
Van Benthem, Groenendijk, De Jongh and Verkuyl; translated from Dutch.

[35] M.R. Garey and D.S. Johnson. Computers and Intractability: A Guide
to the Theory of NP-completeness. W. H. Freeman and Company, San
Francisco, 1979.

[36] A. Gibbard. Manipulation of voting schemes: a general result. Economet-

rica, 41(4), 1973.

[37] H. Gintis. Game theory evolving. Princeton University Press: Princeton,

NJ, 2000.

[38] J. Glasner. eBay bidders sold on sniping, September 2002.

http://www.wired.com/news/business/0,1367,55204,00.html.
182 Bibliography

[39] J. Goeree, S. Onderstal, E. Maasland, and J. Turner. How (not) to raise

money. Journal of Political Economy, 2005. forthcoming.

[40] V. Goranko. Coalition games and alternating temporal logics. In Pro-

ceedings of the 8th conference on Theoretical Aspects of Rationality and
Knowledge(TARK), pages 259–272, 2001.

[41] V. Goranko and G. van Drimmelen. Complete axiomatization and decid-

ability of the alternating-time temporal logic. 2004.

[42] J. Halpern. Defining relative likelyhood in partially-ordered preferential

structures. Journal of Artificial Intelligence Research, pages 1–24, 1997.

[43] J. Halpern. Reasoning about uncertainty. The MIT Press: Cambridge, MA,
2003.

[44] S. Hansson. Preference logic. In D. Gabbay and F. Guenthner, editors,

Handbook of Philosophical Logic (Second Edition), volume 4, chapter 4,
pages 319–393. Kluwer, 2001.

[45] B. Harrenstein, W. van der Hoek, J.-J. Ch. Meyer, and C. Witteveen. A
modal characterization of Nash equilibrium. Fundamenta Informaticae,
57(2–4):281–321, 2003.

[46] J. Harsanyi. A general theory of rational behavior in game situations.

Econometrica, 34:613–634, 1966.

[47] J. Harsanyi. Games with incomplete information played by bayesian players.

management science, 14:159–182,320–334,486–502, 1967-1968.

[48] J. Hintikka. Knowledge and Belief: an introduction to the logic of the two
notions. Cornell University Press: Ithaca, NY, 1962.

[49] J. Hintikka. Principles of mathematics revisited. Cambridge University

Press: Cambridge, England, 1996.

[50] W. Hodges. Formal aspects of compositionality. Journal of Logic, Language

and Information, pages 7–28, 2001.

[51] G. Holzmann. The model checker Spin. IEEE Trans. on Software Engi-
neering, 23:279–295, May 1997.

[52] Z. Huang. Logics for Agents with Bounded Rationality. PhD thesis, Uni-
versity of Amsterdam, 1994. ILLC Dissertation Series 1994-10.

[53] M. Huth and M. Ryan. Logic in Computer Science: Modelling and reasoning
about systems. Cambridge University Press: Cambridge, England, 2000.
Bibliography 183

[54] W. Jamroga and W. van der Hoek. Some remarks on alternating-time

temporal epistemic logic. submitted, 2003.
[55] G. Jonker. Feasible strategies in alternating-time temporal epistemic logic,
2003. Universiteit Utrecht Master Thesis.
[56] M. Kacprzak, A. Lomuscio, and W. Penczek. Verification of multiagent
systems via unbounded model checking. In Proceedings of the International
Joint Conference on Autonomous Agents and Multi-Agent Systems (AA-
MAS), New York, July 2004.
[57] D. Koller and N. Megiddo. The complexity of two-person zero-sum games
in extensive form. Games and Economic Behavior 4, 4:528–552, October
1992.
[58] D. Koller and A.J. Pfeffer. Generating and solving imperfect information
games. In Proceedings of the 14th International Joint Conference on Arti-
ficial Intelligence (IJCAI), pages 1185–1192, Montreal, August 1995.
[59] B. Kooi. Knowledge, Chance, and Change. PhD thesis, University of
Groningen, Groningen, 2003. ILLC Dissertation Series 2003-01.
[60] B. Kooi. Probabilistic dynamic epistemic logic. Journal of Logic, Language
and Information, 12:381–408, 2003.
[61] B. Kooi and J. van Benthem. Reduction axioms for epistemic actions. In
Proceedings of Advances in Modal Logic (AiML 2004), University of Manch-
ester, September 2004.
[62] D. Kreps and R. Wilson. Sequential equilibria. Econometrica, 50:863–894,
1989.
[63] V. Krishna. Auction theory. Academic Press, San Diego, 2002.
[64] H. Kuhn. Extensive games and the problem of information. Contributions
to the Theory of Games, II:193–216, 1953.
[65] D. Lewis. Convention — A Philosophical Study. Harvard University Press:
Cambridge, MA, 1969.
[66] D. MacKenzie. Mechanizing proof:computing, risk, and trust. The MIT
Press: Cambridge, MA, 2004.
[67] R. Maheswaran, J. Pearce, P. Varakantham, E. Bowring, and M. Tambe.
Valuations of possible states (vps): A quantitative framework for analysis of
privacy loss among collaborative personal assistant agents. In Proceedings of
the International Joint Conference on Autonomous Agents and Multi-Agent
Systems (AAMAS), Utrecht, July 2005.
184 Bibliography

[68] J. Maynard Smith. Evolution and the theory of games. Cambridge Univer-
sity Press: Cambridge, England, 1982.

[69] K. McMillan. Symbolic Model Checking. PhD thesis, Carnegie Mellon Uni-
versity, 1992.

[70] J.-J. C. Meyer and R. J. Wieringa. Deontic logic: A concise overview.

In Deontic Logic in Computer Science, volume 15 of Wiley Professional
Computing Series. John Wiley & Sons, 1993.

[71] J.-J. Ch. Meyer and W. van der Hoek. Epistemic Logic for AI and Computer
Science. Cambridge University Press: Cambridge, England, 1995.

[72] R. Myerson. Refinements of the Nash equilibrium concept. International

Journal of Game Theory, 7:73–80, 1978.

[73] J. Nash. Non-cooperative games. Annals of mathematics, 54:286–295, 1951.

[74] J. Von Neumann and O. Morgenstern. Theory of Games and Economic

Behaviour. Princeton University Press: Princeton, NJ, 1944.

[75] J. Von Neumann and O. Morgenstern. Theory of Games and Economic

Behaviour. Princeton University Press: Princeton, NJ, 3rd edition, 1953.

[76] A. Ockenfels and A. Roth. The timing of bids in internet auctions: Market
design, bidder behavior, and artificial agents. Artificial Intelligence Maga-
zine, 2005.

[77] A. Odlyzko. Privacy, economics, and price discrimination on the Internet.

In Fifth International Conference on Electronic Commerce(ICEC), pages
73–80, Pittsburgh, 2003.

[78] O. Ore. Cardano, the gambling scholar. Princeton University Press: Prince-
ton, NJ, 1953.

[79] M. J. Osborne and A. Rubinstein. A Course in Game Theory. The MIT

Press: Cambridge, MA, 1994.

[80] Oxford english dictionary (online edition), 2005.

http://dictionary.oed.com/.

[81] C. Papadimitriou. Computational Complexity. Addison Wesley Longman,

Reading, 1994.

[82] R. Parikh. Social software. Synthese, 132:187–211, 2002.

Bibliography 185

[83] D. Parkes. Classic mechanism design. In Iterative Combinatorial Auctions:

Achieving Economic and Computational Efficiency. Ph.D. dissertation, Uni-
versity of Pennsylvania, 2001.
[84] M. Pauly. Logic for Social Software. PhD thesis, University of Amsterdam,
2001. ILLC Dissertation Series 2001-10.
[85] M. Pauly. A modal logic for coalitional power in games. Journal of Logic
and Computation, 12:149–166, 2002.
[86] N. Rescher. Topics in philosophical logic. D. Reidel publishing company,
Dordrecht, 1968.
[87] M. Roberts, W. van der Hoek, and M. Wooldridge. Knowledge and social
laws. In Proceedings of the International Joint Conference on Autonomous
Agents and Multi-Agent Systems (AAMAS), Utrecht, July 2005.
[88] T. Schelling. The strategy of Conflict. Harvard University Press, 1960.
[89] B. Schneier. Applied Cryptography. John Wiley & Sons, 1996.
[90] P. Schnoebelen. The complexity of temporal logic model checking. In
Advances in Modal Logic (AiML’2002), Sep.-Oct. 2002, Toulouse, France.,
2002.
[91] P.-Y. Schobbens. Alternating-time logic with imperfect recall. In W. van der
Hoek, A. Lomuscio, E. de Vink, and M. Wooldrige, editors, Electronic Notes
in Theoretical Computer Science, volume 85. Elsevier, 2004.
[92] H. Seely. The poetry of D.H. Rumsfeld, 2003. web column, posted April 2,
2003, http://slate.msn.com/id/2081042/.
[93] R. Selten. Reexamination of the perfectness concept for equilibrium points
in extensive games. International journal of game theory, 4:25–55, 1975.
[94] M. Sevenster. A player semantic on IF semantic games. In Logic, Games and
Philosophy: foundational perspectives, Logic, Epistemology and the unity
of sciences, Prague, oct 2004.
[95] C. E. Shannon. A mathematical theory of communication. The Bell
System Technical Journal, 27:379–423, 623–656, July, October 1948.
http://cm.bell-labs.com/cm/ms/what/shannonday/shannon1948.pdf.
[96] R. Smullyan. First Order Logic. Springer-Verlag: Berlin, Germany, Berlin,
1968.
[97] J. van Benthem. Logic and games. continuing electronic lecture notes,
1999–2004.
186 Bibliography

[98] J. van Benthem. Games in dynamic-epistemic logic. Bulletin of Economic

Research, 53(4):219–248, 2001.

[99] J. van Benthem. Extensive games as process models. Journal of Logic,

Language and Information, 11:289–313, 2002.

[100] D. van Dalen. Variants of rescher’s semantics for preference logic and some
completeness theorems. Studia Logica, 33:163–181, 1974.

[101] W. van der Hoek and M. Wooldridge. Model checking knowledge and time.
In D. Bos̆nac̆ki and S. Leue, editors, Model Checking Software, Proceedings
of SPIN 2002 (LNCS Volume 2318), pages 95–111. Springer-Verlag: Berlin,
Germany, 2002.

[102] W. van der Hoek and M. Wooldridge. Tractable multiagent planning for
epistemic goals. In Proceedings of the First International Joint Conference
on Autonomous Agents and Multiagent Systems (AAMAS-2002), pages
1167–1174, Bologna, Italy, 2002.

[103] W. van der Hoek and M. Wooldridge. Cooperation, knowledge, and time:
Alternating-time temporal epistemic logic and its applications. Studia Log-
ica, 75(4):125–157, 2003.

[104] R. van der Meyden and K. Su. Symbolic model checking the knowledge of
the dining cryptographers. under review, 2004.

[105] H. P. van Ditmarsch. Knowledge Games. PhD thesis, University of Gronin-

gen, Groningen, 2000.

[106] H. P. van Ditmarsch. The russian cards problem. Studia Logica, 75(4):31–
62, 2003.

[107] S. van Otterloo. Reasoning about extensive games. ESSLLI student session,
2005.

[108] S. van Otterloo. The value of privacy: optimal strategies for privacy minded
agents. In Proceedings of the International Joint Conference on Autonomous
Agents and Multi-Agent Systems (AAMAS), Utrecht, July 2005.

[109] S. van Otterloo and G. Jonker. On epistemic temporal strategic logic. In

Proceedings of the second workshop on logic and communication in Multi
Agent Systems (LCMAS), Nancy, 2004.

[110] S. van Otterloo, W. van der Hoek, and M. Wooldridge. Knowledge condition
games. In S. Parsons and P. Gmytrasiewicz, editors, Sixth Workshop on
Game Theoretic and Decision Theoretic Agents, New York, 2004.
Bibliography 187

[111] S. van Otterloo, W. van der Hoek, and M. Wooldridge. Knowledge con-
dition games. In S. van Otterloo, P. McBurney, W. van der Hoek, and
M. Wooldridge, editors, Proceedings of the first Knowledge and Games
Workshop. University of Liverpool, 2004.

[112] S. van Otterloo, W. van der Hoek, and M. Wooldridge. Model checking
a knowledge exchange scenario. Applied Artificial Intelligence, 18:937–952,
2004.

[113] S. van Otterloo, W. van der Hoek, and M. Wooldridge. Preferences in game
logics. In Proceedings of the International Joint Conference on Autonomous
Agents and Multi-Agent Systems (AAMAS), New York, July 2004.

[114] S. van Otterloo, W. van der Hoek, and M. Wooldridge. Knowledge condition
games. Journal of Logic, Language and Information (to appear), 2006.

[115] S. van Otterloo and M. Wooldridge. On the complexity of knowledge con-

dition games. In Proceedings of the second European workshop on Multi
Agent Systems(EUMAS), Barcelona, 2004.

[116] R. van Rooij. Utility, informativity and protocols. Journal of Philosophical

Logic, pages 389–419, 2004.

[117] G. Vergouw. De Strafschop. Reed business information, 2003.

[118] J. von Neumann. Zur theorie der gesellschaftspiele. Mathematische An-

nalen, pages 295–320, 1928.

[119] G. von Wright. The logic of preference. Edinburgh University Press, Edin-
burgh, 1963.

[120] J. Weibull. Evolutionary Game Theory. The MIT Press: Cambridge, MA,
1995.

[121] E. Weisstein. Compact set. http://mathworld.wolfram.com/CompactSet.html,

2004. from Mathworld - A Wolfram web resource.

[122] E. Weisstein. Trivium. http://mathworld.wolfram.com/Trivium.html,

2005. from Mathworld - A Wolfram web resource.

[123] T. Williamson. Knowledge and its limits. Oxford University Press, 2000.

[124] M. Wooldridge and N. Jennings. Intelligent agents: Theory and practice.

Knowledge Engineering Review, 2, 1995.
 Index

3SAT, 30 conjunctive normal form, 14

consistent, 10
agent, 2 constant-sum games, 41
algorithm, 27 cooperative, 35
Alternating-time Temporal Logic, 52
anti-symmetric, 102 decision problem, 27
ATL, 52 disjunctive normal form, 14
automated mechanism design, 24 Distribution, 17
dynamic epistemic, 54
backward induction, 36, 100 dynamic epistemic logic, 48
behavioural strategy, 43
bi-matrix, 38, 39 effectivity, 48
bias, 77 effectivity logic, 57
bisimilar, 18, 104 efficiently, 28
bisimulation, 18 end situation model m(F ), 129
bit, 27 entropy, 157
bits, 27, 157 envy-freeness, 1
epistemic model, 19
canonical model, 18 equivalence relation, 19
coalition logic, 48, 49 equivalent, 104
coalition model, 50 extensive game, 42, 43
coalition preferences, 79
coalition strategy, 43 fairness, 1
common knowledge, 22 functional, 121
complete, 10, 30
complete information, 35, 81, 99, 100 game form, 42
complexity class, 28 general games, 35
computation time, 27 Hamiltonian cycle, 29
Computation tree logic, 25
computational complexity, 27 imperfect information, 35
computational mechanism design, 24 incomplete information, 36

189
190 Index

independence friendly logic, 49 polynomial time, 28

independent decision problem, 5, 57 positive formulas, 147
instance, 12 power bisimulation, 52
interpretation function, 16 preference model, 102
interpreted game form, 42, 128 preference tree model, 114
intractable, 29, 32 problem, 27
proof system, 9
joint decision problem, 5, 57 proto-model, 114
linear temporal logic, 24 PSPACE, 28
LTL, 24 pure strategy, 43
pure strategy game, 39
maximal, 50
maximally consistent, 17 QBF, 31
maximally consistent subset, 117 quantified boolean formula, 31
mechanism design, 24
reduction, 30
minimal information game, 160
reflexive, 102
minimal preference model, 102
reflexive frame, 102
mixed strategy game, 40
Mocha, 53 satisfiability problem, 30
Modus Ponens, 12 satisfiable, 9
most normal game, 164 sequence set, 42
multi-matrix, 39 side effects, 80
skew, 77
Nash equilibrium, 36, 41
sniping, 2
subgame perfect, 36
solution concept, 36
Necessitation, 16
solved game logic, 120
negative formulas, 147
sound, 10
nested abilities, 79
SPIN, 25
nondeterministic polynomial time, 29
state formulas, 26
nondeterministic strategy, 43
state-space explosion, 173
NP, 29
strategic game, 37
outcome bisimulation, 62 strategy vector, 38
strict-transitive, 102
P, 28 subgame form, 47
path, 25 subgame perfect equilibrium, 45
path formulas, 26 superadditive, 51
payoff, 38 symbolic model checking, 23
perfect information, 35
perfect memory, 35, 47 tautologies, 9
perfect recall, 35, 47 total, 102
politeness, 79 tractable, 28
polynomial shrinking, 87 tree model, 114
polynomial space, 28 Turing machines, 27
Index 191

uniform, 46, 150

uniform strategies, 130
uniform substitution, 12
utility, 38

valid formulas, 9
validities, 9

weakly playable, 51
win-loss games, 41

Zermelo’s algorithm, 36
zero-sum game, 41
 List of Symbols

2 modal operator, page 15

3 dual of 2, page 15
⊥ the ‘false’ constant, page 11
kdk size (number of bits), page 27
|= validity, page 9
fφ in the next state φ, page 25
` provability, page 10
∼X equivalence relation, page 45
s−j s with jth element removed, page 38
[s, x] s with x inserted, page 38
∇ exclusive or, page 11
a single action, page 41
AB set of functions B → A, page 37
A axioms, page 9
A(H, h) actions following h, page 41
allsub(F ) set of all subgames, page 44
argmax set of maximizing arguments, page 39
At(Φ) atom set, page 113
Aφ for all (preference logic), page 97
AXi (~s) expected payoff, page 39
m
B closed sphere, page 152
b(~s) best response to ~s, page 40
Cφ φ is common knowledge, page 22
cl(Φ) closure, page 113
dnf disjunctive normal form, page 102
Eφ Everybody knows φ, page 21
Eφ exists (preference logic), page 97
E entropy, page 147
 the empty sequence, page 41

193
194 Index

E rel (x, y) relative entropy, page 149

F (interpreted) game form, page 41
f (x, y) entropy helper function, page 147
f∅ function with empty domain, page 128
F
f (R) game form denoted by R, page 69
Γ coalition of agents, page 42
G strategic game, page 37
H set of sequences, page 41
h sequence of actions, page 41
KX φ X knows φ, page 19
kcg knowledge condition game, page 128
L logical language, page 9
l(φ) nesting level of φ, page 102
lg base 2 logarithm, page 147
L2 standard modal logic, page 15
Lp propositional logic, page 11
f
Lp full propositional logic, page 13
LP preference logic, page 97
L2P alternative preference logic, page 108
M set of models, page 83
M logical model, page 9
m(F ) end situation model, page 126
MX φ X thinks φ is possible, page 19
Mi minimal information game, page 150
M PL Modus Ponens, page 12
N natural numbers, page 24
N1 set of information objects, page 83
N2 set of basic formulas, page 83
N ecL necessitation, page 16
O(x) running time, page 27
Pm balanced vector space, page 39
P set of atomic propositions, page 11
π interpretation function, page 16
φhPref iψ preference logic operator, page 97
Qm nonzero balanced vector space, page 148
R reasoning rules, page 9
R real numbers, page 37
R set of runs, page 25
R accessibility relation, page 16
r(H, h) reduced model, page 79
reach(R, w) reachable points from w using R, page 110
ρXG ability of X, page 50
∆ set of modalities, page 15
Index 195

S proof system, page 9

Σ set of all agents, page 19
SbX (F ) set of behavioural strategies, page 43
σΓe (φ) φ-effective strategy, page 80
SnX (F ) set of nondeterministic strategies, page 43
SpX (F ) set of pure strategies, page 43
SP preference logic proof system, page 101
SP2 alternative pref. logic proof system, page 109
Sp propositional logic proof system, page 12
Rs strict version of relation R, page 98
ST finite tree logic proof system, page 111
subg(F, h) subgame of F starting at h, page 44
a[x \ 1] substitute 1 for x in a, page 68
SX set of strategies, page 37
> truth constant, page 13
turn turn function, page 41
φ Uψ φ until ψ, page 25
U utility function, page 37
u imperfect inf. update function, page 126
Up nondet. strategy update function, page 80
Up pure strategy update function, page 57
v value function, page 58
V set of variables, page 68
~s vector of strategies, page 38
ViX set of strategy profiles in which X plays i, page 39
W set of walks, page 25
W set of worlds, page 16
w possible world, page 15
w(G) winner function, page 127
X single agent, page 16
Z(H) terminal histories, page 41
 Samenvatting

Multi-agent protocollen zijn collecties van regels die aangeven hoe meerdere par-
tijen met elkaar in contact kunnen treden. Een veiling bijvoorbeeld heeft strikte
regels die aangeven hoe er geboden kan worden. Ook de mogelijke zetten van
een schaakpartij zijn vastgelegd in een collectie regels, en vormen dus een multi-
agent protocol. Tenslotte zijn ook verkiezingen een voorbeeld van een multi-agent
protocol. Deze activiteiten hebben gemeen dat ze in het echte leven, zonder on-
dersteuning van computers gedaan kunnen worden. Men kan zich echter ook
voorstellen dat computerprogramma’s meedoen aan veilingen en verkiezingen,
misschien zelfs met of tegen menselijke spelers. Aangezien computerprogramma’s
nog niet zo intelligent zijn als wetenschappers soms wensen, is het vaak van be-
lang dat protocollen aan bepaalde veiligsheidseisen voldoen. Men wil dus kunnen
nagaan aan welke eigenschappen een protocol voldoet.
Het doel van mijn onderzoek is om methodes te ontwikkelen waarmee men
multi-agent protocollen kan vergelijken en analyseren. Om dit te kunnen doen
moet men een onderscheid maken tussen verschillende klassen protocollen, en ook
verschillende soorten eigenschappen onderscheiden. Protocollen waarin iedere
‘speler’ geheel op de hoogte is van de huidige toestand (schaak bijvoorbeeld) wor-
den behandeld in het eerste deel van dit proefschrift. Voor deze protocollen geldt
dat de eigenschappen die te beschrijven zijn in de logische taal van hoofdstuk 4,
efficient door een computer te verifiëren zijn. Ook kan men formeel redeneren over
deze eigenschappen. Echter, ook voor deze relatief eenvoudige protocollen zijn er
eigenschappen, die uitgedrukt kunnen worden in logische talen uit hoofdstuk 5,
waarvoor automatische verificatie erg complex is. Over sommige ingewikkeldere
eigenschappen kan men echter wel formeel redeneren met het bewijssysteem uit
hoofdstuk 6.
Er zijn ook veel protocollen waarin niet alle spelers van alle details van de
situatie op de hoogte zijn. Denk bijvoorbeeld aan spelen zoals Stratego of Poker.
In deze protocollen is informatie over de huidige situatie, en kennis over wat
andere spelers weten een belangrijke factor. In hoofdstuk 7 worden situaties

197
198 Samenvatting

behandeld waarin het doel van bepaalde spelers is om bepaalde kennis juist wel
of juist niet te hebben. De complexiteit van het analyseren van dit soort situaties
kan hoog zijn, afhankelijk van welke aannames men maakt.
In al deze eerste hoofdstukken wordt kennis als een kwalitatieve eigenschap be-
handeld: als iets wat men wel of niet heeft. In het laatste hoodstuk gebruiken we
kwantitatieve methoden uit de informatie-theorie, om de hoeveelheid informatie
is bepaalde situaties te minimaliseren. Er worden spelen gedefinieerd waarin het
de bedoeling van bepaalde spelers is om zo weinig mogelijk informatie bloot te
geven, en voor deze spelers worden de optimale strategieën berekend. Een mo-
gelijke toepassing van dit onderzoek ligt in de bescherming van privacy tegen
privacy-schendende technologie.
 Abstract

The research goal behind this dissertation is to develop ways to compare and anal-
yse multi-agent protocols. In order to do so one has to distinguish different types
of protocols, and one has to distinguish different classes of properties. Protocols
that can be modelled as imperfect information game forms are therefore discussed
in the first part of this dissertation, whereas protocols that can be modeled as
imperfect information are the subject of the second part. In both parts we define
concepts that help us to analyse and understand protocols, demonstrate these
concepts on example protocols, and investigate the computational properties of
these concepts.
In chapter 4, a logic for reasoning about what coalitions can achieve in proto-
cols is presented. For this logic, a complete proof system is given, and the model
checking complexity is determined.
In chapter 5, logics for reasoning about more complicated properties are pre-
sented. Specifically we compare the model checking complexity of logics for rea-
soning about side-effects and nested abilities.
In chapter 6, protocols are analysed using logics that deal with preferences
explicitly. For two different variants of preference logics we give completeness
proofs, and as an example, a characterisation of backward induction is given.
Protocols with imperfect information are the topic of the second part of this
dissertation. In these protocols the knowledge that agents have plays a leading
role. One can look at knowledge in a qualitative way, using epistemic logic, and
this is done in chapter 7. In this chapter, it is shown how the computational
complexity of protocol verification, depends on the presence of opponents, on
whether strategies are known, and on the monotonic nature of the knowledge
requirements. In chapter 8, it is shown that one can also model knowledge in a
quantitative way. Using this approach, we compute optimal strategies for privacy
preservation.

199
 Curriculum Vitae

Sieuwert van Otterloo was born on 20 Maart 1979 in IJsselstein (Utrecht, the
Netherlands). He graduated from the Christelijk Gymnasium in Utrecht in 1997,
and went on to study at the Universiteit Utrecht. In 2001 he received the title
of Doctorandus (Master of Science) in Mathematics Cum Laude, and the year
after he also became Doctorandus (Master of Science) in Cognitive Artificial
Intelligence.
In 2002 Sieuwert van Otterloo joined the Department of Computer Science
at the University of Liverpool, where he worked as a PhD scholar. In the final
year he also was a visiting researcher at the Institute for Logic, Language and
Computation of the University of Amsterdam. He successfully defended this PhD
dissertation in Liverpool in November 2005. At the same time he made a next
step in his career, by joining McKinsey&Company in Amsterdam as a business
consultant.

201
Titles in the ILLC Dissertation Series:

ILLC DS-2001-01: Maria Aloni

Quantification under Conceptual Covers

ILLC DS-2001-02: Alexander van den Bosch

Rationality in Discovery - a study of Logic, Cognition, Computation and Neu-
ropharmacology

ILLC DS-2001-03: Erik de Haas

Logics For OO Information Systems: a Semantic Study of Object Orientation
from a Categorial Substructural Perspective

ILLC DS-2001-04: Rosalie Iemhoff

Provability Logic and Admissible Rules

ILLC DS-2001-05: Eva Hoogland

Definability and Interpolation: Model-theoretic investigations

ILLC DS-2001-06: Ronald de Wolf

Quantum Computing and Communication Complexity

ILLC DS-2001-07: Katsumi Sasaki

Logics and Provability

ILLC DS-2001-08: Allard Tamminga

Belief Dynamics. (Epistemo)logical Investigations

ILLC DS-2001-09: Gwen Kerdiles

Saying It with Pictures: a Logical Landscape of Conceptual Graphs

ILLC DS-2001-10: Marc Pauly

Logic for Social Software

ILLC DS-2002-01: Nikos Massios

Decision-Theoretic Robotic Surveillance

ILLC DS-2002-02: Marco Aiello

Spatial Reasoning: Theory and Practice

ILLC DS-2002-03: Yuri Engelhardt

The Language of Graphics

ILLC DS-2002-04: Willem Klaas van Dam

On Quantum Computation Theory

ILLC DS-2002-05: Rosella Gennari

Mapping Inferences: Constraint Propagation and Diamond Satisfaction
ILLC DS-2002-06: Ivar Vermeulen
A Logical Approach to Competition in Industries
ILLC DS-2003-01: Barteld Kooi
Knowledge, chance, and change
ILLC DS-2003-02: Elisabeth Catherine Brouwer
Imagining Metaphors: Cognitive Representation in Interpretation and Under-
standing
ILLC DS-2003-03: Juan Heguiabehere
Building Logic Toolboxes
ILLC DS-2003-04: Christof Monz
From Document Retrieval to Question Answering
ILLC DS-2004-01: Hein Philipp Röhrig
Quantum Query Complexity and Distributed Computing
ILLC DS-2004-02: Sebastian Brand
Rule-based Constraint Propagation: Theory and Applications
ILLC DS-2004-03: Boudewijn de Bruin
Explaining Games. On the Logic of Game Theoretic Explanations
ILLC DS-2005-01: Balder David ten Cate
Model theory for extended modal languages
ILLC DS-2005-02: Willem-Jan van Hoeve
Operations Research Techniques in Constraint Programming
ILLC DS-2005-03: Rosja Mastop
What can you do? Imperative mood in Semantic Theory
ILLC DS-2005-04: Anna Pilatova
A User’s Guide to Proper names: Their Pragmatics and Semanics
ILLC DS-2005-05: Sieuwert van Otterloo
A Strategic Analysis of Multi-agent Protocols
ILLC DS-2006-01: Troy Lee
Kolmogorov complexity and formula size lower bounds
ILLC DS-2006-02: Nick Bezhanishvili
Lattices of intermediate and cylindric modal logics
ILLC DS-2006-03: Clemens Kupke
Finitary coalgebraic logics