Ethics and Information Technology

[Link]

ORIGINAL PAPER

Friendly AI
Barbro Fröding1 · Martin Peterson2

© The Author(s) 2020

Abstract
In this paper we discuss what we believe to be one of the most important features of near-future AIs, namely their capacity
to behave in a friendly manner to humans. Our analysis of what it means for an AI to behave in a friendly manner does not
presuppose that proper friendships between humans and AI systems could exist. That would require reciprocity, which is
beyond the reach of near-future AI systems. Rather, we defend the claim that social AIs should be programmed to behave in
a manner that mimics a sufficient number of aspects of proper friendship. We call this “as-if friendship”. The main reason
for why we believe that ‘as if friendship’ is an improvement on the current, highly submissive behavior displayed by AIs is
the negative effects the latter can have on humans. We defend this view partly on virtue ethical grounds and we argue that
the virtue-based approach to AI ethics outlined in this paper, which we call “virtue alignment”, is an improvement on the
traditional “value alignment” approach.

Keywords AI · Friend · Friendly · Value alignment · Virtue ethics

Introduction in 2018. One of the problems with CIMON was that it was
perceived as mean and unfriendly by crew members:
In December 2019 the second generation of the Crew Inter-
In an early demonstration in 2018, it was CIMON
active Mobile Companion robot, known as CIMON-2,
— not Gerst [a German astronaut] — that needed a
arrived at the International Space Station. Designed by the
morale boost. After Gerst asked CIMON to play his
German branch of Airbus, it uses artificial intelligence pow-
favorite song, the 11-lb bot refused to let the music
ered by IBM’s Watson technology. One of CIMON-2’s tasks
cease, defying Gerst’s commands. And, rather than
is to serve as a conversational companion for lonely astro-
acknowledging it had jumped rank, CIMON accused
nauts. According to Matthias Biniok, Lead Watson Archi-
Gerst of being mean and finished with a guilt-trip
tect at IBM, “studies show that demanding tasks are less
flourish by asking Gerst, “Don’t you like it here with
stressful if they’re done in cooperation with a colleague”.1
me?”2
CIMON-2 is programmed to behave like an artificial col-
league by answering questions and engaging in conversation. Sophisticated AI technologies currently reserved for
This enables astronauts to perform better and thereby make space missions are likely to become more widely available
space missions less stressful and more successful. in the future. Some of these AIs will be designed to fulfil
CIMON-2 is a significant improvement of its predecessor, social functions in our daily lives and in the healthcare sec-
CIMON, which was tested at the International Space Station tor.3 Already now there is demand for social companions
(robot dogs and robot cats) for senior citizens residing in
assisted living facilities. It is, arguably, time to discuss what
* Barbro Fröding type of behavior these AIs should be programmed to display.
[Link]@[Link]
In this paper we focus on what we believe to be one of
Martin Peterson the most important features of near-future AIs, namely their
martinpeterson@[Link]
1
Strickland (2020).
1
Department of Philosophy and History, KTH Royal Institute 2
Ibid.
of Technology, Teknikringen 76, 100 44 Stockholm, Sweden 3
When we use the term “AIs” or “AI systems” in this paper we
2
Department of Philosophy, Texas A&M University, 4237 include assistants, companions, facilitators etc. which may or may not
TAMU, College Station, TX 77843-4237, USA be presented as robot technology.

13
Vol.:(0123456789)
 B. Fröding, M. Peterson

ability to (appear to) be friendly to humans, as opposed to Yet another difference, which bears specifically on the
mean, hostile or unfriendly in other ways.4 Presumably, the discussion of friendly AI, is that an AI can, and sometimes
claim that AIs designed to fulfill social functions ought to should, be friendly to you in situations that you would
behave in a friendly manner is an intuitive claim that is hard never encounter together with a human friend. Consider,
to reject. The challenge that lies ahead of us is primarily to for instance, AIs that help users keep track of and analyze
construct a plausible interpretation of what it means for an millions of business transactions. A friend of a human user
AI to behave in a friendly manner, not to show that friendli- would never be asked to perform such a monotone task.
ness is desirable. Therefore, this task cannot be performed by a human in a
Our analysis does not presuppose that proper friendships friendly manner. But as monotonicity would not bother AIs,
between humans and AI systems like CIMON-2 could exist. we can imagine AIs that perform such tasks in friendly as
That would require reciprocity, which is beyond the reach well as unfriendly ways.
of the Watson technology and other current, or near-future, These and other examples indicate that it is far from triv-
social AI systems discussed here. We merely claim that ial to base a theory of friendly AI on an analysis of how a
social AIs should behave in a manner that mimics a suf- proper friend of the user would perform this task. However,
ficient number of aspects of proper friendship. We will call as we will explain shortly, our suggestion for how AI sys-
this “as-if friendship”. Although AIs could perhaps mimic tems should be programmed to behave can take care of the
reciprocity (‘as-if reciprocity’), we take proper friendship to objections outlined here. Our proposal is that for a human
require proper reciprocity, meaning that as-if reciprocity can person to qualify as friendly, she should mimic a sufficient
at most generate as-if friendship.5 number of aspects, but not all, of proper friendship. For
To start with, we note that being friendly to someone example, behaving friendly could plausibly require sincere
is not the same as being that person’s friend. Consider, for well-wishing, the intrinsic valuing of the other, helpfulness,
example, the staff in your favorite coffee shop, bar, grocery and empathy. Proper friendship, on the contrary, requires
store or gym. Presumably they are friendly to you, but that more, for example a commitment to honesty which in exam-
does not mean that they are your friends in a deeper sense. ples of self-destructive behavior would require the proper
A way to think about the notion of human friendliness is friend to confront the other. Hence, a friendly AI system
the following: an individual is friendly in case she behaves as should, in the case of the addict, not be required to confront
a proper friend would behave in a similar situation. Because the user (although a future AI capable of proper friendship
not all friends behave in the same way, this criterion entails would), and in the case concerning monotone work tasks,
that people can be friendly in numerous ways, which we the friendly AI would not complain (unlike the first version
take to be plausible. However, the above suggestion needs of CIMON). Our strategy for analyzing AI friendliness can
to be qualified as there are situations where friendliness does thus be summarized as follows: We use the notion of proper
not require the same type of behavior as proper friendship. human friendship as a point of departure; we will return to
Consider, for example, how a proper friend will sometimes what we mean by proper human friendship later in the paper.
be required to behave in a way that is not friendly in the We then define human friendliness as behavior that mimics
everyday sense. Imagine that you come to know that your sufficiently many aspects of proper friendship. Although AIs
friend has some very negative habits (e.g. over-consumption cannot be proper friends, we claim that AIs can be friendly
of alcohol). In this case true friendship could require that if they mimic sufficiently many aspects of proper friendship,
you intervene, and this involves confrontation, truth-tell- but as noted we maintain that the ways in which humans and
ing and accountability. However, this is not expected from AIs mimic friendship relations need not always be the same.
someone who is merely friendly to you. In addition, we also As we see it, the most troubling ethical challenge with
note that a proper friend can, in other cases, be legitimately designing friendly AIs is that we may end up treating the
required to sacrifice herself to some extent; she sometimes AIs as mere slaves rather than as-if friends. This would be
has to inconvenience herself and put the good of the friend undesirable because, as will be elaborated on in the follow-
before her own good. Again, this is typically not required of ing sections, this is likely to have a negative impact on the
someone who is merely friendly. development of the human users’ character virtues. For the
sake of clarity, we will articulate and discuss three types of
as-if relationships that can hold between AIs and humans,
4 which we believe it is important to keep apart:
In this paper we limit our discussion to non-antagonistic AI sys-
tems. If it is ever morally permissible to use AI systems in warfare,
which is a controversial claim we will not discuss here, then such mil-
1. Slave AIs: These AIs are programmed to behave as-if
itary AIs should arguably be allowed to be hostile to the enemy.
5 they were slaves controlled by human masters.
Note that we do not take all forms of as-if friendship to require as-if
reciprocity. Some lesser forms of as-if friendship require no type of
reciprocity at all; see “Aristotle on friendship” section.

13
Friendly AI

2. Utility AIs: These AIs are programmed to behave as-if i.e. as a facilitator of human needs and desires. Our concern
they were facilitators of the common good in human is not that Siri, Alexa or CIMON can suffer, feel pain, or
societies. have moral rights. We do not claim that we can wrong AIs
3. Social AIs: These AIs programmed to behave as-if they by treating them as artificial slaves. Our concern is rather the
were friends of individual humans. negative effects this way of thinking of AIs may have on us.
At first look this might sound counterintuitive. Why
We reserve the terms “friendly AI” for AIs of the second should we not have the AI technology at our disposal to do
and third types, that is, utility AIs and social AIs. Whether our bidding and facilitate our lives as much as possible? To
slave AIs could be friendly is debatable, but irrelevant in this we have two answers, the first is based on virtue ethics
the present discussion as we claim that slave AIs are mor- and the second on consequences. The virtue ethics response
ally problematic and should be transformed into friendly goes as follows: In order to flourish and lead the good life
utility AIs or friendly social AIs. We present our argument we need to develop a set of moral and epistemic virtues
for this claim in the section called “Why treating AIs as that inform our behavior. Much of this instilling of virtues
slaves is morally wrong” and then contrast the notion of is about practicing and mimicking good behavior. If we are
friendly utility AIs and friendly social AIs with Asimov’s surrounded by artificial intelligent entities programmed
well-known laws of robot ethics. We point out that if the to behave like slaves, then that is unlikely to facilitate the
notion of friendly AI is construed in terms of as-if friend- development of our virtues. Indeed, it seems to allow, per-
ship, then two of Asimov’s three laws of robot ethics have haps even encourage, us to behave viciously. Were the AIs
to be rejected for what we take to be convincing reasons. rather to behave in a friendly manner, which includes setting
In the section called “Aristotle on friendship” we summarize boundaries, then they would regulate our behavior and at the
some of the key aspects of Aristotle’s theory of friendship, very least not actively undermine the development of virtue.
which is put to work in the section called “Two types of More speculatively, but not impossibly so, given that AIs
friendly AIs” for articulating our preferred notion of friendly will be more advanced in the future, utility AIs and social
AI in terms of as-if friendship. In the final section “Conclud- AIs could become role models and inspire virtue in humans.
ing remarks” we conclude the paper by pointing out some The consequentialist objection to tolerating slave AIs is
differences between our approach and the traditional value the following: If we get used to having our AI slaves doing
alignment project. our bidding that might spill over on how we behave toward
human beings. While this claim is of course based on empir-
ical assumptions, it seems likely that a certain behavior that
Why treating AIs as slaves is morally wrong we get used to in one setting (i.e. at home) may influence
how we behave in other settings, say work, society and with
Nearly all AIs developed so far, including CIMON and friends. This holds true especially if the AI is advanced and
CIMON-2, are premised on the idea that the purpose of the perceived to be similar (capacity wise and/or look wise) to
AI is to improve, facilitate, or simplify human activities. a human. Consider, for example, how over-consumption of
This is, to a certain extent, unproblematic from a moral point violent films, video games etc. can have a normalizing effect
of view. The near-future AIs we discuss in this paper have and stimulate certain individuals to export their fantasies to
no moral standing, so it would make little sense to raise the real life.7
Kantian objection that the AI is being used as mere means Our concerns about treating AIs as slaves differ in at least
to an end. It is indeed true that AIs are being used as mere two ways from more mainstream approaches to AI ethics
means to an end, but that is permissible because they have such as Asimov’s laws of robot ethics. Firstly, the prime
no moral agency. concern of Asimovs’s principles is that humans might be
However, we would like to highlight another, more prob- harmed by robots. Secondly, the laws of robot ethics do not
lematic attitude toward the different versions of CIMON and seem to allow for the type of as-if friendship we advocate.
near-future AIs, namely, our tendency to think of AIs as arti- Consider Asimov’s first law.8
ficial slaves. Some of the best-known AIs we use in our daily First law: A robot may not injure a human being or,
lives have ordinary human names such as Siri or Alexa. 6 through inaction, allow a human being to come to harm.
These AIs are frequently ascribed, explicitly or implicitly, a This law is incompatible with our account of friendly AI.
range of human-like properties. Notably, we think nothing It is possible that a friendly social AI behaving in a man-
of treating such AIs as we would treat any other machine, ner that mimics a sufficient number of aspects of proper

7
6
Cimon is the name of an influential Athenian statesman who lived Greitemeyer and Mügge (2014), Adachi and Willoughby (2013)
8
c. 500 BC. Asimov (1950)

13
 B. Fröding, M. Peterson

friendship will sometimes injure humans, or allow that to on the airline’s website is a real person or an AI. If you
happen, so doing so is not always wrong. The problem is become accustomed to treating AIs as slaves and believe that
that Asimov’s first law is overly absolutistic. It is sometimes the “agent” chatting with you is an AI you are more likely
permissible to injure a friend physically or cause her psycho- to not treat the unknown entity you are chatting with dignity
logical harm, or even allow a friend to injure herself. Avoid- and respect. If the airline representative is a real person she
ing harm is not the only moral value friends care about; will be harmed by your behavior, at the same times as you
other important values include truth and overall well-being. may also be harming yourself indirectly by becoming less
Imagine, for instance, that a person under the influence of generous, moderate and respectful to others.
alcohol is trying to start their car with the intention to drive Third law: A robot must protect its own existence as long
off. A friendly social AI would intervene even if doing so as such protection does not conflict with the first or second
would result in (light) physical harm to come to the driver. law.
Second law: A robot must obey the orders given it by Unlike the first two laws, we believe that the third law
human beings except where such orders would conflict with is fairly uncontroversial. Indeed, we agree that friendly AI
the first law. should, within reason, protect their own existence. How-
Asimov’s second law is, just like the first, incompat- ever, the discussion in this section shows that the notion of
ible with the notion of friendly AI developed in this paper. friendly AI introduced above differs in fundamental ways
An AI behaving in a friendly way should not always obey from Asimov’s theory of AI ethics. In the next couple of sec-
orders issued by a higher ranked authority. Unethical orders tions we will render our notion of friendly AI more precise.
should typically be disobeyed. Imagine, for instance, that
an authoritarian regime uses AI systems for erasing valu-
able historical records cataloging human thoughts, or for Aristotle on friendship
manipulating elections in foreign countries. Erasing histori-
cal records does not always under all circumstances harm Before we critically discuss the suggestion that we ought
anyone (even if that may of course sometimes be the case) to transform slave AIs into friendly utility AIs or friendly
and it is theoretically possible that the manipulation of an social AIs, it is helpful to provide an account of friendship.9
election in a foreign country has no consequences at all for A natural point of departure is Aristotle’s seminal account
anyone. If, say, a single vote is counted twice that may not in the Nichomachean Ethics (NE). Aristotle argues that there
affect the distribution of seats in parliament. However, even are three main qualities for which someone is cherished. The
if no one is harmed it would nevertheless be unethical to use first is usefulness, the second is pleasantness, and the third
AIs for erasing historical records or manipulating elections. is excellence. According to Aristotle these qualities translate
We may, for example, believe that the collected thoughts into three types of friendship.10
of past generations have intrinsic value in an impersonal
sense, just as fair elections. The problem for Asimov is that 1. Friendships based on mutual admiration
according to his second law, no AI would be allowed to diso- 2. Friendships based on mutual pleasure
bey the orders described here even when they are unethical. 3. Friendships based on mutual advantage
A friendly social AI, on the other hand, would not partici-
pate in the destruction of valuable historical documents or According to Aristotle the first type of friendship is more
manipulate elections. valuable than the other two because it is based on excellence.
Another reason for rejecting the second law is that it In this type of friendship, what the two friends admire is the
permits us to treat AIs as electronic slaves. As explained virtue of the other; it deals with the inner qualities of a per-
above we think that the electronic slave itself (the AI) is not son. It has been observed that this is a highly moralized idea
harmed by being enslaved, but if slavery is permitted that is of friendship which does not capture all the good of such
likely to have negative effects on us. If we get accustomed relationships (see Cocking and Kennett 2000). While that
to the idea that we somehow own and control another intel- may well be true, the mutual admiration type of friendship
ligent (electronic) being, then that is likely to make us less is not a description of the as-if friendship that we believe
sensitive to other moral issues. The wrongness of AI slavery that AIs should be programmed to display. As made clear
is thus not supervenient on the wrongness of ordinary slav- earlier we do not believe that human beings ought to have
ery, but rather on the negative effects AI slavery has on us. friendship feelings towards a machine. One reason for this
If we get accustomed to treating AIs as slaves this is likely
to affect our dispositions toward each other in harmful ways,
which will eventually harm ordinary humans. Consider, for 9
This account draws on Björkman (2008).
instance, a Turing-like scenario in which you struggle to 10
See Book 8.3, /NE1156a6-8/. We discuss this account in Fröding
determine if the airline representative you are chatting with and Peterson (2012).

13
Friendly AI

is that there can be no reciprocity since the AI will merely in her own right. A relationship like that between Ana and
be programmed to display a behavior that mimics, but is not Dr. Mona does not meet Aristotle’s friendship criteria of
proper, friendship feelings. goodness. Ana does not love Dr. Mona for her own sake, or
In what follows we will focus on the other two forms vice versa—quite to the contrary they seem to appreciate
of friendship described by Aristotle. While they are lesser very different things in one another. What Ana finds good
forms of friendship (as they are less complete), it has been and useful in Dr. Mona might be her medical skills and her
argued that they might still qualify as genuine forms of willingness to help Ana’s children. What Dr. Mona finds
friendship as they are similar enough.11 This resembles our good, pleasant and useful in Ana might be her music skills
idea of as-if relationships mimicking a sufficient number of and her willingness to offer piano lessons.
the aspects that constitute proper friendship. Notably, on the What would be the problem in this situation? Very
Aristotelian account it is possible to have minor ends, e.g. broadly it seems it could be rejected on two grounds. Firstly,
utility and pleasure, and at the same time be committed to this is not love; to the extent that there is any genuine good-
eudaimonia as the most teleios end. will and love here it is drowned out by the (mutual) exploita-
Friendships of the second and third type are less excel- tion, and secondly, there is goodwill and even a little love but
lent forms of friendship but as argued by Aristotle in / the problem is that the feelings are primarily based on how
NE1157a30-33/ such relationships can still be called friend- useful the other is. If either party ceases to deliver it seems
ships as they can also contain elements of genuine love and plausible that the relationship will lose its appeal to either
goodwill.12 A possible reading is that while friendships one or both of the parties. But if this type of relationships
based on mutual pleasure or mutual advantage may not con- are rejected, some scholars (see, for instance, Cooper 1977)
tain all the features of friendships based on mutual admira- have expressed concerns that a first-class friendship might
tion, they contain enough such features to qualify as friend- become an unattainable ideal.15 He, and others, argue that
ships. This, however, is a contested reading. Critics have a better reading of Aristotle is that the perfect friendship
argued that friendships based on mutual pleasure or mutual is intended as a paradigm case, not as a description of the
advantage do not qualify at all, while others have suggested only acceptable form of friendship we can have. It would
various intermediate positions.13 Part of the debate regards then follow that the lesser forms of friendship can also be
how much goodwill friendships based on mutual pleasure or worthwhile and contain genuine well-wishing and goodwill
mutual advantage have to contain in order to ‘count’. for the sake of the other. Such a reading appears to be in
To further complicate matters, Aristotle does not make line with Aristotle’s general use of paradigm cases, i.e. the
it entirely clear what is meant by “goodwill”. In some parts examples he uses for illustrating perfect instances of virtues
of the NE the concept of goodwill is used in a very broad like courage, generosity and indeed friendship. But while
sense, seemingly covering everything from the very low such instances undoubtedly are excellent, perfection is not
level philia one might feel towards all of humanity to strong the only way. There can also be friendships of a lesser, but
feelings of love and affection. In other parts, however, the good enough kind. Exactly which elements such relation-
notion of goodwill seems to refer to the general benevolence ships would have to contain is an open question but this
and respect we ought to feel towards other humans14 whereas idea is in line with the argument in this article where we
both “liking” and “loving” are narrower. talk of AIs being programmed to mimic a sufficient number
To make this more applicable to the subject discussed as aspects of proper friendship and act in line with them.
here, consider the following example. Ana loves her friend In the NE (e.g. in Book 8.9–8.12) Aristotle also talks
Dr. Mona for her willingness to let Ana jump the queue at about another form of friendship, namely that of civic
the emergency room and attend to medical issues whenever friendship. Civic friendship is the type of general affection
Ana or her children need medical care. Ana, who is a piano people in the same city or country could feel for each other,
teacher, returns the favor to the best of her ability by offering and such a feeling is central to the good functioning of the
Dr. Mona free piano lessons. Ana does not love Dr. Mona in political society as it maintains the whole social project.16
her own right as she would if they had had a friendship According to Aristotle both justice and friendship are cen-
based on mutual admiration, nor does Dr. Mona love Ana tral to the well-functioning society: the friendship binds the
citizens together and enable them to go after a common goal
but, equally, without justice the whole cooperative system
11
See e.g. Cooper (1977).
12
See also /NE1157a25-35/ and /NE1158b5-11/).
13 15
For a debate see e.g. Price, Walker and Cooper in Price (1989), Cooper (1977)
Cooper (1977), Walker (1979). 16
Aristotle compares the structure of the family to that of the struc-
14
This is what we take Aristotle to mean when he talks about of ture of political society see e.g. /NE1160b24-27/ /NE1160b33-
philia for all others. 1161a3/.

13
 B. Fröding, M. Peterson

just collapses. So while justice is key to maintaining politi- displays sincere well-wishing, the intrinsic valuing of the
cal order, friendship also plays an important role in keeping other, the commitment to honesty, loyalty and other shared
society together: values.
The reason why the AI should be transformed into
…friendship would seem to hold cities together, and
friendly utility AIs or friendly social AIs is that such inter-
legislators would seem to be more concerned about
action will facilitate the development of human virtues
it than justice. For concord would seem to be simi-
required for eudemonian lives. In order to maintain the vir-
lar to friendship, and they aim at concord among all,
tues we need to exercise them, and as AI technologies get
while they try above all to expel civil conflict, which
increasingly advanced they will be embedded in the fabric
is enmity. Further, if people are friends, they have no
of society. Plausibly, then, many human-to-human contacts
need of justice, but if they are just they need friendship
will be replaced by interactions with machines and, if those
in addition; and the justice that is most just seems to
machines are not programmed to be friendly utility AIs or
belong to friendship. (/NE1241a16-18/.)
friendly social AIs, humans would be deprived of many
While friends might not agree on everything they would opportunities to practice the virtues. This would be bad for
certainly have a joint basic understanding of how life should us as it reasonably could be expected to result in less vir-
be lived and what overall goals are worthy of pursuit.17 tuous behavior, as we would get less skilled at identifying
Notably, however, this would not require the friends to lead and being sensitive to relevant moral features of situations.
very similar lives—they might certainly pursue the good Notably, our only concern is what is good for humans. We
in different ways but they would, as Aristotle put it, share are not arguing that the AI itself could benefit, be harmed,
“conversation and thought”.18 or be affected in any other morally relevant way.
To reconnect the tripartite Aristotelian account of friend- For an example of a friendly utility AI, consider a sce-
ship with our account of AI friendliness recall how we nario in which an AI functions as a decision-support system
reserved the term friendly AI for utility AIs and social AIs. nudging towards environmentally friendly transport choices:
While our two categories do not map on to any single type On a rainy morning it is more likely that car owners will
of friendship as defined by Aristotle they both share features drive to work than they are to use public transport. Pollution
from pleasure and advantage but not from mutual admiration from cars has a negative impact on the environment so were
as that would require reciprocity and, possibly, complete vir- more people to use public transport that would be positive.
tue. These relationships will be further elaborated on below. Therefore, a friendly utility AI could function as a decision-
support system designed to nudge behavior promoting the
collective good by for example providing information about
Two types of friendly AIs local timetables, transport time to destination (it could well
be faster than the car option), cost savings, remind the per-
Against this Aristotelian account of friendship, we are now son of the positive effects on the environment their behavior
in a position to flesh out our suggestion that slave AIs ought would have and so on. Suggestions and prompters of a simi-
to be transformed into to friendly utility AIs or friendly lar kind could equally well have come from a friendly human
social AIs. Recall that there is no reciprocity between AIs e.g. one’s partner or a neighbor. Admittedly someone in a
and humans. Neither the human user, nor the AI, ought to position of authority (instead of a friend) could make a simi-
feel any proper friendship feelings toward each other.19 The lar suggestion but a difference then would be that they could
human user should simply recognize that the AI is a machine force you to comply, which is not the case with a friend or
and the AI can at best be programmed to mimic friendly the friendly AI: it merely makes a suggestion. Compare this
behavior. This could, for instance, include behavior that with the notion that the AIs around us are here simply to
facilitate our lives and make them as pleasurable and easy as
possible. That would be the behavior we would expect from
17
/NE1241a16-18/. a slave AI. Plausibly, such AIs could fuel a sense of indiffer-
18
/NE1170b12/. ence and stifling inertia with regards to decisions that impact
19
Recall that we in “Introduction” section defined proper friendship others in a tangible way. As many decisions which are in
as containing sincere well-wishing, the intrinsic valuing of the other, line with the common good collide with short term personal
helpfulness, empathy and a commitment to honesty as well as an obli-
gation to sometimes put the good of the friend above your own. Can convenience—e.g. taking the bus—an AI which incentivizes
an AI behave as if it has such feelings? We believe that it could be and prompts a less selfish and more socially and environ-
programed to behave in such a way. After all it is the actual behaviour mentally sustainable behavior would be significantly better
of the AI that will trigger the virtuous response and enable the culti- than an AI programmed to accept being treated as a slave.
vation of virtue in the human and it does not require the human to be
under the illusion that the AI is something else or something more For an example of a friendly social AI, consider a sce-
than a machine. nario in which an AI system functions as a decision-support

13
Friendly AI

system as well as a partner or coach in the habituation of the AIs be designed to respect? Even assuming that the relevant
virtues. Imagine, for instance, something like a future gen- values could be identified, individuated, agreed upon, and
eration of CIMON. Such an AI could plausibly function as a value conflicts mitigated, it would still be a complex task
partner facilitating emotional regulation and development of to identify the relevant utility function to be programmed
the moral and intellectual virtues of the human user. In addi- into the AI.
tion to prompting positive behavior friendly AIs like future The suggestion of the present article, namely that AIs
versions of CIMON could also assist in reducing behavior should be programmed to behave in a manner that mimics a
that would be negative for the user. Imagine for instance a sufficient number of aspects of proper friendship, avoids at
person who frequently lies in their social media communica- least some of these problems. Computers are good at analyz-
tion. On no plausible account of virtuous behavior could this ing similarities and differences in large data sets. Therefore,
qualify as acceptable. A friendly social AI should therefore if the aim is to mimic certain examples of human behavior,
assist not only in flagging the lie and possibly stop the post- which are identified ex ante by humans as virtuous, and then
ing but also play an important role in the development of use those examples as prototypes in a training process, this
the user’s virtues. For example, a friendly social AI could seems more likely to result in stable AI systems that can be
explain why the behavior is bad, offer a set of training sce- called friendly and promote human virtues in society. There
narios, and be a partner in practicing. The friendly social will be no need to identify a complete set of moral values
AI can be expected to contradict, criticize and reprimand ex ante, nor to somehow balance conflicting values against
when required and this would be required for facilitation of each other with the aim of constructing a suitable utility
the development of the virtues. Such behavior will not be function. By instead focusing the human input on identifying
displayed by a friendly utility AI or a slave AI. behavior that exemplifies prototypical virtues, the question
Understandably, these scenarios might raise concerns about values becomes redundant, or at least secondary. So
pertaining to lack of privacy, autonomy, informed consent instead of a value alignment process, we recommend a virtue
and a reduction of the individual’s freedom. We agree that alignment process.
these are important concerns. The development and use of
friendly social AIs (and perhaps also of some friendly util-
ity AIs) should be held to high ethical standards regarding Funding Open access funding provided by Royal Institute of
Technology.
transparency, safety, responsibility, justice, social sustain-
ability and the promotion of wellbeing. However, the fact Open Access This article is licensed under a Creative Commons Attri-
that friendly social AIs may raise ethical concerns of this bution 4.0 International License, which permits use, sharing, adapta-
sort is not a reason for not developing them. Nearly every tion, distribution and reproduction in any medium or format, as long
new technology can trigger reasonable moral concerns if it as you give appropriate credit to the original author(s) and the source,
provide a link to the Creative Commons licence, and indicate if changes
is used in the wrong way. Consider for example, gene editing were made. The images or other third party material in this article are
technologies or various forms of surveillance technologies. included in the article’s Creative Commons licence, unless indicated
Both have great potential for good and for bad. What this otherwise in a credit line to the material. If material is not included in
shows is that we have to be careful when we develop and the article’s Creative Commons licence and your intended use is not
permitted by statutory regulation or exceeds the permitted use, you will
use friendly social AIs, not that it is unconditionally wrong need to obtain permission directly from the copyright holder. To view a
to do so. copy of this licence, visit [Link]

Concluding remarks References

We believe the virtue-based approach to AI ethics outlined Adachi, P. J., & Willoughby, T. (2013). Demolishing the competition:
in this paper has several advantages over the traditional The longitudinal link between competitive video games, competi-
tive gambling, and aggression. Journal of Youth and Adolescence,
value alignment approach advocated by Stuart Russell and 42(7), 1090–1104.
others.20 There is much disagreement on how such a value Aristotle. (1980). The nicomachean ethics, (W. D. Ross, Trans.), OUP
alignment process is supposed to be conducted.21 For exam- Asimov, I. (1950). I Robot. New York: Gnome Press.
ple, how can AI systems be prevented from inadvertently Björkman, B. (2008). Virtue ethics, bioethics and the ownership
of biological material. Theses in Philosophy from the Royal
acting in a way that is harmful to humans or incompatible Institute of Technology 28. viii + 203 pp. Stockholm. ISBN:
with human values? And what specific list of values should 978-91-7178-993-8.
Cocking, D., & Kennett, J. (2000). Friendship and moral danger. The
Journal of Philosophy, 97(5), 278–296.
20
Cooper, J. M. (1977). Aristotle on friendship. In A. O. Rorty (Ed.), Essays
See, for instance, Russell (2016). on aristotle’s ethics. California: University of California Press.
21
For an overview, see Peterson (2019).

13
 B. Fröding, M. Peterson

Fröding, B., & Peterson, M. (2012). Why virtual friendship is no Russell, S. (2016). Should we fear supersmart robots. Scientific Ameri-
genuine friendship. Ethics and Information Technology, 14(3), can, 314(6), 58–59.
201–207. Strickland, A. C. (2020). Robot arrives on the space station to keep
Greitemeyer, T., & Mügge, D. O. (2014). Video games do affect social astronauts company. Retrieved January 29, 2020, from https://
outcomes: A meta-analytic review of the effects of violent and [Link]/2019/12/06/world/cimon-space-station-scn-trnd/
prosocial video game play. Personality and Social Psychology [Link]
Bulletin, 40(5), 578–589. Walker, A. D. M. (1979). Aristotle’s account of Friendship in the Nico-
Peterson, M. (2019). The value alignment problem: A geometric machean Ethics. Phronesis, 4(2), 180–196.
approach. Ethics and Information Technology, 21, 19–28.
Price, A. W. (1989). Love and friendship in plato and aristotle. Oxford: Publisher’s Note Springer Nature remains neutral with regard to
Clarendon. jurisdictional claims in published maps and institutional affiliations.

13