Advanced Software Design

Techniques for Agent Systems

Serge Stinckwich
[email protected]
 “We coud take advantage of
advanced software engineering
concepts to built more elaborate
agent systems.”
Agents Architecture

• The software architecture of a program or computing system is the structure or

structures of the system, which comprise software components, the externally
visible properties of those components, and the relationships between them.

• Why architecture for agent systems ?

• a methodology or blue-print for building agents systems

• to predict what an agent in its current state will do in its current environment

• Using software engineering principles : modularization, decomposition,

separation of concerns ... and associated technology : objects, components,
Aspect-Oriented Programing (AOP), reflection, ...

• But specific concerns: programming in the large, adaptation, resources

management, reasoning, cognitive models, ...
Context

• More decentralized and distributed computing (agents, components, service

oriented architectures)

• More mobility and communication (wireless networks, WSN, robotics)

• More embedded systems (“ambiant intelligence”, “pervasive computing”)

Induced problems

•Late interconnection of heterogenous systems

•Variable and impredictable context of deployment

•Computing ressources available not known at design-

time

•Varying load during execution

•Hight-availability and living systems

Solution : adaptable architectures

• Adaptation : The act or process of adapting, The state of being adapted.

Something, such as a device or mechanism, that is changed or changes so
as to become suitble to a new or special application or situation.

• To be adaptable, a system must be built in order to allow modifications a

posteriori before or during its operations.

• Various software mechanisms to do adaptations: hooks, reflection,

framework, ...
When does the adaptation occur ?

•Design time

•Compilation time

•Deployement time

•but static adaptions ...

Dynamic adaptation

• Definition : adaptation during program execution

• To be able to take into account the characteristics of a given execution

(parameters and data), the load or the ressources availability.

• Can be done with:

• with dynamic loadable libraries,

• with self-modifiyng code.

Dynamic self-adaption

• Definition : adaptation where the program carries out itself it own adaptation

• The program must have:

• a knowledge of itself, his code and contents,

• a knowledge of its execution context,

• the tools to do the modifications,

• the capacity to determine when and how to adapt.

Design space of Agents (Jung, 1999)
Architecture vs representation vs algorithms

“Artificial Intelligence” has always

been concerned with algorithms
and representation, but we also
need to understand how to put
various parts together into complete
working systems, within an
architecture.”, Aaron Sloman, 2002
Adaptations to resources

• In Zilberstein (1995), three level of adaption are distinguish :

• Resource-adapted systems that are pre-configure to a specific domain

• Resource-adaptive systems that are able to react to resouce

modifications,

• Resource-adapting systems that explicitly manage and represent

resources
Outline of the talk

(1) Layered architectures

(2) Component-based architectures

(3) Reflective architectures

(1) Layered architectures
From traditional AI view to Reactive Agent
Architecture
• Traditional AI view :

• Internal symbolic representation

• Manipulation of this representation to work out what to do next

• Many unsolved problems associated with symbolic AI, have led some researchers to question the viability of the
whole paradigm and to the development of reactive architectures :

• Effective behaviour does not need representation

• Symbolic representation is very ressource consuming

• Agents must be situated and embodied with “reflex” rules

Traditional decomposition into functional modules
Advantages of reactive agents

•Simplicity

•Economy

•Computational tractability

•Robustness against failure

•Elegance
Limitations of Reactive Agents

• Agents without environment models must have sufficient information available

from local environment

• Short-term view

• Difficult to have learning reactive agents

• Since behavior emerges from component interactions (and environment), how

to engineer specific agents (methodology problem)

• Complexity when large numbers of behaviours (doesn’t scale)

Hybrid architectures

• Neither a completely deliberative nor completely reactive approach is suitable

for building agents

• Hybrid systems : attempt to marry classical and alternative approaches

• An obvious approach is to build with two sub-systems :

• a deliberative one, containing a symbolic world model, which process

plans and makes decisions (symbolic AI)

• a reactive one, which is capable to react to event without complex

reasoning
From hybrid architectures to layered architectures

• Meta-level control of interactions between these components becomes a key

issue in hybrid architectures

• Commonly used: layered approaches

Vertical layering vs horizontal layering

• A key problem is such architectures is what kind of control framework to

embed the agent’s subsystems in, to manage the interactions between the
various layers

• Horizontal layering : layers are directly connected to the sensory input and
action output. Each layer work concurrently and act as an agent producing
suggestions as to what action to perform.

• Vertical layering : Sensory input and action output dealt with by at most one
layer each.
TouringMachines

• TM is a an horizontal layering architecture

• 3 sub-systems : perception sub-system, control sub-system and action sub-

system

• control subsystems consist of:

• reactive layer: situation-action rules

• planning layer: construction of plans and action selection

• modelling layer: contain symbolic representation of mental state of other

agents

• Communication between the 3 layers via explicit control rules

TouringMachines
Vertical Layering : InteRRaP (Müeller, 1996)

• Vertical layering architecture

• InteRRaP = Integration of rational planning and reactive behavior

• Three layers:

• Behavior-based layer: manage reactive behavior of agent

• Local planning layer: individual planning capabilities

• Social planning layer: determine interaction/cooperation strategies, joint

plans
InteRRaP is a 2-pass vertical layering architecture
Conclusion

• Agent architectures : deliberative, reactive and hybrid

• Layers are a way to organize modularization in an agent architecture

• Hybrid architecture : attempt to balance both aspects but increased

complexity
(2) Component based
architecture
Evolution of programming on 3 axis (I)
1st axis : action selection

Monolithic Modular OO Agent-oriented

programming programming programming programming

Behavior non modular modular modular modular

State external external internal internal

jump procedure call method call agent decision

Invocation
(external) (external) (external) (internal)
Evolution of programming on 3 axis (II)
2nd axis : coupling flexibility

Objects Components Agents

implicit, internal (object explicit, external implicit, external

Structure
references) (connectors) (organizational roles)

unidirectional
Communication procedure call protocol
(events) or bidirectional

synchronous or
Synchronization synchronous protocol
asynchronous
Evolution of programming on 3 axis (III)
3rd axis : abstraction level

• Agents, not purely data/procedural. Knowledge (beliefs, goals, ...)

• Semantic/knowledge-level coupling than data-type-level coupling

• Communication (e.g FIPA, ACL vs OMG CORBA)

• content languages (e.g KIF)

• performative (intention of communication, e.g., inform, recruit)

• ontology

• protocol
(Software) Components

• Source of inspiration : electronics, integrated circuits

• Szyperski definition (2002) : “A Component is a unit of composition which

contractually specified interfaces and explicit context dependecies only. A
software component can be deployed independently and is subject to
composition by third parties.”

• Objective : reuse of sofware components

• Objective : ease

• replacement, addition or removal

• of components and/or connectors

 Hollywood principle:
“Don’t call us, we’ll call you”
Components are objects evolution

Objects Components

implicit, internal
explicit, external
structure (object
(connectors)
references)
bidirectional unidirectional
communication (procedure call, (events) or
return value) bidirectionnal

synchronous or
synchronization synchronous
asynchronous
What agents can bring to components based
systems ?

• More flexibility for assembling (match-making)

• Mechanims for dynamic reconfiguration (reorganization)

• More intelligent behavior (adaptive and/or cooperative components)

What component can bring to
agents ?
Components are self-contained units

• Includes all the code

• “Ready to use”, “Ready to deploy” Classes

• Includes documentation Component

object
Examples of components architecture

• Tools/techniques (time series, filters, rules, back propagation, reinforcement

learning, ...)
=> ABLE

• Protocol components
=> AgentTalk, SCD

• Behaviors (gradient following, obstacle avoidance, random move, ...)

=> MALEVA
ABLE : tools decomposition [Bigus 2002]

IBM Autonomic computing

programme
Using components for protocol reusing : AgenTalk
(1995)
SCD : Reusing protocol components (1998)
MALEVA is a component model for agent-based
systems

• J.P Briot, T. Meurisse, F. Peschanski, “Architectural Design of Component-

Based Agents: A Behavior-Based Approach”, ProMAS’2006.

• Domain : multi-agent simulation (trafic simulation, eco-systems, ...)

• Unit of decomposition : agent behavior

• Assembling behaviors into more complex behaviors (composite component),

libraries of behaviors

• Support dynamic change (e.g. from egg to a larva to a worker ant)

• Several implementation : JavaBeans, Smalltalk (MalevaST, 2007)

MALEVA : a general agent architecture
In MALEVA, a distinction is made between data
flow and control flow connections

Input data port Input control port

Output control port Output data port

Data and controls ports in MALEVA

Data Control

Data Activation entry

Input port
consumption point

Activation exit
Output port Data production
point

Activation
Connexion Data transfer
transfer
MALEVA components can be sequentially
composed
MALEVA components can be concurrently
composed
Control structure are reified as control components

• dispatch of control flow

• Example : switch (traditional control structure)

• Other control components :

• iteration component : Repeat

• synchronization barrier : Sync

If a prey detects a predator, it flees away, otherwise
it moves randomly
Predator : reuse of a Prey component !
if the Predator detects a prey, it follows the prey, otherwise, it acts as a Prey
White box or black box are two kind of
compositions
Build an ant worker with hierarchical compositions
Dynamic change of behaviors

• egg => larva => ant

• behavior meta-component :

• setup future behavior

• check which components to keep, remove, add

• reorganize connexions
Advantages of explicit control flow

• Decoupling of activation logic from functionality

• genericity

• fine grain control of intra-agent scheduling

Anticipatory system

An anticipatory system has a model of itself

and of the relevant part of its environment and
will use this model to predict the future. The
predictions are then utilised to determine the
agent’s behaviour, i.e. it lets future states
affect its present states.
Anticipatory agents (Davidsson, 1996)
Anticipatory agent as a kind of hybrid system

• A kind of hybrid agent system

• Two layers
• Reactor: perform reflexes
• Anticipator: make long term plans
• Anticipator
• runs a simulation of the reactor and the world
• make predictions of future states
• detects undesirable states
• modifies the reactor to avoid undesirable states
Using MALEVA to implement an anticipatory agent
(Bouraqadi&Stinckwich 2007)

Reactor
Anticipation

ticks
Ticker

Modifications
World

Reactor
State Current Predicted
Undesirable

Sensed Data
Builder World World
State
State State Modification
Predictor Analyzer
Builder

Modifications
Trigger
Caption
Passive Control flow output interface
Component Control flow input interface
Active Data flow output interface
Component Data flow input interface
Conclusion

• Components can be useful to help at decomposing/recomposing agent

architectures

• Reuse is difficult

• Dual movement :

• distributed systems/applications are getting more adaptable/dynamic

(more semantic support, dynamic reconfiguration)

• agents based systems have greater software maturity (deployment,

methodologies, configuration, ...)
(3) Meta-level and reflective
architectures
 Reflection : “An entity’s integral ability to represent,
operate on, and otherwise deal with its self in the same
way that it represents, operates on and deals with its
primary subject matter.”, Brian Smith (OOPSLA’90
Workshop on Reflection and MetaLevel Architectures)
 “A computational system is something that reasons
about and acts upon some part of the world, called the
domain of the system.”
“ [...] a computational system can be causally
connected to its domain. This means that the system
and its domain are linked in such a way that if one of the
two changes, this leads to an effect upon the other.”
A meta-system is a computational system that has as its
domain another computational system, called its object-system.
[...] A meta-system has a representation of its object-system in its
data. Its program specifies meta-computation about the object-
system and is therefore called a meta-program.”

“Reflection is the process of reasoning about and/or acting

upon oneself. A reflective system is a causally connected meta-
system that has as object-system itself. [...]  When a system is
reasoning or acting upon itself, we speak of reflective
computation.”
Patties Maes (1987)
Examples of meta-system in CS

• When programming, the program is the base system.

• A program manipulating others programs is a meta-system :

• interpreter, compiler

• pretty-printer

• ...
 Metaprogramming vs reflection (for P. Maes)

“The main difference between a meta-level architecture and a

reflective architecture is that a meta-level architecture only
provides static access to the representation of the computation
system, while a reflective architecture also provides a dynamic
access to this representation.”
Example : an object reflective system

the Smalltalk object model

 Until you
Object find the
method.

Color
TranslucentColor Then look in
the superclass.

TranslucentColor
Key
First look in the instance-of
class of the object. message send
translucentBlue reply
class lookup
 Object class
Object

Color class
Color

TranslucentColor class
TranslucentColor
translucentBlue
Key
instance-of
 Object class
Object
Then look in the
Color class superclass, and so on,until
Color you find the method.

Color blue
TranslucentColor class
TranslucentColor
Key
First look in the instance-of
class of the object. message send
reply
blue lookup
 Behavior

ClassDescription

Class

Object class
Object
Color class Metaclass
Color

TranslucentColor class
TranslucentColor

Key
ntBlue instance-of
 Behavior

aTranslucentColor
ClassDescription

Class

Object class
Object
Color class
Color

TranslucentColor class
TranslucentColor
Key
instance-of
message send
reply
new lookup
 Behavior class

Behavior
ClassDescription class

ClassDescription
Class class

Class
Metaclass class

Object class
Object
Color class Metaclass
Color

TranslucentColor class
TranslucentColor

Key
translucentBlue instance-of
Agent applications for reflection

• Meta-control : Russell&Wefald (1991), Stinckwich (1994)

• Anticipatory system : Ekdhal&Davidsson (1995), Bouraqadi&Stinckwich (2007)

• Self-diagnostic, anomaly detection : Kennedy&Sloman

• Adjustable autonomy : Kennedy&Sloman

• Bounded rationality : Jung (1999)

Meta-control for boundedly rational agent
(Russell&Wefald, 1991)
InteRRaP-R (Jung, 1999)
Hybrid reflective agent (Bouraqadi & Stinckwich
2007)

Observes
and
Acts Upon

Knowledge Base Social Planning Layer

Social Other Adaptation

Model Social Coordination
Plans Plans

Communication
Interfaces
Local Planning Layer

Mental Other
Model Adaptation
Local
Plans
Plans

Reactive Layer

Actuators
World
Model Other Adaptation
Reflexes Reflex

Sensors
Adaptations dimensions

• What is adapted ?

• Who performs the adaptation ?

• When does the adaption occur ?

• How is the adaptation performed ?

Separation of concerns (SoC)

• Process of breaking a computer program into distinct features that overlap in

functionality as little as possible.

• A concern is any piece of interest or focus in a program.

• All programming paradigm aid developers in the process of improving SoC :

e.g. object-oriented programming, design pattern, component based
systems, ...
Crosscutting concerns

• In computer science, cross-cutting concerns are aspects of a program which

affect (crosscut) other concerns. These concerns often cannot be cleanly
decomposed from the rest of the system

• Result in either scattering or tangling of the program

• Example : logging is a crosscutting concern

• Solution : AOP (Aspect Oriented Programming)

How AOP works
Using AOP in an example : the Conference
Management System (CMS)

• CMS application supports paper submissions to scientific conference

• Multi-phase process, involving individuals and groups

• Agents represent people such as chairs, reviewers, authors, ...

• Coordination protocols are essential.

• Error Handling is also important in order to enhance system robustness.

?