This document summarizes a keynote talk on multi-agent modeling and its applications to robotics and cognition. The talk discusses what constitutes an agent and examines cognition from the perspectives of senses, thinking, emotions, and cognitive architectures. It also explores two types of agent embodiment: robots, which impose challenges related to physical limitations and neurology; and avatars, which raise questions about virtual bodies. The talk aims to bring together different areas of research in developing cognitive systems and modeling human behavior.

1. Mul$-­‐Agent
Modelling
With
applica$ons
to
robo$cs
and
cogni$on
Dr.
Aladdin
Ayesh
Co-­‐ordinator
of
Intelligent
Mobile
Robo$cs
and
Crea$ve
Compu$ng
Group
(IMRCC-­‐Group)
School
of
Informa$cs,
De
MonEort
University
email:
aayesh@dmu.ac.uk;
URL:
www.aladdin-­‐ayesh.info
Keynote
talk
at
ESM
2008
Conference

2. Agenda
• What
is
an
agent?
• Cogni$on,
thinking
and
conciseness
• Robo$cs:
the
ques$on
of
neurology
(senses)
• Avatars:
the
ques$on
of
body
• Applica$ons
– From
AI
to
SoUware
Engineering
– Swarms
and
Social
Modelling
– Human-­‐Computer
Interac$on
Keynote
talk
at
ESM
2008
Conference

3. What
is
an
agent?
• This
ques$on
has
been
a
philosophical
minefield.
• To
simplify
this
ques$on
let
us
look
at
two
separate
issues
associated
with
agents:
being
and
the
no$on
of
agency.
• In
being,
let
us
take
a
human
being
Keynote
talk
at
ESM
2008
Conference

4. What
is
an
agent?
• What
are
we
as
systems
concerned?
– Senses?
– Motors?
(Neuromotor-­‐psychomotor)
– Mind?
Agent
Structure
• Now
is
the
no$on
of
agency
senses
cogni$on
motor
Keynote
talk
at
ESM
2008
Conference

5. Cogni$on,
thinking
and
conciseness
• Cogni$on
encompasses
the
different
facul$es
associated
with
human
operability:
thinking,
memory,
language,
etc.
(cogni$ve
psychology)
• Affects
are
important
and
highly
connected
to
cogni$on
but
they
usually
stand
separate.
(phenomenological
psychology)
Keynote
talk
at
ESM
2008
Conference

6. Interest
in
emo$on
modeling
has
been
increasing
in
the
last
few
years
specially
with
the
increase
interest
in
cogni$ve
systems,
especially
as
they
are
implemented
into
games
and
virtual
worlds.
There
are
number
of
issues
related
to
emo$on
modeling:
selec$on
of
emo$ons,
represen$ng
an
emo$on,
expressing
emo$ons,
recognizing
emo$ons
and
so
on.
Each
one
of
these
issues
is
a
research
area
by
itself.
h]p://www.computa$onal-­‐emo$ons.org/
is
a
web
site
under
construc$on
with
the
aim
to
provide
informa$on
on
research
in
this
area.
NOTE
ON
EMOTION
MODELING
Keynote
talk
at
ESM
2008
Conference

7. Cogni$on,
thinking
and
conciseness
• Similar
to
emo$on
modelling
each
cogni$ve
faculty
is
a
research
area
by
itself,
e.g.
memory
architectures,
learning
algorithms,
etc.
• What
brings
these
different
areas
together
to
form
a
cogni$ve
system?
Keynote
talk
at
ESM
2008
Conference

8. Cogni$on,
thinking
and
conciseness
• A
cogni$ve
architecture
is
oUen
developed
for
the
purpose
of
pu_ng
together
the
building
blocks
of
the
internals.
• Most
architectures
are
developed
to
explore
par$cular
idea,
e.g.
child
development
through
implemen$ng
cogni$ve
learning
theories.
Few
aim
to
develop
a
generic
fits
all.
Keynote
talk
at
ESM
2008
Conference

9. Cogni$on,
thinking
and
conciseness
• All
of
these
architectures,
however,
need
a
container
in
which
to
be
implemented.
An
agent
embodiment!
Keynote
talk
at
ESM
2008
Conference

10. ROBOT
OR
AVATAR
So
when
we
explore
cogni$on
we
have
to
explore
body-­‐mind
rela$onship
or
the
embodiment
of
that
cogni$on.
As
systems
and
engineering
concerned,
there
are
two
forms
of
decision-­‐making
embodiments,
each
goads
number
of
outstanding
philosophical
inquires
and
evocates
new
systems
related
ques$ons.
Keynote
talk
at
ESM
2008
Conference

11. Robo$cs:
the
ques$on
of
neurology
• Robo$cs
are
effec$vely
a
physical
form
of
agents.
• They
have
the
full
structure:
senses-­‐control-­‐
motor
• But
because
they
are
physical
they
impose
ques$ons
related
to
physical
limita$ons,
e.g.
sensory
errors,
real-­‐$me
response,
and
physical
embodiment.
Keynote
talk
at
ESM
2008
Conference

12. Robo$cs:
the
ques$on
of
neurology
• These
physical
limita$ons
forces
us
to
look
at
the
neural
construc$on
of
human
being
and
the
signal
processing
aspect
of
cogni$on.
• However,
we
are
not
all
signal
processing,
and
a
domes$c
robot
should
stop
an
think
from
$me
to
$me.
• This
gives
a
rise
to
hybrid
architectures
of
localized
responses
with
global
reasoning
(conscious).
Keynote
talk
at
ESM
2008
Conference

13. NOTE
ON
AESTHETICS
The
shape
of
the
robot,
func$onality
and
the
aesthe$cs
of
look
and
behavior
has
great
effects
on
accep$ng
the
physical
en$ty.
One
of
the
projects
we
done
on
this
front
in
AIBO
stories
and
AIBO
oracle.
But
can
we
formalize
the
aesthe$cs
of
buddying
devices
in
a
quan$fied
rules?
Keynote
talk
at
ESM
2008
Conference

14. Avatars:
the
ques$on
of
body
• Are
avatars
agents?
• The
strict
answer
is
no,
not
really!
• Many
agent
systems
use
no
graphical
representa$on
or
oUen
a
simplified
one,
i.e.
a
dot!
This
was
the
big
cri$cism
against
agents
and
simula$ons,
no
realis$c
physical
simula$on.
Keynote
talk
at
ESM
2008
Conference

15. NOTE
ON
PHENOMENOLOGY
OF
BEING
Many
people,
nowadays,
develop
their
avatars
to
reflect
their
percep$on
of
themselves
rather
than
themselves.
I
have
started
about
a
year
ago
to
collect
evidence
on
how
people
use
avatars
and
online
profiles
to
project
their
internal
representa$on
of
themselves
or
what
they
wish
they
were.
Far
from
developing
avatars
to
carry
their
conscious
to
immortalise
them,
they
are
reincarna$ng
themselves
of
an
an$-­‐existen$alist
phenomena.
Keynote
talk
at
ESM
2008
Conference

16. Applica$ons
• Agent
technologies
have
many
applica$ons
and
almost
appear
as
if
they
are
the
answer
to
all
system
problems.
• To
men$on
but
few
applica$ons:
– Web
enterprise
applica$ons
– Ubiquitous
and
pervasive
intelligence
(e.g.
Intelligent
homes)
– eLearning
– Controllers
And
many
many
many
more!
Keynote
talk
at
ESM
2008
Conference

17. From
AI
to
SoUware
Engineering
• Agent-­‐based
system
analysis
and
design
is
becoming
fast
the
new
OOSAD.
• The
reasons
are:
– Support
of
distributed
systems
– Support
of
complex
systems
scalability
portability
Interoperability
versa$lity
Keynote
talk
at
ESM
2008
Conference

18. Swarms
and
Social
Modelling
• The
very
early
applica$ons
of
mul$-­‐agent
systems
were
ecological,
economical
and
social
modelling
applica$ons.
• Simply
the
applica$ons
had
a
number
of
independent
actors,
with
what
seems
as
free
well,
but
governed
by
clear
infinite
set
of
(simple)
rules.
The
intelligence
is
distributed
among
these
en$$es
and
emerge
from
the
interac$on
of
the
rules
DAI!
Keynote
talk
at
ESM
2008
Conference

19. Swarms
and
Social
Modelling
• With
the
development
of
object-­‐based
agents
the
rules
are
distributed
in
the
en$$es
and
intelligence
appears
as
part
of
interac$on
within
the
collec$ve
Collec$ve
Intelligence
• Endless
analogical
examples
in
nature
provided
inspira$on
for
a
whole
new
sets
of
algorithms:
ACO,
PSO,
Beehives,
flocking,
etc.
Keynote
talk
at
ESM
2008
Conference

20. Swarms
and
Social
Modelling
• While
many
of
swarm
intelligence
algorithms
opt
for
simple
set
of
interac$on
rules,
collec$ve
intelligence
has
deeper
philosophical
founda$on
in
common
knowledge/beliefs
and
social
behavior.
• Thus
one
of
the
distributed
decision
making
techniques
being
developed
in
mul$-­‐agent
systems
is
vo$ng
and
consensus
techniques.
• But
…
Keynote
talk
at
ESM
2008
Conference

21. Swarms
and
Social
Modelling
• Assume
we
are
simula$ng
human
swarm.
• Now
introduce
cogni$ve
facul$es
• Now
introduce
mo$vators,
esp.
emo$ons.
• We
start
to
get
interes$ng
revela$ons
on
human
cogni$on
and
social
interac$ons
and
their
impact
on
behavior
and
decision
making.
• We
are
so
complicated
we
have
to
be
simple!
Keynote
talk
at
ESM
2008
Conference

22. Human-­‐Computer
Interac$on
• An
agent
on
every
mobile
phone?
• Assis$ve
agents
have
been
in
limited
use
or
some$me.
• The
cogni$ve
architectures
enable
user
profiling,
memorising
and
behaviour
selec$on.
• Embedded
agents
–
assisted
living
spaces
• Domes$c
robots
–
reshaped
Keynote
talk
at
ESM
2008
Conference

23. Human-­‐Computer
Interac$on
• And
of
course,
serious
gaming!
• The
applica$on
of
game
technology
to
develop
serious
applica$ons,
e.g.
medical
training.
• With
advances
of
visualiza$on
techniques
and
physics
modeling
engines,
make
the
development
of
augmented
reali$es
easier
in
which
agents
become
legi$mate
residents.
Keynote
talk
at
ESM
2008
Conference

24. Extending
the
Boundaries
• Emo$ons
–
non
verbal
communica$on
• Emo$ons
–
caring
ubiquitous
surroundings
• Conscious
machines
–
immortal
avatars
•
Companions
and
domes$c
robo$c
being
• Singularity
–
ethical
issues
Terminators
or
saviours?!
Keynote
talk
at
ESM
2008
Conference

25. Some
published
papers
• W.
Blewi],
A.
Ayesh,
R.
I.
John,
and
S.
Coupland,
"A
Millenson-­‐based
approach
to
emo$on
modelling,"
Human
System
Interac$ons,
2008
Conference
on,
2008,
pp.
491-­‐496.
• A.
Ayesh,
J.
Stokes,
and
R.
Edwards,
"Fuzzy
Individual
Model
(FIM)
for
Realis$c
Crowd
Simula$on:
Preliminary
Results,"
Fuzzy
Systems
Conference,
2007.
FUZZ-­‐
IEEE
2007.
IEEE
Interna$onal,
2007,
pp.
1-­‐5,
1098-­‐7584.
• A.
Ayesh,
"Structured
Sound
Based
Language
for
Emo$onal
Robo$c
Communica$ve
Interac$on,"
Robot
and
Human
Interac$ve
Communica$on,
2006.
ROMAN
2006.
The
15th
IEEE
Interna$onal
Symposium
on,
2006,
pp.
135-­‐140,
DOI.
• A.
Ayesh,
"Emo$onally
mo$vated
reinforcement
learning
based
controller,"
Systems,
Man
and
Cyberne$cs,
2004
IEEE
Interna$onal
Conference
on,
2004,
pp.
874-­‐878
vol.1,
1062-­‐922X.
• A.
Ayesh,
"Talking
to
one's
self:
a
li]le
architecture
for
thinking
"
in
UK
Grand
Challenges
(GC'04)
-­‐
GC5:
the
Architecture
of
Brain
and
Mind.
Newcastle,
UK:
UKCRC,
2004.
Keynote
talk
at
ESM
2008
Conference

26. Some
published
papers
• A.
Ayesh,
"Percep$on
and
Emo$on
Based
Reasoning:
A
Connec$onist
Approach,"
Informa$ca,
vol.
27,
pp.
119-­‐126,
2003.
ISSN:
0350-­‐5596,
DOI.
1854-­‐3871
• A.
Ayesh,
"Memory
Architecture
for
Argumenta$on-­‐based
Adap$ve
System,"
IASTED
Interna$onal
Conference
Applied
Informa$cs
(AI
2002),
Innsbruck,
Austria,
2002,
pp.
DOI.
• A.
Ayesh,
"Towards
Memorizing
by
Adjec$ves,"
AAAI
Fall
Symposium
on
Anchoring
Symbols
to
Sensor
Data
in
Single
and
Mul$ple
Robot
Systems,
2001,
pp.
117-­‐118,
DOI.
• A.
Ayesh,
"Thinking-­‐Learning
by
Argument,"
in
Intelligent
Agent
Technology:
Research
and
Development,
N.
Zhong,
J.
Liu,
S.
Ohsuga,
and
J.
Bradshaw,
Eds.
New
Jersey:
World
Scien$fic,
2001,
pp.
230-­‐234.ISBN:
9810247060.
• A.
Ayesh,
"Argumenta$ve
Agents-­‐based
Structure
for
Thinking-­‐Learning,"
IASTED
Interna$onal
Conference
Ar$ficial
Intelligence
and
Applica$ons
(AIA
2001),
Marbella,
Spain,
2001,
pp.
DOI.
•
Keynote
talk
at
ESM
2008
Conference

27. And
unpublished
• A.
Ayesh,
S.
Thomas,
S.
Perill,
and
C.
Joseph,
"Aesthe$cs
of
a
Robot:
Case
Study
on
AIBO
Dog
Robots
for
Buddy-­‐ing
Devices.”
• A.
Ayesh,
"Integra$ng
Physical
and
Cogni$ve
Features
in
Cogni$ve-­‐Causa$on
Maps
for
Spa$al
Reasoning.”
Keynote
talk
at
ESM
2008
Conference

28. CONCLUSION
AND
QUESTIONS
Dr.Aladdin
Ayesh
email:
aayesh@dmu.ac.uk;
URL:
www.aladdin-­‐ayesh.info
Keynote
talk
at
ESM
2008
Conference