talk given at COMSNETS 2013

1. An Ad-hoc Distributed Execution
Environment for Multi-Agent
Systems
Subhajit Sidhanta, Supratik Mukhopadhyay
Louisiana State University
COMSNETS, Bangalore, India, January 9,2013

2. Roadmap
• Motivations
• Our contributions
• Differentiation with related work
• Execution Environment Architecture
• PPDOS Design and Architecture
• Agent Architecture
• PPDOS services
• Experimental Results
• Conclusions
2

3. Motivations
Amazon Cloud
Data Center
NO INTERNET
Need to run
a multiagent
Need to
video-
run a
analytics
multiagent
application
expert
system

4. Our Contributions
• We present a novel distributed ad-hoc execution environment for
multiagent systems running on (mobile) nodes linked by an
intermittently connected network.
– We present the design and implementation of a prototype peer-to-
peer multikernel-based operating system for managing the
environment.
– The OS provides the user with a strongly-consistent view of a
single machine with a single file system and a single programming
model while allowing elasticity, availability, and scalability.

5. Multiagent Systems
• Multiagent systems have a wide range of applications in video and text
analytics, robotic control systems, etc.
– Many of these systems implement computationally expensive algorithms
(e.g., Bayesian inference)
• Existing high performance computing infrastructures such as clusters or
cloud data centers may not be accessible in remote areas with intermittent
network connectivity
5

6. Multiagent Systems and Ad-hoc Peer-to-Peer
Networks
• Peer to peer networked systems are becoming popular Examples:
Kadmelia, Chord, Skype, Kazaa, BigTable, Scalaris.
• Multiagent systems have become a dominant paradigm within AI for
deploying reasoning and analytics applications. Such applications are
compute-intensive.
• In disadvantaged networks the ad-hoc architecture is the most
suitable one. Examples: military scenarios, disaster scenarios.
• We combine peer-to-peer ad-hoc networks and multiagent systems
to develop a novel execution environment that can provide a high
performance computing infrastructure for sophisticated AI
applications in “disadvantaged” regions
• We have designed and implemented a peer to peer operating system
–PPDOS, that can leverage the computing power of such an
infrastructure.

7. Differentiation With Related Work
• Peer-to-peer grids like jalapeno [24] do not operate on disadvantaged
ad-hoc mobile networks.
– Our architecture is an asynchronous loosely coupled one
– We provide a strongly consistent view of a single machine with a
Unix-like interface and provides the user with a single agent-
oriented programming model.
• Rather than a shared memory model we use a Linda-like [16] tuple
space (a distributed key-value store) in the form of a distributed
global address space (DGAS).
• The agent-oriented model of PPDOS allows transactions: a
computation can be split into “transactions” with each transaction
executing on a location that also contains the required data
• PPDOS implements locks for mutually exclusive access of data. It
has access control for managing use of resources and flow of
information.

8. Differentiation with Related Work (Contd.)
• FOS [22] from MIT is a scalable operating system for multicore machines and
clouds
– As opposed to synchronous message passing model of FOS, PPDOS
message-passing model is asynchronous; asynchrony helps improve
scalability
– Fault-tolerance: as long as more than 50% of the nodes in the network are
up, PPDOS provides an eventually consistent view of a single machine
– Partition-tolerance: PPODS is able to operate in disadvantaged networks
where there are frequent network partitions.
– Elasticity: Within PPOS there is a group communication system that
manages (strong) consistency under agents joining and leaving the
network. As nodes leave and join, the system stabilizes to a consistent
state. This enables the system to be elastic while providing the user with
the view of a single machine
– Uniformity of time: Through an implementation of Mattern’s global virtual
time algorithm PPOS provides a uniform notion of time across the network
• Vmware [29] provides a completely virtualized set of hardware to the
operating system but it is limited by elasticity constraint and does not provide
a single machine view with a uniform programming model to the user as
opposed to PPDOS.

9. Differentiation with Related Work (Contd.)
• The Robot Operating System (ROS) [23] is based on synchronous
message passing or multicast while PPDOS supports asynchronous
communication.
• XtreemOS [33] and Barrelfish [4] cannot maintain strong consistency in
an environment where nodes continually join and leave the network.
• Other multiagent systems like Jason (Belief-Desire-Intention or BDI
agents) [2] do not provide an elaborate operating system like PPDOS
• Fawn [35] describes a high-performance execution environment created
by combining together mobile devices.
– It provides strong consistency using chain replication; we provide
strong consistency using the Paxos protocol.
– Unlike Fawn, we allow migration of agents from one node to another
while preserving state.
9

10. Architecture of Execution Environment
PPDOS SHELL
CONSOLE
agents
Distributed Key
Value Store

11. Agents
An agent is a program that receives percept streams from the
environment and uses reasoning to deduce reactions to them that are
actuated through actuators. [21]
Multiagent systems are software frameworks where autonomous
intelligent agents interact with each other to achieve a computational
goal.
goal
11

12. The Execution Environment
• Heterogeneous mobile devices communicate among each other
(peer-to-peer) asynchronously through a distributed transactional
key-value that supports transactions.
• Agents run on virtual machines deployed on these devices executing
tasks.
• The PPDOS operating system manages execution of agents
controlling access to resources and providing the user with the view
of a single eventually consistent machine and a single programming
model .
• Devices can join and leave at any time.

13. The PPDOS Operating System
• Tasks are partitioned into agents that run on peer machines.
Partitioning can be temporal as well as spatial. Agents can
asynchronously communicate among each other
– An agent schedules these agents on different machines. Currently,
scheduling is based on user policies, heuristics, and metadata
• Agents are both publishers and subscribers in contrast with the
traditional client server model where servers produce and clients
consume.

14. PPDOS: Design Principles
• Agents implement tasks and run on virtual machines deployed on the
hosts of the network. Multiple virtual machines can be tied to a single
device.
• PPDOS allows agents to migrate while execution from one host to
another.
• PPDOS allows hot swapping of one agent with another at runtime
• Agents communicate peer-to-peer asynchronously through the
distributed key-value store. A group management module within
PPDOS manages agent communication.
• Agent deployment and execution is managed by PPDOS
• PPDOS provide the user with the view of a single eventually
consistent machine with a single file system and a single
programming model and a Unix-like interface.
– PPDOS provides tolerance to network partitions and faults and
controls access to resources.
• PPDOS allows interaction with the native operating systems of the
hosts.

15. PPDOS Architecture
•.
15

16. Design
• The microkernel of PPDOS is a collection of agents that collaborate
with each other to provide operating system services
– Operating system services include file
services, deploying, unloading, and migrating
agents, time, garbage collection, etc.
• Users interact with PPDOS through a command shell deploying tasks
and accessing resources.
16

17. Agent Architecture
The state of an agent is described by three types of variables
– Monitored variables: variables in the environment that an agent monitors
(by subscribing to them)
– Controlled variables: variables in the environment that an agent controls
(publishes them)
– Internal variables: variables for storing temporary values
• An agent is reactive: its state changes only in response to changes in the
environment; and deterministic: it implements a function in the mathematical
sense
• The dependency graph between agents is acyclic.
• The body of an agent describes updates to the controlled variable as
functions of events triggered by changes in values of the monitored variables
• The body has a method that brings the agent to a quiescent state for
possible migration by the execution environment in response to a signal
17

18. Agent Structure
public class Reactive Agent implements Deterministic, Reactive {
// Declaration of controlled variables
// Declaration of Monitored variables
// Declaration of Internal variables
void init ( ) // initializes controlled variables
Reactive Agent ( ) { // constructor method
init( ) ;
for each monitored variable x
SUBSCRIBE( x , leaseX ) //Subscribes to the monitored variable x with leaseX
for each controlled variable y
REGISTER( y ) } //Publishes the controlled variable y to the key-value store
18

19. Agent structure(contd)
void Run ( ) { //Method specifying update of controlled variables
while ( true ) {
if ( migrateSignalPresent ) //checks for the migrate signal
prepareForMigration ( ) ; //prepares the agent for migration
else
{ for each monitored variable x
<valX , seqX> = read ( x ) ; //Reads the value of monitored variable x
for each controlled variabley {
y=update ( valX ) ; //calls update function with value of monitored
variable as argument
PUBLISH( y ) } //publishes the controlled variable y
for each monitored variable x
DELETE( x , state , seqX) ; } //marks the old values of the monitored
variables for garbage collection
}}}
void prepareForMigrat ion ( ) { //the Migration procedure
writeToKeyValueStore ( " ready for migration " ) ; //write the migrate signal
while ( true ) ;}}

20. Distributed Key Value Store
• Agents publish and subscribe to variables (keys) in a distributed key-value
store.
• For a distributed key-value store, we use Scalaris. It implements
transactions.
• It implements the Paxos protocol to ensure consistency and quorum
algorithms to recover from random crashes and node restarts.
• The current state of a multiagent system (i.e., the active agents, the groups
that they belong to, etc.) is maintained on Scalaris.
20

21. PPDOS Services
Basic Services: Core services provided by the microkernel.
• REGISTER: The REGISTER operation is invoked by a user agent for a controlled
variable to indicate to PPDOS that it intends to publish values of that variable.
• SUBSCRIBE: The SUBSCRIBE is invoked by a user agent to inform PPDOS that it
intends to subscribe to a monitored variable. A lease time is provided to indicate to
PPDOS that any value for that variable must be available for at least that period of
time from the instant of its publication
• PUBLISH: After updating the value of a controlled variable, an agent invokes the
PUBLISH operation to make the value available to the environment.
• DELETE: After computing the response triggered by the value of a monitored
variable, the agent uses the DELETE method to signal the operating system for
marking that value for possible garbage collection. While invoking this service, an
agent provides PPDOS its current state that is checkpointed and used later if the
agent gets disconnected, migrated, or modified
• READ: The READ operation fetches the earliest value of a monitored variable that is
yet unread by the agent provided that value has not been garbage collected.
21

22. Agent Communication
Publisher Seq = PUBLISH(Variable, Value); Continuations
Id = SUBSCRIBE(Variable, Lease);
<Value, Seq> = READ(Variable, Id);
DELETE(Variable, Id, Seq, State);
<State, Seq> = SUBSCRIBE(Variable, Lease, Id);
Subscriber
Receiver

23. PPDOS Services(contd)
Resource Management Services:
• Group Management Services: The Group Management services are the
services of the microkernel that maintain consistency in the event of nodes
joining and leaving the network.
– RDGRPSTATE: The state of a group contains its name, the current list of
its members, the current list of members who are aware of the state of the
group since the last change took place, the node(s) that initiated the last
change, and a status that is either stable or unstable; the state is
maintained as a record in Scalaris.
– JOINGROUP: The JOINGROUP operation is invoked when a node joins
a group during booting. If the group does not already exist, then it creates
the group and its state by adding itself to the list of members.
– LEAVEGROUP: The LEAVEGROUP operation is invoked when a node
leaves a group. It checks if the state of the group (as recorded in the key-
value store) is stable. If stable, it updates the state appropriately marking
the group as unstable and leaves it.
23

24. PPDOS Services(contd)
Resource Management Services:
• File Management Services: The File Management services are the PPDOS
microkernel services that manage the filesystem. The state of the file system
is maintained as a metadata on Scalaris.
• Garbage Collection: The GARBAGECOLLECTOR runs periodically; a value
of a variable is garbage collected if for each agent subscribing to that
variable either the lease expired (calculating from the time the value was
published in terms of GVT) or
– PPDOS has received an explicit delete message for that value from the
agent.
24

25. PPDOS Services (contd)
• Agent Execution and Deployment Services:
– Fetch: This service fetches byte code (of a user agent) from an external URL and
dumps it on Scalaris with an instruction to the Loader agents for deploying it on a
specified node.
– Load: Upon receiving Load instruction, a Loader agent fetches the byte code
corresponding to a user agent from Scalaris and deploys it on a virtual machine on
the specified node.
– PS: Displays the deployment record.
25

26. PPDOS Services (contd.)
• Migrate: If an agent needs to be migrated to another node (while preserving
state),
– PPDOS signals such intent to it
– in response, it responds by transitioning to a “quiescent” state, sending a
delete message to PPDOS for each of the last values that it has read
together with its current state and a “ready for migration” message.
– PPDOS then unloads the agent and loads it on the target node.
– The rejuvenated agent invokes the SUBSCRIBE operation (overloaded
version) providing PPDOS with its ID.
– PPDOS responds with the last check pointed state as obtained from the
last delete message sent before migration;
– The agent starts operation at the same state it was before migration
– A similar protocol is used for reconnecting agents after
disconnection and hot-swapping agents
26

27. OS Performance with respect to Load
The experiments were conducted on an ad-hoc network consisting of ten Dell i5
laptops (2nd Gen Intel Core i5-2467M processor, 1.6 Ghz) connected through a
wireless router.
27

28. Performance of a agents implementing the Metropolis
Hastings algorithm and exact Bayesian inference
(junction-tree decomposition algorithm)
28

29. PPDOS performance with intermittent connectivity
29

30. Conclusion
• We have presented the architecture and implementation of a novel
execution environment for running multiagent systems on ad-hoc networks
• We have presented a prototype operating system PPDOS for managing
agents.
• PPDOS provides a single machine, strongly consistent view of the
infrastructure with a single file system.
30

31. Thanks!
• Questions ?
• Contact: Subhajit Sidhanta
–email:ssidha1@tigers.lsu.edu
31

32. References
• [1]Carla T.L.L.Silva, Jaelson Castro, Patricia Azevedo
Tedesco:Requirements for Multi-agent Systems . WER 2003: 198-212
• [2] R. H. Bordini, J. F. Hubner, and M. Woolridge. Programming Multiagent
Systems in Agentspeak using Jason. John Wiley, 2007.
• [3] A. Darwiche. Modeling and Reasoning with Bayesian Networks.
Cambridge University Press, 2009.
• [4] A. Baumann, P. Barham, P. E. Dagand, T. Harris, R. Issacs, S. Peter, T.
Roscoe, A. Schupbach, and A. Singhania. The Multikernel: A new OS
architecture for scalable multicore systems. In Proceedings of SOSP 2009.
• [5] Chang, F., Dean, J., Ghemawat, S., Hsieh, W.C., Wallach, D.A., Burrows,
M. , Chandra, T. , Fikes, A. and Gruber, R.E. Bigtable: A Distributed Storage
System for Structured Data . In 7th Symposium on Operating System Design
and Implementation (OSDI’06) . (Seattle, USA) 2006 .
• [6] D. Stavens, G. Hoffmann, and S. Thrun. Online speed adaptation using
supervised learning for high-speed, off-road autonomous driving. In
Proceedings of the International Joint Conference on Artificial Intelligence
(IJCAI), Hyderabad, India, 2007.
32

33. References(contd.)
• [7] Dean, Jeffrey and Ghemawat, Sanjay (2004). “MapReduce: Simplified
Data Processing on Large Clusters". Retrieved Mar. 2, 2011.
• [8] David Wentzlaff , Charles Gruenwald, III , Nathan Beckmann , Kevin
Modzelewski , Adam Belay , Lamia Youseff , Jason Miller , Anant Agarwal,
An operating system for multicore and clouds: mechanisms and
implementation, Proceedings of the 1st ACM symposium on Cloud
computing, June 10-11, 2010, Indianapolis, Indiana, USA [doi
10.1145/1807128.1807132]
• [9] Dijkstra, EdsgerW. “EWD472: Guarded commands, non-determinacy and
formal. derivation of programs." (PDF). Retrieved February 29, 2012.
• [10] Douglas Thain, Todd Tannenbaum, and Miron Livny, “Condor and the
Grid", in Fran Berman, Anthony J.G. Hey, Geoffrey Fox, editors, Grid
Computing: Making The Global Infrastructure a Reality, John Wiley, 2003.
ISBN: 0-470-85319-0
• [11] E.E. Marinelli, “Hyrax: Cloud Computing on Mobile Devices using
MapReduce", Master Thesis, Carnegie Mellon University, 2007
• [12] fallabs, [online] 2012, http://fallabs.com/tokyocabinet/ (Accessed: 28
February 2012)
33

34. References(contd.)
• [13] Fernando, N., Loke, S.W., Rahayu, W., “Dynamic Mobile Cloud
Computing: Ad Hoc and Opportunistic Job Sharing," Utility and Cloud
Computing (UCC), 2011 Fourth IEEE International Conference on , vol. ,no. ,
pp.281-286,5-8Dec.2011
• [14] Friedemann Mattern Efficient Algorithms for Distributed Snapshots and
Global Virtual Time Approximation. Journal of Parallel and Distributed
Computing, Vol. 18, No. 4, pp. 423-434, 1993 Abstract, BibTeX, Paper (.pdf)
(Reprinted in: Z. Yang, T.A. arsland (Eds.), “Global States and Time in
Distributed Systems", IEEE, 1994, pp. 27-36)
• [15] G Kirby, A Dearle, A Macdonald, A Fernandes . “An Approach to Adhoc
Cloud Computing" . Technical report:arxiv 1002.4738. Feb, 2010
• [16] Gelernter, David. “Generative communication in Linda". ACM
Transactions on Programming Languages and Systems, volume 7, number
1, January 1985.
• [17] Harold Trease, Tim Carlson, Ryan Moony, Robert Farber, and Lynn
Trease. 2007. “Unstructured data analysis of streaming video using parallel,
high-throughput algorithms". In Proceedings of the Ninth IASTED
International Conference on Signal and Image Processing (SIP ’07), Rui J. P.
de Figueiredo (Ed.). ACTA Press, Anaheim, CA, USA, 305-310.
• [18] Hastings, W.K. (1970). “Monte Carlo Sampling Methods Using Markov
Chains and Their Applications". Biometrika 57 (1): 97U˝ 109.
doi:10.1093/biomet/57.1.97. JSTOR 2334940. Zbl 0219.65008.34

35. References(contd.)
• [19] Lamport, Leslie (May 1998). "The Part-Time Parliament". ACM
Transactions on Computer Systems 16 (2): 133U˝ 169.
doi:10.1145/279227.279229. Retrieved 2012-03-02.
• [20] M. Berna-Koes, I. Nourbakhsh and K. Sycara, “Communication
Efficiency in Multiagent Systems," Proc. Int"l Conf. Robotics and Automation
(ICRA 04), IEEE Press, 2004.
• [21] Michael J. Wooldridge: An Introduction to Multiagent Systems (2. ed.).
Wiley 2009: I-XXII, 1-461
• [22] Modzelewski, Kevin, Miller, Jason, Belay, Adam, Beckmann, Nathan,
Gruenwald, Charles, III, Wentzlaff, David . . . Agarwal, Anant .(2009) . A
Unified Operating System for Clouds and Manycore: fos, DSpace@MIT:
Massachusetts Institute of Technology . Retrieved at July 27, 2011, from the
website temoa : Open Educational Resources (OER) Portal at
http://www.temoa.info/node/60356
• [23] Morgan Quigley, Brian Gerkey, Ken Conley, Josh Faust, Tully Foote,
Jeremy Leibs, Eric Berger, Rob Wheeler, Andrew Ng . “ROS: an opensource
Robot Operating System". Retrieved 3 April 2010
• [24] Niklas Therning, [online] 2012, http://jalapeno.therning.org/report.pdf
(Accessed: 27 February 2012)
35

36. References(contd.)
• [25] Pepitone Julianne, “The NIST definitionn of Cloud Computing,“ (NIST),
[online] 2011, http://csrc.nist.gov/publications/nistpubs/800-145/ SP800-
145.pdf (Accessed: 16 February 2012)
• [26] Pepitone Julianne, “IBM’s Jeopardy supercomputer beats humans in
practice bout," (CNN), [online] 2011,
http://money.cnn.com/2011/01/13/technology/ibm_jeopardy_watson/index.ht
ml (Accessed: 16 February 2012)
• [27] Stan Franklin , Art Graesser, Is it an agent, or Just a Program?: A
Taxonomy for Autonomous agents, Proceedings of the Workshop on
Intelligent agents III, agent Theories, Architectures, and Languages, p . 21-
35, August 12-13, 1996
• [28] Thorsten Schütt , Florian Schintke , Alexander Reinefeld, Scalaris:
reliable transactional p2p key/value store, Proceedings of the 7th ACM
SIGPLAN workshop on ERLANG, September 27-27, 2008, Victoria, BC,
Canada [doi 10.1145/1411273.1411280]
• [29] Vmware.com, [online] 2012, http://www.vmware.com/pdf/virtualization.
pdf (Accessed: 27 February 2012)
• [30] Wikipedia, [online] 2012, http://en.wikipedia.org/wiki/Cray (Accessed: 27
February 2012)
• [31] Wikipedia, [online] 2012, http://en.wikipedia.org/wiki/Grid_computing
(Accessed: 28 February 2012) 36

37. References(contd.)
• [32] Wikipedia, [online] 2012, http://en.wikipedia.org/wiki/Apache_Hadoop
(Accessed: 28 February 2012) Wikipedia, [online] 2012, http://en.
wikipedia.org/wiki/Linda_(coordination_language) (Accessed: 28 February
2012) Wikipedia, [online] 2012, http://en.wikipedia.org/wiki/Paxos_
(computer_science) (Accessed: 28 February 2012)
• [33] XtreemOS grid checkpointing architecture, J . Mehnert-Spahn, M
.Schoettner, D . Margery, C . Morin . In IEEE International Symposium on
Cluster Computing and the Grid (CCGRID 2008), poster session, Lyon,
France, May 2007 .
• [34] Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin and J.
Hellerstein. GraphLab: A New Framework for Parallel Machine Learning. In
the 26th Conference on Uncertainty in Artificial Intelligence (UAI), Catalina
Island, USA, 2010. arxiv presentation
• [35] D. G. Andersen, J. Franklin, M. Kaminsky, A. Phanishayee, Lawrence
Tan, and Vijay Vasudevan. FAWN: A Fast Array of Wimpy Nodes. In Proc.
22nd ACM Symposium on Operating Systems Principles (SOSP), 2009.
• [36] R. Bharadwaj and S. Mukhopadhyay. A Formal Approach to Developing
Reliable Event-Driven Service-Oriented Systems. In Proceedings of IEEE
COMPSAC 2008
• [37] S. Sidhanta and S. Mukhopadhyay. Managing a Cloud for Multi-agent
37
Systems on Ad-Hoc Networks. In Proceedings of IEEE Cloud 2012, 996– 997