Distributed computing bsics

Distributed Systems
Deepak John
Department Of Computer Applications
SJCET-Pala

Distributed System Definitions
2
A system in which hardware or software components located at networked
computers communicate and coordinate their actions only by message
passing(synchronous or asynchronous ).
Consequences
Concurrency of components: Components in distributed systems are executed in
concurrent processes. Integrity of the system may be violated if concurrent updates are
not coordinated.
No global clock.
Independent failures.
A collection of independent computers that appear to the users of the
system as a single computer.
2

Centralized vs. Distributed Computing
mainframe computer
workstation
network host
network link
terminal
centralized computing
distributed computing
3

Why Distributed Systems
4
The need to share data across remote geographies
Functional distribution: computers have different functional capabilities
(i.e., sharing of resources with specific functionalities).
Load distribution / balancing: assign tasks to processors such that the
overall system performance is optimized.
Replication of processing power: independent processors working on the
same task.
Use of heterogeneous components
•Less resource-demanding sub-tasks run on less powerful computers
More efficient use of resources
Physical separation: systems that rely on the fact that computers are
physically separated.
Economics: collections of microprocessors offer a better
price/performance ratio than large mainframes.
mainframes: 10 times faster, 1000 times as expensive
4

Administrative/functional issues
Scalability
The system can be extended by adding more components (i.e.,
WWW)
Resilience to failures
If one component fails, others can proceed with work on the task
5

Distributed Systems Examples-The Internet
6
The Internet is a vast interconnected collection of computer
networks of many types.
Its design enabling a program running anywhere to address
messages to programs anywhere else.
Allowing its users to make use of many services as: WWW, E-Mail,
Web hosting, and File transfer.
Small organizations and individual users can to access internet
services through Internet Service Providers (ISPs).
Independent intranets are linked together by high transmission
capacity circuits called backbones.
6

7
intranet
ISP
desktop computer:
backbone
satellite link
server:
network link:
A typical portion of the Internet
7

Distributed Systems Examples-Intranets
8
• An Intranet is a portion of the internet that is administrated
separately and its local security policies are enforced by a
configured boundary.
• Composed of several local area networks (LANs) linked by
backbone connections to allow its users to access the provided
services.
• Connected to the Internet via a router.
• Many organization protect their own services from unauthorized
use by filtering incoming and outgoing messages using a firewall.
8

9
the rest of
email server
Web server
Desktop
computers
File server
router/firewall
print and other servers
other servers
print
Local area
network
email server
the Internet
A typical Intranet
9

Distributed Systems Examples-
Mobile and Ubiquitous Computing
10
The portability of many computing devices and the ability to connect
to networks in different places makes mobile computing possible.
Mobile computing is the performance of computing tasks while the
users are on the move and away from their residence intranet but still
provided with access to resources via the devices they carry with
them.
Ubiquitous computing is the harnessing of many small cheap
computational devices that are present in user’s physical
environments.
10

11
Laptop
Mobile
Printer
Camera
Internet
Host intranet Home intranet
WAP
Wireless LAN
phone
gateway
Host site
Portable and handheld devices in a distributed system
11

Resource Sharing
12
Resources in a distributed system are:
Encapsulated within computers.
Can only accessed from other computers by communication.
Managed by a program that offers a communication interface.
The term service is used for a distinct part of a computer system which
manages a collection of related resources and presents their functionality
to users and applications via a well-defined set of operations.
Server refers to a running program on a networked computer that accepts
request messages from programs, clients, running on other computers to
perform a service and responds appropriately by reply messages.
A complete interaction between a client and a server is called a remote
invocation.
12

The World Wide Web (WWW)
13
WWW is an open client-server architecture implemented on top of the
Internet.
Users use the Web through available web browsers to retrieve and view
documents of many types and interact with unlimited set of services.
Web is based on three main standard technological components:
The HyperText Markup Language (HTML) specifies the contents and
layout of pages.
Uniform Resource Locators (URLs) identify documents and other
resources accessible via the web.
The HyperText Transfer Protocol (HTTP) determines the standard rules
for interaction between browsers and web servers to fetch documents
and other resources.
13

14
Internet
BrowsersWeb servers
www.google.com
www.cdk3.net
www.w3c.org
Protocols
Activity.html
http://www.w3c.org/Protocols/Activity.html
http://www.google.com/search?q=kindberg
http://www.cdk3.net/
File system of
www.w3c.org
Web servers and web browsers
14

World Wide Web Components-HTML
15
HTML is used to specify:
The text and images that make up the contents of a web page,
How web page components are formatted for presentation to the user, and
Links and which resources are associated with them.
HTML is produced using a standard text editor or an HTML tool.
HTML uses tags to specify content:
<IMG SRC = “http://www.cdk3.com/WebExample/Images/earth.jpg”>
<P> Welcome to Earth! Visitor may also be interested in taking a look at the
<A HREF = “http://www.cdk3.com/WebExample/Moon.html”> Moon </A> </P>
………………………
HTML text is stored in a file accessible by a web server.
15

World Wide Web Components-URLs
16
Browsers examine URLs in order to fetch the corresponding resources
from web servers.
A URL is typed by the user into the browser or selected from their
bookmarks.
Also, the browser looks up the corresponding URL when the user clicks
on a link or fetches a resource embedded in a web page such an image.
Every URL in its full form has two top-level components:
scheme : scheme-specific-location
URL examples:
http://www.cdk3.net/
http://www.google.com/search?q=kindberg.
ftp://ftp.downloadIt.com/software/aProg.exe
mailto:metwally@ccis.ksu.edu.sa
16

World Wide Web Components-HTTP
17
HTTP defines the ways in which browsers and any other types of client
interact with web server.
HTTP is a request-reply protocol:
The client sends a request message to the server containing the URL of
the required resource,
Then the server looks up the pathname and if it exists, sends back the
file’s contents in a reply message to the client.
HTTP allows the client to request one resource at a time.
17

Distributed Systems Challenges
18
A number of challenges are recognized in the design of distributed
systems.
These challenges have been tackled with varying degrees of success in
existing systems.
The list below gives some indication of measures that are employed to
meet each challenge:
Heterogeneity : standards and protocols; middleware; virtual machine;
Openness: publication of services; notification of interfaces;
Security: firewalls; encryption;
Scalability: replication; caching; multiple servers;
Failure Handling: failure tolerance; recover/roll-back; redundancy;
Concurrency: concurrency control to ensure data consistency;
Transparency: middleware; location transparent naming; anonymity
18

Distributed Systems Challenges-Heterogeneity
19
The internet enables users to access services over a variety and
difference:
Networks.
Operating systems.
Implementations by different developers.
The middleware software layers provide a programming abstraction as
well as masking the heterogeneity of the underlying networks, hardware,
operating systems and programming languages.
Virtual machines approach provides a way of making code executable on
any hardware – mobile code.
Computer hardware.
Programming languages.
19

Distributed Systems Challenges-Openness
20
The openness of distributed systems is determined by the degree to
which new resource-sharing services can be added and be available for
use by variety of client programs (services publication).
The specification and documentation of key software interfaces of the
system components must be available(interfaces notification).
Systems that are designed to support resource sharing in this way are
termed open distributed systems to emphasize the fact that they are
extensible.
20

Distributed Systems Challenges-Security
21
Security for information resources has three components:
Confidentiality: protection against disclosure to unauthorized
individuals.
Integrity: protection against alteration or corruption.
Availability: protection against interference with the means to access the
resources.
A firewall can be used to form a barrier around an intranet to protect it
from outside users but does not deal with ensuring the appropriate use of
resources by users within the intranet.
Encryption can be used to provide adequate protection of shared resources
and to keep sensitive information secret when is transmitted in messages
over the internet.
21

Distributed Systems Challenges-Scalability
22
A system is described as scalable if will remain effective when there is a
significant increase in the number of resources and the number of users.
The design of scalable dist. systems presents many challenges:
Controlling the cost of physical resources.
Controlling the performance loss.
Preventing software resources running out.
Avoiding performance bottlenecks.
22

Distributed Systems Challenges-Failure Handling
23
Failures in a distributed system are partial, therefore the handling of it is
particularly difficult.
Some failures can be detected but others are difficult or impossible to
detect it.
Some detected failures can be hidden or made less severe.
Recovery from failures involves the design of software so that the state of
permanent data can be rolled back after a server has crashed.
Services can be made to tolerate failures by the use of redundant
components:
At least two different routes between any two routers in the internet.
Databases and domain name tables must be replicated in several
servers.
23

Distributed Systems Challenges-Concurrency
24
Several clients in a distributed system can access a shared resource at the
same time
Any object that represents a shared resource in a distributed system (a
programmer that implement it) must be responsible for ensuring that it
operate correctly in a concurrent environment.
Operations of objects in a concurrent environment must be synchronized
in a way that its data remains consistent.
The synchronization of an object operations can be achieved by standard
techniques such as semaphores.
24

Distributed Systems Challenges-Transparency
25
Access transparency: enables local and remote resources to be accessed using
identical operations.
Location transparency: enables resources to be accessed without knowledge of their
location.
Concurrency transparency: enables several processes to operate concurrently using
shared resources without interference between them.
Replication transparency: enables multiple instances of resources to be used to
increase reliability and performance without knowledge of the replicas by users or
application programmers.
Failure transparency: enables the concealment of faults, allowing users and
application programs to complete their tasks despite the failure of hardware or
software components.
Mobility transparency: allows the movement of resources and clients within a
system without affecting the operation of users or programs.
Performance transparency: allows the system to be reconfigured to improve
performance as loads vary.
Scaling transparency: allows the system and applications to expand in scale without
change to the system structure or the application algorithms.
Access and location transparency together provide network transparency.
25

System Models
26
Architectural model defines the way in which the components of the
system are placed and how they interact with one another and the way
in which they are mapped onto the underlying network of computers.
Fundamental models:
Interaction model deals with communication details among the
components and their timing and performance details.
Failure model gives specification of faults and defines reliable
communication and correct processes.
Security model specifies possible threats and defines the concept of
secure channels.
26

Architectural Model
27
Concerned with placement of its parts and relationship among them.
Example: client-server model, peer-to-peer model
Abstracts the functions of the individual components.
Defines patterns for distribution of data and workload.
Defines patterns of communication among the components.
Example: Definition of server process, client process and peer process
and protocols for communication among processes; definition
client/server model and its variations.
27

Software and hardware service layers in distributed systems
28
Applications, services
Computer and network hardware
Platform
Operating system
Middleware
28

Middleware
29
Layer of software whose purpose is to mask the heterogeneity and to
provide a convenient programming model for application programmers.
Middleware supports such abstractions as remote method invocation,
group communications, event notification, replication of shared data,
real-time data streaming.
Examples: Java RMI, grid software (Globus, Open grid Services), Web
services.
Platform
Implemented independently in each computer.
Provide services to the layers above them
29

Clients invoke individual servers
30
Server
Client
Client
invocation
result
Server
invocation
result
Process:
Key:
Computer:
EX: browser,
web client
EX: Web server
EX: 1. File server,
2. Web crawler
Client-Server
•Servers provide services
•Clients access services
30

Peer to Peer
31
•potentially large number of participants.
•communication pattern varies over time
31

A service provided by multiple servers
32
Server
Server
Server
Service
Client
Client
32

Web proxy server and caches
33
Client
Proxy
Web
server
Web
server
server
Client
Cache: local copies of remote objects for faster access
•Browser cache
•Proxy server
Additional roles: filtering, firewall
33

Mobile code and web applets
34
a) client request results in the downloading of applet code
Web
server
Client
Web
serverApplet
Applet code
Client
b) client interacts with the applet
EX: Code streaming; mobile code
34

Mobile agents
35
•running program is moved from node to node (with both code and data).
Thin Clients
•Client performs just I/O, no computation
•Basically a display with keyboard
•Remote computation, storage
35

Interfaces and Objects
36
At the programming level
•Need standard interfaces
Access/provide services/objects
RPC/RMI
36

Design Requirements for Distributed Architectures
37

39
two variants of interaction: synchronous and asynchronous
synchronous distributed systems
• lower and upper bounds for execution of a step
•message transmission in bounded time
•clock drift rate is bounded
•failures can be detected when bounds are exceeded
•accomplished by allocating sufficient resources
asynchronous distributed systems
•no bounds on process speed, message delay, clock drift rate
•failures are harder to detect
•performance can't be guaranteed
39

40
•Event ordering
Relative ordering might be more important than exact time
logical clocks--ordering events without physical clocks.
logical time: Assign a number L to each event, such that L(e1) < L(e2) if
e1 happens before e2
40

Failure Model
41
Failure of processes or communication channels
41

Omission and arbitrary failures
42
Class of failure Affects Description
Fail-stop Process Process halts and remains halted. Other processes may
detect this state.
Crash Process Process halts and remains halted. Other processes may
not be able to detect this state.
Omission Channel A message inserted in an outgoing message buffer never
arrives at the other end’s incoming message buffer.
Send-omission Process A process completes a send, but the message is not put
in its outgoing message buffer.
Receive-omission Process A message is put in a process’s incoming message
buffer, but that process does not receive it.
Arbitrary
(Byzantine)
Process or
channel
Process/channel exhibits arbitrary behaviour: it may
send/transmit arbitrary messages at arbitrary times,
commit omissions; a process may stop or take an
incorrect step.
42

Timing failures
43
Class of Failure Affects Description
Clock Process Process’s local clock exceeds the bounds on its
rate of drift from real time.
Performance Process Process exceeds the bounds on the interval
between two steps.
Performance Channel A message’s transmission takes longer than the
stated bound.
43

• masking failures
44
hiding--use another server to respond,
converting it into more acceptable--drop the packet if
it is corrupted
• reliable one-to-one communication
validity: eventually delivered
integrity: content not corrupted or duplicated
44

Security Model
45
• securing processes and channels against unauthorized access
• Protecting objects:
authorization (access rights to principals)
authentication (identity of parties/principals)
45

46
• Threat to processes & communication channels
46

• Introduction
• The API for the Internet protocols
• External data representation and marshalling
• Client-Server communication
• Group communication
• Case study: interprocess communication in UNIX
• Summary
Interprocess Communication

Distributed systems are built around communication.
Because communications are critical to distributed systems, A key form of
communications is inter process communications, based on low-level
message passing over the network.
Inter Process Communication
49

Middleware layer
TCP ( UDP) from a programmers point of view
TCP : two way stream
UDP : datagram, message passing
Java interface(Provides both datagram and stream communication
primitives/interfaces – building blocks for communication protocols),
UNIX Socket
Marshalling(process of converting the data or the objects into a byte-
stream) and demarshalling
data form translation
Client-Server communication
Request-Reply protocols
Java RMI, RPC
Introduction
50

Middleware layers
Applications
Middleware
layersRequest reply protocol
External data representation
Operating System
RMI, RPC and events
Programming model
Interprocess communication

The API for the Internet protocols
Characteristics of IPC – message passing using send/receive facilities for
sync and addressing in distributed programs
Use of sockets as API for UDP and TCP implementation
Synchronous
Queues at remote sites are established for message placement by clients
(sender). The local process (at remote site) dequeues the message on
arrival
If synchronous, both the sender and receiver must synchronize on each
message, i.e., both send and receive invocations are blocking-until
52

Asynchronous communication
Send from client is non-blocking and proceeds in parallel with local
operations
Receive could be non-blocking (requiring a background buffer) AND if
blocking, perhaps, remote process needs the message, then the process
must wait on it
Having both sync/async is advantageous, e.g., one thread of a process can
do blocked-receive while other thread of same process perform non-block
receive or are active – simplifies synchronization. In general non-
blocking-receive is simple but complex to implement due to messages
arriving out-of-order in the background buffer
53

Message destinations:
Typically: send(IP, port#, buffer) – a many-to-one (many senders to a single
receiving port), except multicast, which is many-to-group.
Possibility: receiving process can have many ports for different message types
Server processes usually publish their service-ports for clients
Reliability
validity: messages are guaranteed to be delivered despite a reasonable
number of packets being dropped or lost
Integrity: messages arrive uncorrupted and without duplication
Ordering
the messages be delivered in sender order
54

Endpoint for communication between processes
Both forms of communication (UDP and TCP ) use the socket abstraction
Each computer has 216 possible ports available to local processes for
receiving messages
a process cannot share ports with other processes on the same computer
Socket
message
agreed port
any port
socketsocket
Internet address = 138.37.88.249Internet address = 138.37.94.248
other ports
client server
55

App 3App 2App 1
Sockets
The application level API to TCP & UDP
Allows users to write application
“protocol objects” which sit on top of
TCP and UDP.
Execution of TCP/UDP/IP are in kernel
space. Sockets provide the bridge to user
space.
In Java provides 3 types of sockets
DatagramSocket – for UDP
ServerSocket – for TCP server
Socket – endpoint of a TCP stream
IP
TCP UDP
socket socket socket
56

Java API for IPs
For either TCP or UDP, Java provides an InetAddress class, which
contains a method: getByName(DNS) for obtaining IP addresses,
irrespective of the number of address bits (32 bits for IPv4 or 128 bits
for IPv6) by simply passing the DNS hostname. For example, a user
Java code invokes:
InetAddress aComputer = InetAddress.getByName(“nsfcopire.spsu.edu”);
The class encapsulates the details of representing the IP address
57

UDP datagram communication
UDP Datagram communication
Steps:
Client finds an available port for UDP connection
Client binds the port to local IP (obtained from
InetAddress.getByName(DNS) )
Server finds a designated port, publicizes it to clients, and binds it to
local IP
Server process issues a receive method and gets the IP and port # of
sender (client) along with the message
58

Issues
Message size – set to 8KByte for most, general protocol support 216 bytes,
possible truncation if receiver buffer is smaller than message size
Blocking – send is non-blocking and operation returns if message gets pass
the UDP and IP layers; receive is blocking.
Timeouts – reasonably large time interval set on receiver sockets to avoid
indefinite blocking
Receive from any – no specification of sources (senders), typically many-to-
one, but one-to-one is possible by a designated send-receive socket (know by
both C/S)
59

Failure model
omission failure: message be dropped due to checksum error or no
buffer space at sender side or receiver side
ordering: message be delivered out of sender order
application maintains the reliability of UDP communication
channel by itself (ACK)
60

DatagramPacket Methods
It provides a constructor to make an array of bytes comprising
Message content ,Length of message,Internet address ,Local port number
getAddress():Returns the IP address of the machine to which this
datagram is being sent or from which the datagram was received.
getData():Returns the data received or the data to be sent.
getLength():Returns the length of the data to be sent or the length of
the data received.
getPort():Returns the port number on the remote host to which this
datagram is being sent or from which the datagram was received.
It provides another similar constructor for receiving a message.
Java API for UDP datagrams
61

DatagramSocket Methods
This class supports sockets for sending and receiving UDP datagram.
getLocalAddress():Gets the local address to which the socket is bound.
getLocalPort():Returns the port number on the local host to which this
socket is bound.
getSoTimeout():Retrive setting for SO_TIMEOUT.
receive(DatagramPacket):Receives a datagram packet from this socket.
send(DatagramPacket):Sends a datagram packet from this socket.

Datagram Sockets
Creating a datagram socket
DatagramSocket soc = new DatagramSocket(port-no.)
Creating a datagram packect
DatagramPacket p = new DatagramPacket(byte[] data, int len, InetAddress dest,
int dest-port);
Sending a packet
soc.send( p )
Receiving a packet
q = new DatagramPacket(buff, len);
soc.receive( q );
63

UDP client sends a message to the server and gets a reply
import java.net.*;
import java.io.*;
public class UDPClient{
public static void main(String args[]){
// args give message contents and server hostname
DatagramSocket aSocket = null;
try {
aSocket = new DatagramSocket();
byte [] m = args[0].getBytes();
InetAddress aHost = InetAddress.getByName(args[1]);
int serverPort = 6789;
DatagramPacket request = new DatagramPacket(m, args[0].length(), aHost, serverPort);
aSocket.send(request);
byte[] buffer = new byte[1000];
DatagramPacket reply = new DatagramPacket(buffer, buffer.length);
aSocket.receive(reply);
System.out.println("Reply: " + new String(reply.getData()));
}catch (SocketException e){System.out.println("Socket: " + e.getMessage());
}catch (IOException e){System.out.println("IO: " + e.getMessage());}
}finally {if(aSocket != null) aSocket.close();}
}
}
64

UDP server repeatedly receives a request and sends it back to the
client
import java.net.*;
import java.io.*;
public class UDPServer{
DatagramSocket aSocket = null;
try{
aSocket = new DatagramSocket(6789);
while(true){
DatagramPacket request = new DatagramPacket(buffer, buffer.length);
aSocket.receive(request);
DatagramPacket reply = new DatagramPacket(request.getData(),
request.getLength(), request.getAddress(), request.getPort());
aSocket.send(reply);
}
}catch (SocketException e){System.out.println("Socket: " + e.getMessage());
}catch (IOException e) {System.out.println("IO: " + e.getMessage());}
}finally {if(aSocket != null) aSocket.close();}
} } 65

The API to the TCP
provide the abstraction of a stream of bytes to which data may be
written and from which data may be read
Hidden network characteristics
message sizes (how much data to write/read)
lost messages (ACK)
flow control (match the speed)
message duplication and ordering (identifier with each IP Packet)
message destinations (establish connection)
Once connection is established, no need of address & ports
TCP stream communication
66

While connection establishing: client/server
Client :Connect request, Server :Accept request, becomes Peers
Client: creates stream socket, bind to a port, and makes a connect
request to server
Server: creates a listening socket, bind to a server port, and waits for
clients
Use of TCP
Many services that run over TCP connections, with reserved port
number are:
HTTP (Hypertext Transfer Protocol)
FTP (File Transfer Protocol)
Telnet
SMTP (Simple Mail Transfer Protocol)

issues related to stream communication
Matching of data items: agree to the contents of the transmitted data
Blocking: send blocked until the data is written in the receiver’s buffer,
receive blocked until the data in the local buffer becomes available
Threads: server create a new thread for every connection
failure model
integrity and validity have been achieved by checksum, sequence
number, timeout and retransmission in TCP protocol
connection could be broken due to unknown failures
Can’t distinguish between network failure and the destination process
failure
Can’t tell whether its recent messages have been received or not
68

ServerSocket
Intended for use by a server to create a socket at a server port for listening
connect request from clients.
Has an accept method that is listen for connect requests from clients
Socket
Used by a pair of processes with a stream connection.
Has a constructor used by the client to create its stream socket and send a
connect request to the specified remote server.
Provides getInputStream and getOutputStream methods for accessing the two
input and output streams associated with a socket.
read and write methods are used for bytes reading from and writing to
InputStream and OutputStream respectively.
Java API for TCP Streams
69

TCP byte stream sockets
Server socket – waits for connections
soc = new ServerSocket( port-no );
Socket newsoc = soc.accept( );
Client socket – connects to server
client = new Socket(server-addr, server-port);
When connected, get streams
InputStream in = client.getInputStream( );
OutputStream out = client.getOutputStream( )
Server side
Server
socket
Client Side
client
socket
1. Contact server
New
socket
2. Create new
Stream socket
3. Establish stream
communication
70

TCP client makes connection to server, sends request and receives reply
import java.net.*;
import java.io.*;
public class TCPClient {
public static void main (String args[]) {
// arguments supply message and hostname of destination
Socket s = null;
try{
s = new Socket(args[1], serverPort);
DataInputStream in = new DataInputStream( s.getInputStream());
DataOutputStream out =
new DataOutputStream( s.getOutputStream());
out.writeUTF(args[0]); // UTF is a string encoding see Sn 4.3
String data = in.readUTF();
System.out.println("Received: "+ data) ;
}catch (UnknownHostException e){
System.out.println("Sock:"+e.getMessage());
}catch (EOFException e){System.out.println("EOF:"+e.getMessage());
}catch (IOException e){System.out.println("IO:"+e.getMessage());}
}finally {if(s!=null) try {s.close();}catch (IOException
e){System.out.println("close:"+e.getMessage());}}
}
}

TCP server makes a connection for each client and then echoes the client’s
request
import java.net.*;
import java.io.*;
public class TCPServer {
public static void main (String args[]) {
try{
ServerSocket listenSocket = new ServerSocket(serverPort);
while(true) {
Socket clientSocket = listenSocket.accept();
Connection c = new Connection(clientSocket);
}
} catch(IOException e) {System.out.println("Listen :"+e.getMessage());}
}
}
// this figure continues on the next slide

class Connection extends Thread {
DataInputStream in;
DataOutputStream out;
Socket clientSocket;
public Connection (Socket aClientSocket) {
try {
clientSocket = aClientSocket;
in = new DataInputStream( clientSocket.getInputStream());
out =new DataOutputStream( clientSocket.getOutputStream());
this.start();
} catch(IOException e) {System.out.println("Connection:"+e.getMessage());}
}
public void run(){
try { // an echo server
String data = in.readUTF();
out.writeUTF(data);
} catch(EOFException e) {System.out.println("EOF:"+e.getMessage());
} catch(IOException e) {System.out.println("IO:"+e.getMessage());}
} finally{ try {clientSocket.close();}catch (IOException e){/*close failed*/}}
}
}

The information stored in running programs is represented as data
structures, whereas the information in messages consists of sequences of
bytes.
External data representation
An agreed standard for the representation of data structures and
primitive values
Marshalling (unmarshalling)
The process of taking a collection of data items and assembling them
into a form suitable for transmission in a message
Usage: for data transmission or storing in files
Two alternative approaches
CORBA’s(Common Object Request Broker Architecture)common
data representation , Java’s object serialization and XML
External data representation and marshalling
introduction
74

Concerned with an external representation for the structured and primitives
types.
15 primitive types
Short (16bit), long(32bit), unsigned short, unsigned long, float, char,
Constructed types
Types that composed by several primitive types
Marshaling and unmarshalling operations are generated automatically from the
specification of data items transmitted in a message.
• The type definitions of data structures and primitive data items are
described in CORBA Interface Definition Language (IDL).
• CORBA interface compiler generates appropriate marshaling and
unmarshalling operations from the type definitions of remote methods
arguments and results
CORBA’s Common Data Representation (CDR)
75

CORBA CDR for constructed types
Type Representation
sequence length (unsigned long) follow ed by elements in order
string length (unsigned long) follow ed by charactersin order (can also
can have wide characters)
array array elements in order (no length specified because it is fixed)
struct in the order of declaration of the components
enumerated unsigned long (the values are specified by the order declared)
union type tag followed by th e sele cted member
76

CORBA CDR message
Person struct with value: {‘Smith’, ‘London’, 1934}
0–3
4–7
8–11
12–15
16–19
20-23
24–27
5
"Smit"
"h___"
6
"Lond"
"on__"
1934
index in
sequence of bytes 4 bytes
notes
on representation
length of string
‘Smith’
length of string
‘London’
unsigned long
77
The type of a data item is not given with the data representation in
message
It is assumed that the sender and recipient have common knowledge of
the order and types of the data items in a message.
•shows a message in CORBA CDR that contains the three fields of a struct
whose respective types are string, string, and unsigned long.
Struct Person {
string name;
string place;
long year;
};

Serialization (deserialization)
The activity of flattening an object or a connected set of objects into a
serial form that is suitable for storing on the disk or transmitting in a
message
Include information about the class of each object
Handles: references to other objects are serialized as handles
Serialization of a specific object is done by creating an instance of the
class ObjectOutputStream and invoking its writeObject method with the
object name as rgument.
ObjectInputStream class is opened on a stream of data to deserialize an
object from that stream using the readObject method.
Java object serialization
78

•Only class name and object’s data is saved
•If that data is an object, it is also saved
•Each object is given a serial number
•If an object has already been saved then only the serial number is saved
•Methods are not saved
•Static information not saved

Indication of Java serialized form
Serialized values
Person
3
1934
8-byte version number
int year
5 Smith
java.lang.String
name:
6 London
h0
java.lang.String
place:
h1
Explanation
class name, version number
number, type and name of
instance variables
values of instance variables
Public class Person implements Serializable {
private String name;
private String place;
private int year;
public Person (String aName, String aPlace, int aYear){
name = aName;
place = aPlace;
year = aYear;
}
// followed by methods for accessing the instance variables
}
Person p = new Person(“Smith”, “London”, 1934);
80

Representation of a remote object reference
Internet address port number time object number interface of
remote object
32 bits 32 bits 32 bits 32 bits
Remote object references are needed when a client invokes an object that is
located on a remote server.
A remote object reference is passed in the invocation message to specify
which object is to be invoked.
Remote object references must be unique over space and time.
generic format for remote object references is shown
81

XML
•Extensible markup language (XML)
•User-defined tags (vs. HTML has a fixed set of tags)
Ex: Person struct in XML
Tag names: person id, name, place, year
Element: <name>Smith</name>
Attribute: id="123456789” of person
Binary data need to be converted to characters (base64)
<person id="123456789">
<name>Smith</name>
<place>London</place>
<year>1934</year>

</person >

XML namespace
•Name clashes within an application
•Namespaces: a set of names for a collection of element types and attributes
•xmlns: xml namespace
•pers: name of the name space (used as a prefix)
• http://www.cdk4.net/person :location of schema
<person pers:id="123456789" xmlns:pers = "http://www.cdk4.net/person">
<pers:name> Smith </pers:name>
<pers:place> London </pers:place >
<pers:year> 1934 </pers:year>
</person>

XML schema
•Defines elements and attributes
•xsd: namespace for xml schema definition
<xsd:schema xmlns:xsd = URL of XML schema definitions >
<xsd:element name= "person" type ="personType" />
<xsd:complexType name="personType">
<xsd:sequence>
<xsd:element name = "name" type="xs:string"/>
<xsd:element name = "place" type="xs:string"/>
<xsd:element name = "year" type="xs:positiveInteger"/>
</xsd:sequence>
<xsd:attribute name= "id" type = "xs:positiveInteger"/>
</xsd:complexType>
</xsd:schema>

Request-reply protocols are designed to support client-server
communication with the following characteristics:
• Synchronous in normal cases because the client process blocks until the
reply arrives from the server.
• Acknowledgments are not required since requests are followed by
replies.
• Flow control is not needed due to small amounts of data transferred
• Connection establishment involves only two extra pair of messages.
• Usually implemented over UDP datagrames but can also applied over
TCP streams.
85
Client –Server Communication
Request-reply protocol

86
Request-reply protocol is based on a triple of communication primitives:
doOperation.
getRequest.
sendReply.
Request
ServerClient
doOperation
(wait)
(continuation)
Reply
message
getRequest
execute method
message
select object
sendReply

doOperation:
Used by clients to send a request message for invoking remote operations.
Its arguments specify the remote object reference, the method to be invoked
and the arguments of that method.
Its result is a remote method reply.
getRequest:
Used by a server process to acquire service request messages.
sendReply:
Used by a server process to send the reply messages to clients
Its arguments are the reply, IP address and port of designated client.

The structure of a request or a reply message as follow:
Message type: indicate whether the message is a request or a reply.
Message ID: contain a request message identifier generated by the client
and the server refer to it into the corresponding reply message to enable
the client to check that the reply is the result of the current request.
Object reference: the reference of remote object.
Method ID: the identifier for the method to be invoked.
Arguments of the invoked method
messageType
requestId
objectReference
methodId
arguments
int (0=Request, 1= Reply)
int
RemoteObjectRef
int or Method
array of bytes

Failure model
Problems:
Omission failures.
No guarantee of delivery in order.
Timeout
Use timeout and resend request when timeout expires and reply hasn’t
arrived
Duplicate request messages:
filter out duplicates by requestID
Idempotent (unchangable) operations: same result obtained on every
invocation.
Non-idempotent operations: re-send result stored from previous request,
requires maintenance of a history of replies.
History: server contains a record of reply messages that have been
transmitted to avoid re-execution of operations.
89

RPC exchange protocol
Three protocols are used for implementing various types of RPC.
In the R protocol, a single request message is sent by the client to the
server.The R protocol may be used when there is no value to be returned
from the remote method.
The RR protocol is useful for most client-server exchanges because it is
based on request-reply protocol.
RRA protocol is based on the exchange of three messages: request-reply-
acknowledge reply.
Name Messagessent by
Client Server Client
R Request
RR Request Reply
RRA Request Reply Acknowledge reply
90

HTTP is a request-reply protocol for the exchange of network resources
between web clients and web servers.
HTTP protocol steps are:
I. Connection establishment between client and server at the default server
port or at a port specified in the URL.
II.client sends a request.
III.server sends a reply.
IV.connection close.
HTTP specifies:
Messages,Methods,Arguments,Results,Rules for representing data in
messages.
HTTP: an example of a request – reply protocol
91

HTTP 1.1 uses persistent connections.
Persistent connections are connections that remains open over a series of
request-reply exchanges between client and server.
Marshalling
Request and replies are marshalled into messages as ASCII text string
Resources are represented as byte sequences and may be compressed
MIME (Multipurpose Internet Mail Extensions) to send text, images
and so on.

HTTP - methods
GET: Requests the resource whose URL is given as argument.
HEAD: Similar to GET, but no data returned.
POST: Can deal with data supplied with the request.
PUT: data supplied in the request is stored with the given URL as its
identifier either as modification of the an existing resource or as a new
resource.
DELETE: Server deletes resource identified by the URL.
OPTIONS: Server supplies client with list of methods that it allows.
TRACE: Server sends the request back.
93

HTTP request / reply messages
GET //www.dcs.qmw.ac.uk/index.html HTTP/ 1.1
URL or pathnamemethod HTTP version headers message body
HTTP/1.1 200 OK resource data
HTTP version status code reason headers message body
Request/Reply:
A request message specifies the method name, the URL of resource, the
protocol version, some headers and an optional message body (not needed
in case of data resource).
A reply message specifies the protocol version, a status code and reason,
some header and an optional message body.
94

Group Communication
• Multicasting: operation to send a single message from one process to
each of the members of a predefined group of processes transparently.
•useful for:
1. fault tolerance based on replicated services
requests multicast to servers, some may fail, the client will be served
2. discovering services
multicast to find out who has the services
3. better performance through replicated data
multicast updates
4. event notification
new items arrived, advertising services

IP Multicast – group communication
A multicast group is specified by a class D Internet address
Sender is unaware of the identities of the individual recipients
Available only via UDP
The membership of a group is dynamic
It is possible to send datagram to a multicast group without being a
member and join or leave the group at any time.
IPv4
• Multicast IP routers
1. IP packets can be multicast both on local network and on the wider
Internet.
2. Local multicast uses local network such as Ethernet.
3. To limit the distance of propagation of a multicast datagram, the sender
can specify the number of routers it is allowed to pass- called the time to
live, or TTL for short.
96

• Multicast address allocation
1. Multicast addressing may be permanent or temporary.
2. Permanent groups.
3. Multicast addressing by temporary groups must be created before use
and cease to exit when all members have left.
Java API to IP multicast
1. The Java API provides a datagram interface to IP multicast through the
class MulticastSocket, which is a subset of DatagramSocket with the
additional capability of being able to join multicast groups.
2. The class MulticastSocket provides two alternative constructors ,
allowing socket to be creative to use either a specified local port, or
any free local port.

Multicast peer joins a group and sends and receives datagrams
import java.net.*;
import java.io.*;
public class MulticastPeer{
// args give message contents & destination multicast group (e.g. "228.5.6.7")
MulticastSocket s =null;
try {
InetAddress group = InetAddress.getByName(args[1]);
s = new MulticastSocket(6789);
s.joinGroup(group);
byte [] m = args[0].getBytes();
DatagramPacket messageOut =
new DatagramPacket(m, m.length, group, 6789);
s.send(messageOut);
98

// get messages from others in group
for(int i=0; i< 3; i++) {
DatagramPacket messageIn =
new DatagramPacket(buffer, buffer.length);
s.receive(messageIn);
System.out.println("Received:" + new String(messageIn.getData()));
}
s.leaveGroup(group);
}catch (SocketException e){System.out.println("Socket: " +
e.getMessage());
}catch (IOException e){System.out.println("IO: " + e.getMessage());}
}finally {if(s != null) s.close();}
}
}
99

UNIX socket
Datagram communication
Datagram Socket
Bind
Sendto
recvfrom
Stream communication
stream socket , bind
Accept
Connect
Write and read
100
Case Study

Sockets used for datagrams
ServerAddress and ClientAddress are socket addresses
Sending a message Receiving a message
bind(s, ClientAddress)
sendto(s, "message", ServerAddress)
bind(s, ServerAddress)
amount = recvfrom(s, buffer, from)
s = socket(AF_INET, SOCK_DGRAM, 0)s = socket(AF_INET, SOCK_DGRAM, 0)
101

Sockets used for streams
Requesting a connection Listening and accepting a connection
bind(s, ServerAddress);
listen(s,5);
sNew = accept(s, ClientAddress);
n = read(sNew, buffer, amount)
s = socket(AF_INET, SOCK_STREAM,0)
connect(s, ServerAddress)
write(s, "message", length)
s = socket(AF_INET, SOCK_STREAM,0)
ServerAddress and ClientAddress are socket addresses
102

Introduction
Communication between distributed objects
Remote procedure call
Events and notifications
Java RMI case study
Distributed objects and remote invocation

Interface
Specifies accessible procedures and variables
• Interface in distributed system
Can’t access variables directly
Input argument and output argument
Pointers can’t be passed as arguments or returned results
RPC’s Service interface
specification of the procedures of the server, defining the types of the input
and output arguments of each procedure
RMI’s Remote interface
Specification of the methods of an object that are available for objects in other
processes, defining the types of them.
may pass objects or remote object references as arguments or returned result
Interfaces

An Interface Definition Language, or IDL, is a formal specification that
defines a remote module’s:
Parameters
Arguments
Types
Ex:java RMI

CORBA IDL example
//In file Person.idl
struct Person {
string name;
string place;
long year;
} ;
interface PersonList {
readonly attribute string listname;
void addPerson(in Person p) ;
void getPerson(in string name, out Person p);
long number();
};
remote interface
remote interface defines
methods for RMI
CORBA has a struct
parameters are in, out or inout

each process contains objects, some of which can receive remote invocations,
others only local invocations
those that can receive remote invocations are called remote objects
objects need to know the remote object reference of an object in another process in
order to invoke its methods.
the remote interface specifies which methods can be invoked remotely
invocation invocation
remote
invocation
remote
local
local
local
invocation
invocation
A
B
C
D
E
F
Communication Between Distributed Objects
Distributed Objects and Remote Invocation-RMI

interface
remote
m1
m2
m3
m4
m5
m6
Data
implementation
remoteobject
{ of methods
Remote object and remote interface
• Remote interface:
– specifies the methods of an object available for remote invocation
– an interface definition language (or IDL) is used to specify remote
interfaces. E.g. CORBA IDL,Java RMI
– Java RMI would have a class for Person, but CORBA has a struct

Fault Tolerance Measures
Retry Request Message
Determines if the requesting system should retransmit a request until
a response is received
Duplicate Filtering
Determines if the same request received multiple times should be
filtered down to one request, or if all requests should be re-executed
Retransmission of Results
Determines if the results from requests should be saved and resent
instead of re-executing a request

RMI Invocation Semantics
1. Invocation semantics depend upon implementation of Request- Reply
protocol used by R MI
2. Maybe, At-least-once, At-most-once
Transparency
1. remote invocations should be made transparent in the sense that syntax
of a remote invocation is the same as the syntax of local invocation
(access transparency) but programmers should be able to distinguish
between remote and local objects by looking at their interfaces, e.g. in
Java RMI, remote objects implement the Remote interface
Design Issues of RMI

Maybe Invocation Semantics
Remote method requests may be executed once or not at all
Used when request messages are not resent when a fault occurs
Requesting system cannot tell if the server executed the request before the
fault occurred.
At-Least-Once Invocation Semantics
Requesting system receives either a result or an exception
Server will re-execute a request for each retry request message received
Not appropriate when executing the same method multiple times will alter
the result
At-Most-Once Invocation Semantics
Requesting system either receives a result or an exception
If the requesting system receives a result, it is because the server executed
the result exactly once
Used when duplicate request messages are filtered out

object A object B
skeleton
Request
proxy for B
Reply
CommunicationRemote Remote referenceCommunication
modulemodulereference module module
for B’s class
& dispatcher
remote
client server
The architecture of remote method invocation

Proxy - makes RMI transparent to client.Marshals requests and
unmarshals results. Forwards request.
Communication Module:
1. Transmits request and reply messages
2. Concerned with the Message Type, requestId, and the remote
reference of the object
3. Specifies the Invocation Semantics (Ex: At-Most-Once)
Skeleton - implements methods in remote interface. Unmarshals requests
and marshals results. Invokes method in remote object.

RMI software - between application level objects and communication
and remote reference modules
Remote reference module
1. Translates the remote object reference in a message to a local object
message and vice versa
2. Creates new remote object references
3. Maintains the local to remote object references in a Remote Object
Table
Dispatcher - gets request from communication module and invokes
method in skeleton (using methodID in message).

Distributed Objects and Remote Invocation – RPC
• Similar to RMI , a client program calls a procedure in another program
running in a server process.
• may be implemented to have one of the choices of invocation semantics –
at-least-once, at-most-once are generally chosen.
• Implemented using request-reply protocol .
• only addresses procedure calls.
• A client that accesses a server includes one stub procedure for each
procedure in the service interface.
• A client stub procedure is similar to a proxy method of RMI .
• A server stub procedure is similar to a skeleton method of RMI

Sun RPC
Client-server communication in the SUN NFS (network file system)
Also called ONC (Open Network Computing) RPC
In other unix OS as well
UDP or TCP
Interface Definition Language (IDL)
initially XDR is for data representation, extended to be IDL
less modern than CORBA IDL and Java
program numbers instead of interface names
procedure numbers instead of procedure names
single input parameter (structs)
rpcgen: compiler for XDR
client stub
server main procedure, dispatcher, and server stub
XDR marshalling, unmarshaling

binding (registry)
local binder--portmapper
server registers its program/version/port numbers with
portmapper
client contacts the portmapper at a fixed port with
program/version numbers to get the server port
different instances of the same service can be run on different
computers--different ports
authentication
request and reply have additional fields
unix style (uid, gid), shared key for signing, Kerberos

Distributed Objects and Remote Invocation -Events and
notifications
Distributed event-based systems allow objects at different locations to be
notified of events taking place in an object.
This is achieved through the publish subscribe paradigm.
Notifications are objects that represent events.
Events may be of different types. Each event having several attributes
and both are notifications used in subscription and notifications.

Dealer’s computer
Information
provider
Dealer
External
source
External
source
Information
provider
Dealer
Dealer
Dealer
Notification
Notification
Notification
Notification
Notification
Notification
Notification
Notification
Dealer’s computer
Dealer’s computerDealer’s computer
Notification
Notification
subscribers: dealers in stock exchange

subscriberobserverobject of interest
Event service
object of interest
object of interest observer
subscriber
subscriber
3.
1.
2. notification
notification
notification
notification
[3: observer checks for changes; object of interest is not part of the event
service]

Observers
An Observer is one or more objects that sits between an object of interest
and the subscribers that can filter and massage event notifications before
they reach subscribers.
Observers can provide:
Forwarding: Objects of interest send a single notification to an
observer. The observer handles sending bulk notification, letting the
object of interest continue processing without delay
Filtering: Observes can use logic to only send a handful of notification
to subscribers
Notification Mailboxes: Observers can store notification until
subscribers are ready to consume the messages while the object of
interest can continue processing unhindered

possible event notification schemes:
1) Object of interest publishes notifications directly to subscribers
2) Object of interest publishes to an observer. The observer handles
publication to subscribers
3) Object of interest is outside the local event service. An observer
periodically queries the object of interest to see if an event has
happened. The observer will publish notifications to subscribers if
it finds an event.
The participants are: the object of interest, an event, a notification, a
subscriber, an observer object, and a publisher.

Java RMI
it is a library that allows a Java program on one machine to call
functions (methods) inside an object on a remote machine.
A remote object must implement Remote interface.
In the example we do the followings:
Define the remote interface
Implement the server
Implement the client
Compile the source files
Start the Java RMI registry, server, and client
Ex:Shared whiteboard
It allows a group of users to share a common view of a drawing surface
containing graphical objects.
The server maintains the current state of a drawingClients can poll the
server about the latest shape of a drawing.
The server attaches version numbers to new arriving shapes.

Remote Interface
Remote interfaces are defined by extending an interface called
Remote provided in the java.rmi package.
Figure given shows an example of two remote interface called Shape
and ShapeList.
In this example, GraphicalObject is a class that holds the state of a
graphical object.
GraphicalObject must implement the Serializable interface.
Ordinary and remote objects can appear as input and output
arguments.

import java.rmi.*;
import java.util.Vector;
public interface Shape extends Remote {
int getVersion() throws RemoteException;
GraphicalObject getAllState() throws RemoteException;
}
public interface ShapeList extends Remote {
Shape newShape(GraphicalObject g) throws RemoteException;
Vector allShapes() throws RemoteException;
int getVersion() throws RemoteException;
}

Parameter and result passing
1. Passing remote objects
The result is passed by (object) reference.
In line2, the return value of the method newShape is defined as shape -
a remote interface.
When a remote object reference is received, an RMI can be issued on
the object refered to by this reference.
2. Passing non-remote objects
The result is passed by value.
A new object is created locally, with the state differing from the original
object.

Downloading of Classes
Classes can be transferred from one Java VM to another.
Parameters are passed by value or by reference.
If the recipient does not yet possess the class of an object passed by value, its
code is downloaded automatically.
If the recipient of a remote object reference does not yet possess the class for a
proxy, its code is downloaded automatically.
RMIregistry
The RMIregistry is the binder for Java RMI.
The RMIregistry runs on every server that hosts remote objects.
It maps local object names of the form //computerName:port/objName to object
references.
This service is not a global service.
Clients need to query a particular host to get reference.

void rebind (String name, Remote obj):This method is used by a server
to register the identifier of a remote object by name,
void bind (String name, Remote obj):This method can alternatively be
used by a server to register a remote object by name, but if the name is
already bound to a remote object reference an exception is thrown.
void unbind (String name, Remote obj):This method removes a binding.
Remote lookup (String name):This method is used by clients to look up
a remote object by name
String [] list():This method returns an array of Strings containing the
names bound in the registry.
Naming Class of Java RMI Registry

import java.rmi.*;
public class ShapeListServer{
public static void main (String args[]){
System.setSecurityManager (new RMISecurityManager());
try{
ShapeList aShapeList = new ShapeListServant(); 1
Naming.rebind(“Shape List”, aShapeList); 2
System.out.println(“ShapeList server ready”);
}catch(Exception e) {
System.out.println(“ShapeList server main” + e.getMessage());}
}
} Figure 6. Java class ShapeListServer with main method

Security Manager
implements various security policies for client accesses
Main method
1: create instance of ShapeListServant
2: binds name "ShapeList" to newly created instance in RMI Registry.
ShapeListServant implements ShapeList
Figure 7 gives an outline of the class ShapeListServant.
1: UnicastRemoteObject - objects that live only as long as creating
process
2: factory method - client can request creation of a new object

import java.rmi.*;
import java.rmi.server.UnicastRemoteObject;
public class ShapeListServant extends UnicastRemoteObject implements ShapeList{
private Vector theList; // contains the list of Shapes
private int version;
public ShapeListServant()throws RemoteException{….}
public Shape newShape(GraphicalObject g) throws RemoteException{
version++
Shape s = new ShapeServant(g, version);
theList.addElement(s);
return s;
}
public Vector allShapes() throws RemoteException{…}
public int getVersion() throws RemoteException{…}
}

Client program
A simplified client for the ShapeList sever is illustrated in
Figure 8.
polling loop:
• 1: look up remote reference
• 2: invoke allShapes() in remote object

import java.rmi.*;
import java.rmi.server.*;
public class ShapeListClient{
System.setSecurityManager(new RMISecurityManager());
ShapeList aShapeList = null;
try{
aShapeList = (ShapeList) Naming.lookup(“//bruno.ShapeList”);
Vector sList = aShapeList.allShapes();
} catch(RemoteException e) {System.out.println(e.getMessage());
} catch(Exception e) {System.out.println(“Client:”+e.getMessage());}
}
} Figure 8. Java client of ShapeList

The source files for this example can be compiled as follows:
javac Hello.java Server.java Client.java
Start the Java RMI registry :
rmiregistry
Start the Server: java Server
Start the Client: java Client

Figure 9. Classes supporting Java RMI
the inheritance structure of the classes supporting Java RMI servers.

Distributed computing bsics

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (20)

Similar a Distributed computing bsics

Similar a Distributed computing bsics (20)

Más de Deepak John

Más de Deepak John (20)

Último

Último (20)

Distributed computing bsics