Chapter 4 - Communication
Introduction
interprocess communication is at the heart of all distributed
systems
communication in distributed systems is based on message
passing as offered by the underlying network which is
harder as opposed to using shared memory
modern distributed systems consist of thousands of
processes scattered across an unreliable network such as
the Internet
unless the primitive communication facilities of the network
are replaced by more advanced ones, development of large
scale Distributed Systems becomes extremely difficult
2
Objectives of the Chapter
review how processes communicate in a network (the rules
or the protocols) and their structures
introduce the five widely used communication models for
distributed systems:
Remote Procedure Call (RPC) - which hides the details of
message passing and suitable for client-server models
Remote Object (Method) Invocation (RMI)
Message-Oriented Middleware (MOM) - instead of the
client-server model, think in terms of messages and have
a high level message queuing model similar to e-mail
Stream-Oriented Communication - for multimedia to
support the continuous flow of messages with timing
constraints
Multicast Communication - information dissemination for
several recipients
3
4.1 Network Protocols and Standards
why communication in distributed systems? because there is
no shared memory
two communicating processes must agree on the syntax and
semantics of messages
a protocol is a set of rules that governs data communications
a protocol defines what is communicated, how it is
communicated, and when it is communicated
for instance, for one computer to send a message to another
computer, the first computer must perform the following
general steps (highly simplified)
break the data into small sections called packets (message,
datagram, packet, frame)
add addressing information to the packets identifying the
source and destination computers
deliver the data to the network interface card for
transmission over the network
4
the receiving computer must perform the same steps, but in
reverse order
accept the data from the NIC
remove transmitting information that was added by the
transmitting computer
reassemble the packets of data into the original message
the key elements of a protocol are syntax, semantics, and
timing
syntax: refers to the structure or format of the data
semantics: refers to the meaning of each section of bits
timing: refers to when data should be sent and how fast
they can be sent
functions of protocols
each device must perform the same steps the same way
so that the data will arrive and reassemble properly; if
one device uses a protocol with different steps, the two
devices will not be able to communicate with each other
5
Protocols in a layered architecture
protocols that work together to provide a layer or layers of
the model are known as a protocol stack or protocol suite,
e.g. TCP/IP
each layer handles a different part of the communications
process and has its own protocol
Data Communication Standards
standards are essential for interoperability
data communication standards fall into two categories
De facto standards: that have not been approved by an
organized body; mostly set by manufacturers
De jure standards: those legislated by an officially
recognized body such as ISO, ITU, ANSI, IEEE
6
Network (Reference) Models
Layers and Services
within a single machine, each layer uses the services
immediately below it and provides services for the layer
immediately above it
between machines, layer x on one machine communicates
with layer x on another machine
Two important network models or architectures
The ISO OSI (Open Systems Interconnection) Reference
Model
The TCP/IP Reference Model
a. The OSI Reference Model
consists of 7 layers
was never fully implemented as a protocol stack, but a
good theoretical model
Open – to connect open systems or systems that are open
for communication with other systems
7
layers, interfaces, and protocols in the OSI model
8
Media (lower) Layers
Physical: Physical characteristics of the media
Data Link: Reliable data delivery across the link
Network: Managing connections across the network
or routing
Transport: End-to-end connection and reliability (handles
lost packets); TCP (connection-oriented),
UDP (connectionless), etc.
Session: Managing sessions between applications
(dialog control and synchronization); rarely
supported
Presentation: Data presentation to applications; concerned
with the syntax and semantics of the
information transmitted
Application: Network services to applications; contains
protocols that are commonly needed by
users; FTP, HTTP, SMTP, ...
Host (upper) Layers
9
a typical message as it appears on the network
10
b. The TCP/IP Reference Model
TCP/IP - Transmission Control Protocol/Internet Protocol
used by ARPANET and its successor the Internet
design goals
the ability to connect multiple networks (internetworking)
in a seamless way
the network should be able to survive loss of subnet
hardware, i.e., the connection must remain intact as long
as the source and destination machines are properly
functioning
flexible architecture to accommodate requirements of
different applications - ranging from transferring files to
real-time speech transmission
these requirements led to the choice of a packet-switching
network based on a connectionless internetwork layer
has 4 (or 5 depending on how you see it) layers:
Application, Transport, Internet (Internetwork), Host-to-
network (some split it into Physical and Data Link) 11
OSI and TCP/IP Layers Correspondence
12
Layers involved in various hosts (TCP/IP)
when a message is sent from device A to device B, it may
pass through many intermediate nodes
the intermediate nodes usually involve the first three layers
13
Middleware Protocols
a middleware is an application that contains general-purpose
protocols to provide services
example of middleware services
authentication and authorization services
distributed transactions (commit protocols; locking
mechanisms) - see later in Chapter 8
middleware communication protocols (calling a procedure
or invoking an object remotely, synchronizing streams for
real-time data, multicast services) - see later in this Chapter
hence an adapted reference model for networked
communications is required
14
an adapted reference model for networked communication
15
4.2 Remote Procedure Call
the first distributed systems were based on explicit message
exchange between processes through the use of explicit
send and receive procedures; but do not allow access
transparency
in 1984, Birrel and Nelson introduced a different way of
handling communication: RPC
it allows a program to call a procedure located on another
machine
simple and elegant, but there are implementation problems
the calling and called procedures run in different address
spaces
parameters and results have to be exchanged; what if the
machines are not identical?
what happens if both machines crash?
16
Conventional Procedure Call, i.e., on a single machine
e.g. count = read (fd, buf, bytes); a C like statement, where
fd is an integer indicating a file
buf is an array of characters into which data are read
bytes is the number of bytes to be read
Stack pointer
Stack pointer
parameter passing in a local procedure the stack while the called
call: the stack before the call to read procedure is active
parameters can be call-by-value (fd and bytes) or call-by
reference (buf) or in some languages call-by-copy/restore 17
Client and Server Stubs
RPC would like to make a remote procedure call look the
same as a local one; it should be transparent, i.e., the calling
procedure should not know that the called procedure is
executing on a different machine or vice versa
principle of RPC between a client and server program
when a program is compiled, it uses different versions of
library functions called client stubs
a server stub is the server-side equivalent of a client stub
Steps of a Remote Procedure Call
1. Client procedure calls client stub in the normal way
2. Client stub builds a message and calls the local OS
(packing parameters into a message is called parameter
marshaling)
3. Client's OS sends the message to the remote OS
4. Remote OS gives the message to the server stub
5. Server stub unpacks the parameters and calls the server
6. Server does the work and returns the result to the stub
7. Server stub packs it in a message and calls the local OS
8. Server's OS sends the message to the client's OS
9. Client's OS gives the message to the client stub
10. Stub unpacks the result and returns to client
hence, for the client remote services are accessed by making
ordinary (local) procedure calls; not by calling send and
receive
) server machine vs server process; client machine vs client process
Asynchronous RPC
a shortcoming of the original model is that it is blocking: but
no need of blocking for the client in some cases
two cases
1. if there is no result to be returned
e.g., inserting records in a database, ...
the server immediately sends an ack promising that it
will carryout the request
the client can now proceed without blocking
a) the interconnection between client and server in a traditional RPC
b) the interaction using asynchronous RPC
2. if the result can be collected later
e.g., prefetching network addresses of a set of hosts, ...
the server immediately sends an ack promising that it
will carryout the request
the client can now proceed without blocking
the server later sends the result
a client and server interacting through two asynchronous RPCs
the above method combines two asynchronous RPCs
and is sometimes called deferred synchronous RPC
variants of asynchronous RPC
let the client continue without waiting even for an ack,
called one-way RPC
problem: if reliability of communication is not guaranteed
4.3 Remote Object (Method) Invocation (RMI)
(Chapter 10: Distributed Object-Based Systems; from page 443)
resulted from object-based technology that has proven its
value in developing nondistributed applications
it is an expansion of the RPC mechanisms
it enhances distribution transparency as a consequence of
an object that hides its internal from the outside world by
means of a well-defined interface
Distributed Objects
an object encapsulates data, called the state, and the
operations on those data, called methods
methods are made available through an interface
the state of an object can be manipulated only by invoking
methods
this allows an interface to be placed on one machine while
the object itself resides on another machine; such an
organization is referred to as a distributed object
if the state of an object is not distributed, but only the
interfaces are, then such an object is referred to as a remote
object
the implementation of an object’s interface is called a proxy
(analogous to a client stub in RPC systems)
it is loaded into the client’s address space when a client
binds to a distributed object
tasks: a proxy marshals method invocation into messages
and unmarshals reply messages to return the result of the
method invocation to the client
a server stub, called a skeleton, unmarshals messages and
marshals replies
common organization of a remote object with client-side proxy
Object Servers
an object server is a server to support distributed objects
it does not provide a specific service; services are
implemented by the objects that reside on the server
the server provides only the means to invoke local objects
based on remote client requests
4.4 Message-Oriented Communication
RPCs and RMIs are not adequate for all distributed system
applications
the provision of access transparency may be good but
they have semantics that is not adequate for all
applications
example problems
they assume that the receiving side is running at the
time of communication
a client is blocked until its request has been processed
messaging is the solution
4.5 Stream-Oriented Communication
until now, we focused on exchanging independent and
complete units of information
time has no effect on correctness; a system can be slow or fast
however, there are communications where time has a critical
role
Multimedia
media
storage, transmission, interchange, presentation,
representation and perception of different data types
text, graphics, images, voice, audio, video, animation, ...
movie: video + audio + …
multimedia: handling of a variety of representation media
end user pull
information overload and starvation
technology push
emerging technology to integrate media
The Challenge
new applications
multimedia will be pervasive in few years (as graphics)
continuous delivery
e.g., 30 frames/s (NTSC), 25 frames/s (PAL) for video
guaranteed Quality of Service
admission control
storage and transmission
e.g., 2 hours uncompressed HDTV (1920×1080) movie:
1.12 TB (1920×1080x3x25x60x60x2)
videos are extremely large, even after compressed
(actually encoded)
search
can we look at 100… videos to find the proper one?
Types of Media
two types
discrete media: text, executable code, graphics, images;
temporal relationships between data items are not
fundamental to correctly interpret the data
continuous media: video, audio, animation; temporal
relationships between data items are fundamental to
correctly interpret the data
a data stream is a sequence of data units and can be applied
to discrete as well as continuous media; e.g., TCP provides
byte-oriented discrete data streams
stream-oriented communication provides facilities for the
exchange of time-dependent information (continuous media)
such as audio and video streams
timing in transmission modes
asynchronous transmission mode: data items are
transmitted one after the other, but no timing constraints;
e.g. text transfer
synchronous transmission mode: a maximum end-to-end
delay defined for each data unit; it is possible that data can
be transmitted faster than the maximum delay, but not
slower
isochronous transmission mode: maximum and minimum
end-to-end delay are defined; also called bounded delay
jitter; applicable for distributed multimedia systems
a continuous data stream can be simple or complex
simple stream: consists of a single sequence of data; e.g.,
mono audio, video only (only visual frames)
complex stream: consists of several related simple streams,
called substreams, that must be synchronized; e.g., stereo
audio, video consisting of audio and video (may also contain
subtitles, translation to other languages, ...)
a stream can be considered as a virtual connection between a
source and a sink
the source or the sink could be a process or a device
streaming means a user can listen (or watch) after the
downloading has started
we can stream stored data or live data (compression, actually
encoding is required)
A general architecture for streaming stored multimedia data over a network
setting up a stream directly between two devices - live data
the data stream can also be multicasted to several receivers
if devices and the underlying networks have different
capabilities, the stream may be filtered, generally called
adaptation (filtering?, transcoding?)
an example of multicasting a stream to several receivers
Quality of Service (QoS)
timing and other nonfunctional requirements are expressed
as Quality of Service requirements
QoS requirements describe what is needed from the
underlying distributed system and network to ensure
acceptable delivery; e.g. viewing experience of a user
for continuous data, the concerns are
timeliness: data must be delivered in time
initial delay: maximum delay until a session has been
setup
maximum end-to-end delay
maximum delay variance or jitter
volume/bandwidth: the required throughput (bit rate) must
be met
reliability: a given level of loss of data must not be
exceeded
quality of perception: highly subjective
68
4.6 Multicast Communication
multicasting: delivery of data from one host to many
destinations; for instance for multimedia applications
a one-to-many relationship
1. Application-Level Multicasting
nodes are organized into an overlay network (a network
which is built on top of another network) and information is
disseminated to its members (routers are not involved as in
network-level routing)
how to construct the overlay network
nodes organize themselves as a tree with a unique path
between two pairs of nodes or
nodes organize into a mesh network and there will be
multiple paths between two nodes; adv: robust
2. Gossip-Based Data Transmission
use epidemic protocols where information is propagated
among a collection of nodes without a coordinator
for details read pages 166-174
