Chapter 2.

1 Layered Protocols Introduction

 Introduction  Interprocess communication is the heart of all
 Lower-layer Protocols distributed systems
 Physical layer  Modern DS consists of x1000 or more processes scattered
across an unreliable network
 Data link layer
 Network layer  All communications in DS is based on exchanging
 Transport layer low-level messages
 Higher-level Protocols  However, we need to agree on some assumptions
 Session and Presentation Protocols  Volts for 1 or 0?
 Application Protocols  First and last bit of a message?
 Middleware Protocols  Damaged or lost message?

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Introduction (Cont.) Introduction (Cont.)

 Solutions: protocol  Standard : OSI model (or ISO OSI)
 A protocol defines the format and the order of messages sent and
received among network entities, and the actions taken on message  ISO: International Standards Organization
transmission and receipt
 Human protocols:  OSI: Open Systems Interconnection Reference
 “What’s the time?” Model
“I have a question”

 Allows open systems to communicate
 Network protocols:
 machines rather than humans  Consist of sever-layers
 all communication activity in Internet governed by protocols
 A collection of protocols is called a protocol stack or
protocol suite

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 OSI 7-layered Model Message Encapsulation

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Lower-Layer Protocols Physical Layer

 Physical Layer  Contain the specification and implementation
of bits and their transmission between sender
 Data Link Layer and receiver
 Concerned with transmitting 0s and 1s
 How many volts to use for 0 and 1
 Network Layer  Uni- or bi-directional transmission
 Transmission speed (bits/sec)
 Transport Layer

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Data Link Layer Network Layer
 Prescribe the transmission of a series of bits into a  Describes how packets in a network of
frame (packet) to allow for error and flow control. computers are to be routed
 Grouping bits into frames and make sure each frame is
 It may take several hops from a sender to a
correctly received
receiver
 By putting a special bit pattern on the start (header) and
end of each frame (checksum)  Routing is the primary task of this layer
 Computing checksum  The most popular protocol: IP (Internet Protocol)
 Examples  ATM virtual channel is another example
 Ethernet, IEEE 802.11 WLAN

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Transport Protocols Client-Server TCP

 The last part of a basic network protocol stack  In client-server architecture
 Provides the actual communication facilities  The clients may only send a request
for most distributed systems  Then the server respond with a answer
 Standard Internet protocols:  However, TCP must manage the connection
 TCP: connection-oriented, reliable, stream-
 Cause too much overhead
oriented communication
 But high-overhead  Solution: TCP for Transactions (T/TCP)
 UDP: unreliable (best-effort) datagram  Combine the connection management with the
communication request/answer into one packet
 But low-overhead

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Client-Server TCP Higher-Level Protocols
 Session layer: an enhanced version of transport layer
 Provide dialog control
 Keep track of which party is currently talking
 Provide synchronization facilities
 Presentation layer: different machine have different
data representation, thus we need
 Conversion, compression, or encryption
 Application layer
 FTP
 HTTP……
Normal TCP Transactional TCP

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Higher-Level Protocols (Cont.) Higher-Level Protocols (Cont.)

 However, only the application layer is ever  Middleware protocols: protocols to support a variety
of middleware services
used
 Slide. 61 in Chapter 1
 Everything above the transport layer is grouped  Examples:
together  Authentication protocols
 Distributed commit protocols
 The session layer, presentation layer are included  Distributed locking protocols
in the application layer of Internet protocol suite  Middleware protocols support high-level
(TCP/IP). communication services, like RPC, Remote Object
Invocation, message queuing services…

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Middleware Protocols Chapter 2.2: Remote Procedure Call
 Basic RPC operations

 Parameter Passing

 Extended RPC Models

 DCE RPC

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Introduction Basic RPC Operations: Fig. 2.6

 RPC (Remote Procedure Call)  Conventional Procedure Call
 Allows programs to call procedures located on  count = read(fd, buf, nbytes)
other machines
 Not by calling primitive send() and receive()
 Problems
 Different address spaces  pointer?
 Data representations  integer, floating?
 Machine crashes?
the stack before the call to read The stack while the called procedure is active

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Basic RPC Operations (Cont.) Client and Server Stubs
 Problem  The main challenge of RPC is to look as
 Call-by-value or call-by-reference much like LPC, i.e., transparency
 fd, and nbytes are call-by-value
 Their value is copied into the stack
 Modification in the called procedure but no changes in the  Transparency is achieved via “stubs”
calling procedure
 buf is call-by-reference  A library in the client (client stub) and server
 Its address is copied into the stack (server stub)
 Modification in the called procedure is reflected in the
calling procedure
 To be linked with the program

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Client and Server Stubs (Cont.) Client and Server Stubs

 Client stub: is called using the conventional calling sequence
(Fig. 2.6 in previous two slide)
 Packs parameters into a message
 Calls OS to send directly to the server
 Waits for result-return from the server by calling receive()

 Server stub
 Be blocked waiting for incoming message by calling receive()
 Unpacks the parameters
 Calls the corresponding procedure in the usual way (Fig. 2.6)
 Returns results to the caller
 Does a call to receive again to wait for next request

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 RPC Steps Parameter Passing
1. Client procedure calls client stub in normal way  Packing parameters into a message is called parameter
2. Client stub builds message, calls local OS marshaling
3. Client's OS sends message to remote OS  It’s fine if
4. Remote OS gives message to server stub  Client and server machine are identical
 And all the parameters and results are scalar types, like
5. Server stub unpacks parameters, calls server integers, characters, and booleans.
6. Server does work, returns result to the stub  However, possible problems:
7. Server stub packs it in message, calls local OS  IBM mainframes use EBCDIC char code and IBM PC uses
8. Server's OS sends message to client's OS ASCII code
 Integer: one’s complement and 2’s complement
9. Client's OS gives message to client stub
 Floating-point numbers
10. Stub unpacks result, returns to client  Little endian and big endian

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Different Data Representations

Passing Value Parameters
 Execution of add(i,j) via RPC

 (a) Original message on the Pentium (little endian) (5, “JILL”)

 Number their bytes from right to left
 (b) After receipt on SPARC (big endian) (5x2^24, “JILL”)
 Intel sends the message by: 0, 1, 2……., But SPARC puts in big
endian order
 (c) Solution but wrong: invert the bytes
 The message after being inverted. (5, “LLIJ”)
 The little numbers in boxes indicate the address of each byte
PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Parameter Specification and Stub
Passing Reference Parameters Generation
 A pointer is only meaningful in its address space  Parameter specification
 Possible solutions  Agree on the format of the message they are exchange
 Word: 3 bytes……
 Forbid pointer usage in RPC calls
 Copy the entire pointed area (such as arrays or strings) into  Agree on the representation of data structure
 Integer: two’s complement
the message and send it to the server
 i.e., call-by-reference is replaced by call-by-copy/restore  Agree on the actual exchange of message
 Not always identical but is good in most cases  TCP or UDP
 Only suitable for bounded and known areas  Parameter Marshalling
 Optimization: distinguish input parameter and output parameter
 Wrapping parameters into a message with proper coding
 Input parameter: like write, it need not be copied back
 Output parameter: like read, need not be sent over

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Parameter Specification and Stub

Generation (Cont.) Note: call-by-copy/restore
 Stub Generation  Not common in modern programming language
 Stub for different procedure differ only in their  The parameters is copied to the stack and then
interface to the applications copied back after the call
 Interface  Overwrite the caller’s original value
 A collection of procedures  Achieve the same effect as call-by-reference under
 Often specified by an Interface Definition Language most conditions
(IDL)  But in some cases, like the same parameter being
 Interface specified in IDL is then compiled into a present multiple times in the parameter list
client stub and a server stub  The semantics are different

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Extended RPC Models Doors
 Doors
 RPC for processes located on the same machine OS upcalls the procedure
 In some OS, it is called lightweight RPC
(4)
 Asynchronous RPC
 Clients immediately continues after issuing a RPC request, i.e.,
without blocking
(1)
 Deferred synchronous RPC
 A client and server interacting through two asynchronous RPCs
 Server will call the client when the processing is done (2) Associate the fd with a name
(3)
 One-way RPCs
 Client continue immediately after calling the server
 Do not wait for an acknowledgement from the server
 Reliability ? (5)

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Asynchronous RPC Deferred Synchronous RPC

(a) The interconnection between client and server in a traditional RPC

(b) The interaction using asynchronous RPC A client and server interacting through two asynchronous RPCs

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Example: DCE RPC Example: DCE RPC (Cont.)
 DCE: DistriDuted DoDputiDDDDDiroDDeDt  A number of services included
developed by OSF (now called Open Group)  Distributed file service
 A true middleware system
 Support for UNIX, VMS, Windows NT  Directory service
 Idea:  Keep track of the location of all resources
 Add DCE on top of existing machines to behave as an distributed
system (p. 61 in Chap. 1)
 Programming model: client-server model  Security service
 All communication between client and server is by RPC
 Since RPC system hide the details, client and server are independent  Distributed time service
 Client in Java and Server in C  Keep clocks on different machines globally synchronized
 Client and server can run on different hardware or different OS

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Coding Sequence in DCE DCE RPC Coding Sequence

A program used to generate an identifier and prototype IDL file

 Hold everything together: IDL (Interface Definition

Language)
 IDL files contain function prototype, type definition, constant
declarations, and info needed for marshalling and unmarshall
 Step1: use the uuidgen program
 A program used to generate a prototype IDL file containing
a globally unique identifier
 The client sends this ID in its first RPC to the server to prevent
inadvertently binding
 If sent to the wrong server, the server can detect the error
 The ID is obtained by uuidgen program
 Cannot be used again in any other interface file
 Guarantee by encoding ID from location and time of creation

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 DCE RPC Coding Sequence (Cont.) Binding Client to Server in DCE RPC
 Step2: edit the IDL file to fill in names and parameters of RPCs  How does the client locate the server?
 Step3: compilation and the output has three files
 A header file
 The unique identifier, type definition, constant definition, and function
prototype  Binding
 Included in both client and server code
 The client stub  A process of determining the remote procedure
 Contain the actual procedure that the client program will call and the machine on which it executes
 The server stub
 Contain the procedure called by the runtime system on the server machine
 Step 4: write the client and server code and compiler them
 Step 5: link the client code, client stub with runtime library. So
as server

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Binding Client to Server in DCE RPC Performing an RPC

 DCE provides several semantic option
 At-most-once operation
Provide (a) network address of server  No call is ever carried out more than once
(b) server’s name
by server’s name  RPC operation is guaranteed to have been carried out at
most once, but possibly none at all.
3’. return the server IP address  Like server crashes, etc
endpoint=port
 Idempotent
 Declared in IDE file
reply endpoint  It can be repeated multiple times without harm
 Repeat when previous call fails
 Like reading a specified record

A table of (server, endpoint)

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Chapter 2.3: Remote Object Invocation Objects
 Distributed Objects  Object hides its internals from outside world
 Binding a Client to an Object through a well-defined interface
 Allow object to be easily modified, as long as the
 Static/Dynamic RMI interface remains the same
 Parameter Passing  Object includes
 DCE Remote Objects  State: consists of values of its instance variables,
 i.e., encapsulate the data
 Java RMI
 Methods: operations on those data.
 Methods are made available through interface

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Distributed Objects Distributed Objects

 Distributed objects  Notably, Java and C++ allow an instance
 Place an interface at one machine, while the variable to be accessed directly from outside
object itself resides on another machine world
 An object’s interface includes  Not permitted in a distributed object system
 The method of an object for invocation by other  An object’s data should be accessed only by its
objects in other processes method
 The type of input and output parameters
 “Object” can also be passed as parameter
 Object reference would also be used as parameter

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Distributed Objects Distributed Objects (Cont.)
 When a client binds to a distributed object
 A proxy must be in the client’s address space
 Proxy: client-side stub
 Skeleton: server-side stub
 Remote object: one kind of distributed objects
 State of an object is not distributed
 It resides at a single machine
 Only interfaces are made available

Organization of a remote object with client-side proxy

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Compile- time versus Runtime Objects Persistent and Transient Objects

 Compiler-time object:  Persistent object
 Java, C++, or object-oriented languages
 An object is defined an instance of a class
 An object that is guaranteed to live between activations of
 Easier to build distributed application server processes
 But dependent on a particular programming lang.  Persistent object is not dependent on its current server process
 Runtime objects  Sol. its state can be stored in secondary storage
 Construct distributed objects during runtime  A server process exits and store the state in the disk
 Independent of programming language  After a while, a new server process start and read in the state
 App. can be built from objects written in different PL
 But how object are implemented?
 Transient Objects
 Object adapter: a wrapper around the implementation to give it (like C  Live as long as the invoking server does
function) the appearance of an object
 Allow an interface to be converted into something that a client expects

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Implicit & Explicit Binding Implicit & Explicit Binding
 Implicit binding
Distr_object* obj_ref; //Declare a systemwide object reference
 Allows a client to directly invoke methods using obj_ref = …; // Initialize the reference to a distributed object
obj_ref-> do_something(); // Implicitly bind and invoke a method
only a reference to an object
(a)

 Explicit binding
Distr_object objPref; //Declare a systemwide object reference
 The client should call a special function to bind to Local_object* obj_ptr; //Declare a pointer to local objects
obj_ref = …; //Initialize the reference to a distributed object
the object before it actually invoke its methods obj_ptr = bind(obj_ref); //Explicitly bind and obtain a pointer to the local proxy
obj_ptr -> do_something(); //Invoke a method on the local proxy
(b)

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Implementation of Object Reference

Implementation of Object Reference (Cont.)
 Object reference  Problems
 An identifier that can be used as an address of a remote  Server crashes  assign a new endpoint  all references
object
become invalid
 An object reference should have information to allow  Sol: local daemon per machine listen to a well-known endpoint
a client to bind to an object  Keep track of the server-to-endpoint assignments in a table
 IP address (machine), endpoint (port), and object identifier  However, if server is moved to another machine, the local
(object) daemon approach is not valid
Representation of an Object Reference  Sol: location server maintaining where object’s server is running
32 bits 32 bits 32 bits 32 bits  Problem: scalability issue

Internet address port number time object number interface of

remote object

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Implementation of Object Reference Static and Dynamic Remote Method
(Cont.) Invocations
 Problem (Cont.)  RMI (Remote Method Invocation)
 Protocol assumption  client and server must use  Invoke an object’s method through proxy
the same protocol stack  Static invocation
 Sol: include implementation handle in the object  Use predefined interface definitions
reference  Either use interface definition language like RPC approach
 e.g. [Link]  Or use object-based language (Java) that generate stub
 From implementation handle, client can dynamically automatically.
downloaded the needed proxy  A change to the interface requires all applications (i.e.
 Client need not worry about whether it has an clients) to be recompiled
implementation of a specific protocol available
 [Link](para)

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Static and Dynamic Remote Method

Invocations (Cont.) Parameter Passing
 Dynamic invocation  When invoking a method with an object reference as
 Dynamic invocation “composes” a method invocation at run-time parameters
 An application selects at runtime which method it will invoke at a
remote object  For efficiency, differentiate such an object reference is
 invoke(object, method, inpars, outpars) referred to a local object or a remote object
 Usage example: implement a object browser to examine sets of object  For remote object: must issue request between machines,
inefficient when objects are small (integers or Booleans)
 Dynamically present object’s interface to the user
 User then select which method to invoke and show the parameters  If object reference refers to a local object: passed by
 Example: an object called fobject with method append value
 Static: [Link] (int)  The referred object is copied to the server as parameter
 Dynamic: invoke(fobject, id(append), int)
 If object reference refers to a remote object: passed by
reference
 The reference is copied and passed as a parameter

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Parameter Passing Example 1: DCE Remote Objects
Client program at A invoke method of object running at C.
Two object references to O1 and O2 are passed along as parameters  DCE’s RPC mechanism has been “enhanced” to
But reference to O1 is a local object while to O2 is remote directly support remote method invocation
 An extension to their RPC model that is refined from RPC
 DCE Objects = xIDL plus C++.
 Distributed object are specified in IDL and implemented
in C++
So that object at
C can access O1  DEC distributed objects take the form of remote
directly=>
efficient
objects
 A server is responsible for creating C++ objects locally
and making methods available to remote client
 There is no way to create distributed objects
The situation when passing an object by reference or by value.
Note: O1 is passed by value; O2 is passed by reference.
PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Example 1: DCE Remote Objects

Note: Remote Objects (Cont.)
 A distributed object whose state always  Two types of DCE objects are supported
resides on a single machine  Distributed Dynamic Objects – a “private” object
 But whose interface can be made available to created by the server for the client.
remote process  Server creates objects locally on behalf of a client and
is only accessible to the client
 Distributed Named Objects – a “shared” object
that lives on the server and can be accessed by
more than one client
 Created by server and shared by clients

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Example 1: DCE Remote Objects
The DCE Distributed-Object Model
(Cont.)
 Problem in DCE objects
 No mechanism for transparent object reference
 Since refine from RPC, each object invocation is done by RPC
 Thus, lacking a proper system-wide object reference mechanism
 Transparency: client only identifies a remote procedure
in a server
 But in DCE, client identifies a remote procedure in a server’s
object
 Pass the object identifier, the interface identifier, and parameters

a) DCE dynamic objects – requests for creation sent via RPC.

b) DCE named objects – registered with a DS naming service.
PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Java Distributed-Object Model Java Distributed-Object Model (Cont.)

 Java also uses remote objects as the only form of  Cloning objects:
distributed objects  Cloning local object result in a new object of the same
 Java intends to be transparent => making remote object type with the same state at the local machine
access similar to local object  However, semantics of remote object cloning is different
 If client want to clone a remote object
 But network, server machine or server process may fail  Clone the actually object at its server + clone all proxies that are
 Thus, difficult since difference exists between remote currently bound to the remote object at each client (bad)
Thus, remote object can be cloned only by server (to server-> its
object and local one 
local object)
 Cloning local or remote object is different  Proxies are thus not cloned
 Semantics of blocking on an local or remote object  That is, a remote object is not allowed to be cloned by client

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Java Distributed-Object Model (Cont.) Java Remote Object Invocation
 Semantics of blocking on an local or remote object  Any primitive or object type can be passed as
 Java supports objects to be as monitor
 By declaring a method to be synchronized a para to an RMI only if it can be marshaled
Only one process can proceed if two processes call a

synchronized method  Which is called serializable in Java terminology
 The other process is blocked inside a (local) object waiting for a  Platform dependent object, like file descriptors
conditional variables
 But how to blocking on remote object if invoking a and sockets, cannot be serialized
remote synchronized method
 Sol 1: allowing blocking only at server  Like DCE, local objects are passed by value
 What happened when a client crashes when its invocation is being
handled by the server?
and remote objects are passed by reference
 Sol 2: blocking at proxy, i.e., client-side stub
 Need to synchronize different clients at different machine (complex)

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Java Remote Object Invocation (Cont.) Java Remote Object Invocation (Cont.)
 Proxy itself is serializable (proxy just likes local obj.)  A remote object is built from two classes
 Can be marshaled, sent, then unmarshaled remotely and  Server class: implementation of server-side code
used to invoke method on the another remote object  Contain the object’s state and method
 A proxy can be used as a reference to a remote object  Skeleton: server-side stub, is generated from the interface
 i.e., Allow remote invocation at any machine specifications of the object
 Distinguishing features of Java RMI  Client class: implementation of a proxy
 This is possible only when each process is executed the same Java  Also generated from the object’s interface spec.
virtual machine, i.e., the same execution environment  Convert each method call into a message and a reply message
 DCE does not allow stub passing into the result of a method call
 DCE allows difference processes at difference languages, operating  For each call, proxy also sets up a connection with a server
system and hardware platform  Thus, it has all the information to let client finds remote object
 Server’s network address, endpoint, identifier of the object

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Chapter 2.4 Message-Oriented
Communication Introduction
 Persistence and Synchronicity  As a communications mechanism, RPC/RMI is often
inappropriate.
 The receiving side must be executing at the time a request
is made.
 Message-Oriented Transient Communication  The inherent synchronous mode of their operation forces
the client to be blocked
 However,
 Message-Oriented Persistent Communication  What happens if the receiving side is not “awake”?
 In addition, the default “synchronous, blocking” nature of
RPC/RMI is often too restrictive.
 IBM MQSeries  Something else is needed: Messaging

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Persistence and Synchronicity Message-Oriented Communication

 Persistent Communications
 Once sent, the “sender” can stop executing
 The “receiver” need not be operational at this time
 The communications system buffers the message as
required (until it can be delivered).
 E.g. email system:
 Transient Communications
 Message is stored in comm system only when both sender
and receivers are executing
 If problems occur, the message is simply discarded
 E.g. routing

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Persistence and Synchronicity: Six
Persistence and Synchronicity (Cont.) Cases
 Synchronous Communications  Persistence and Synchronicity lead to six
combinations
 A sender DDDcDs, DaiDiDDfor a reply from the
 (a) Persistent asynchronous communication
receiver before doing any other work
 Msgs are stored at the local host or at the communication
server, ex., email
 Asynchronous Communications  (b) Persistent synchronous communication
 Msgs are presistently stored only at the receiving host
 A sender cDDDiDDes with other work immediately  Sender is blocked until its message is stored in receiver’s
upon sending a message to the receiver buffer
 Not necessary the receiving application is executing to store the
message

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Persistence and Synchronicity: Six

Classifying Distributed Communications (1) Cases (Cont.)
 (c) Transient asynchronous communication
 Typically offered by transport-layer datagram, like UDP
 Msgs are stored at the local buffer, the sender continues,
and route by the comm. system.
 But the comm. fails if the receiver is not currently running
 (d) Receipt-based transient synchronous
communication
 Weakest form of synchronous comm
 The sender is blocked until the msg is stored at the recvr’
local buffer at the receiving host
 The sender recvs an ACK and continues
 Persistent asynchronous communication.
 Persistent synchronous communication.

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Persistence and Synchronicity: Six
Classifying Distributed Communications (2) Cases (Cont.)
 (e) Delivery-based transient synchronous
communication
 The sender is blocked until the message is delivered to the
receiver for further processing
 Asynchronous RPC follows this policy
 (f) Response-based transient synchronous
communication
 The strongest form of synchronous behavior
 The sender is blocked until it receives a reply from the
other side
 RPC and RMI adhere to this
c) Transient asynchronous communication.
d) Receipt-based transient synchronous communication.
PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Message-Oriented DraDsieDD
Classifying Distributed Communications (3) Communications
 Transient Messaging Examples
 Sockets

 Message-Passing Interface (MPI)

 Used for Clusters of Worksations, Massively Parallel
Processors, often for scientific applications.

 Remote Procedure Call/ Remote Method

e) Delivery-based transient synchronous communication at message Invocation
delivery.
f) Response-based transient synchronous communication.
PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Berkeley Sockets Socket Operations
Primitive Meaning

Socket Create a new communication endpoint

Bind Attach a local address and port number) to a socket

Announce willingness to accept connections. OS will
Listen
reserve buffer
Accept Block caller until a connection request arrives

Connect Actively attempt to establish a connection

Send Send some data over the connection

Receive Receive some data over the connection

Close Release the connection

Berkeley socket API Connection-oriented communication pattern using sockets.

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Message-Passing Interface (MPI)

Message-Passing Interface (MPI) (Cont.)
 However, DS developers rejected Sockets:  MPI is designed for parallel applications and
 Wrong level of abstraction with only “send” and “receive”. thus tailed to transient communication
 Closely coupled to TCP/IP networks
 Not suitable for proprietary protocol for high-speed interconnection
network.
 MPI assumes that comm takes places within a
 Middleware vendors thus provide a higher-level of known group of processes
abstraction
 (groupID, processID) instead of a transport-level
 Every vendor did their own and cause portability problems
address in socket
 Solution: The “Message-Passing Interface” (MPI).

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Some MPI Primitives MPI Primitives
Primitive Meaning  MPI_bsend =
MPI_bsend Append outgoing message to a local send buffer  Transient asynchronous comm
MPI_send Send a message and wait until copied to local or remote buffer  MPI_send, the semantics depends on implmentation
MPI_ssend Send a message and wait until receipt starts  MPI_senduntil copies to local buffer =
 Receipt-based transient synch comm
MPI_sendrecv Send a message and wait for reply
 MPI_send until copies to remote buffer =
MPI_isend Pass reference to outgoing message, and continue  Delivery-based transient synch comm
MPI_issend Pass reference to outgoing message, and wait until receipt starts  MPI_ssend =
MPI_recv Receive a message; block if there are none  Delivery-based transient synch comm
MPI_irecv Check if there is an incoming message, but do not block  MPI_sendrecv =
 Response-based transient synch comm
Some of the more intuitive (and useful) message-passing primitives
(Note: there are many more in the API).

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Message-Oriented Persistent
Communication Message-Queuing Models
 It offers intermediate-term storage capacity for messages
 Also known as: Message Queuing Systems,  Without requiring either the sender or receiver active during message
or Message Oriented Middleware (MOM) transmission
 Apps communicate by inserting messages into specific queues
 Support persistent, asynchronous comms  Forwarded to destination, even if dest. down when message was sent.
Addressing by a system-wide unique name of the destination
 Basic idea: processes communicate through 
queue
support of middleware queues.  Only guarantee: your message will eventually make it into the
 Queues are buffers at communication servers. receiver’s message queue
 No guarantee about when, or even if the message will be actually read
 Key Example: IBM MQSeries  Typically, transport can take minutes as opposed to
seconds/milliseconds in Sockets or MPI

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Message Queuing Interface Message-Queuing Models System is storing (and possibly transmitting)
message while sender and receiver are passive

Primitive Meaning
Put Append a message to a specified queue (nonblocking)
Block until the specified queue is nonempty, and remove
Get
the first message (blocking)
Check a specified queue for messages, and remove the
Poll
first. Never block. (nonblocking)
Install a handler in receiver to be called when a message
Notify
is put into the specified queue. (callback function)

Basic interface to a queue in a message-queuing system: this is a very Four combinations for “loosely-coupled” communications which use
simple, yet extremely powerful abstraction. message-queues.

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

General Architecture of a Message- General Architecture of a Message-

Queuing System Queuing System (Cont.)
 Messages are put into a source queue  Sol. Relay: special queue manager operates as
 MOM must route to a destination queue. router
 Queues must be mapped to network locations.
 A database of queue names to network locations, see  Instead of using a naming service that maintains a
next Figure queue name to network location mapping
 Problem
 No general naming service in many message-queuing system to
 See next two Figures
maintain queue-to-location mapping
 Each queue manager needs a copy of the queue-to-location
mapping
 Lead to management problem
 DDeDe ManagerDmanage the queue

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 General Architecture of a Message-Queuing General Architecture of a Message-Queuing
System (1) System (2)

The relationship between queue-level addressing and network-level The general organization of a message-queuing system with routers
addressing. (relays).

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Message Brokers Message Broker Organization

 Often, there’s a need to integrate new/existing apps
into a “single, coherent Distributed Information
System”.
 Problem: different message formats exist in legacy
systems
 It may not be convenient to “force” legacy systems to
adhere to a single, global message format (cost!?).
 It is often necessary to live with diversity.
The general organization of a message broker in a message-queuing
system – also known variously as an “interface engine”.

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Message Brokers (Cont.) A Note on Message-Queuing System
 But the message format must be understood by  Message-queuing is different from email system
both sender and receiver  Email is targeted for end users
 MQ is a general-purpose interprocess comm and holds lots
 By the “Message Broker”. features
 Message broker: act as an application gateway  Message priorities, logging facilities, efficient multicasting, fault-
tolerance
to convert incoming messages to a format
 D oDDDmDorDDDDD
D D DDDlDDDDD
oDDreDD
understood by the receiver  The integration of a widely dispersed collection of DDDDbDDe
 By a database of rules DDDlDDDDDoDD(which is impossible to do with traditional
RPC/RMI techniques).

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Example: IBM MQSeries IBM MQSeries Organization

 Popularly used in IBM mainframes
 Especially for finance applications
 All queues are managed by Queue Manager (QM)
 Message Channel: a unidirectional, reliable
connection between a sending and a receiving queue
manager
 Each of the 2 ends of a MC is managed by message
channel agent (MCA)

General organization of IBM's MQSeries message-queuing system.

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 IBM MQSeries: Message Channels Message Transfer in MQSeries
 An address consists of (name of QM, name of Destination
Attribute Description
Queue)
Transport type Determines the transport protocol to be used.  A QM may consists of many queues
FIFO delivery Indicates that messages are to be delivered in the order they are sent.  Routing table entry in QM = (destQM, next_sendQ)
Message length Maximum length of a single message.  QM’s name is systemwide unique
Setup retry
Specifies maximum number of retries to start up the remote MCA.  However, when replace a QM or change a QM’s name affects all apps
count that send messages to this QM
Delivery retries Maximum times MCA will try to put received message into queue.  Solution: local alias: a logical name of a QU
 By local alias
Some attributes associated with message channel agents  Change the name of a QM requires change its alias in all queue
(MCA). managers
 But the applications can be left unaffected

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Alias and Routing Table IBM MQSeries: Message Transfer

Local alias of a QM
Primitive Description

QM’s real name MQopen Open a (possibly remote) queue.

MQclose Close a queue.
MQput Put a message into an opened queue.
MQget Get a message from a (local) queue.

Primitives available in an IBM MQSeries MQI.

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Chapter 2.5: Stream-Oriented
Communications Stream-Oriented Communications
 Continuous Media Support  With RPC, RMI and MOM, timing has no effect on
correctness.
 However, audio and video are time-dependent data streams
 If the timing is off, the resulting “output” from the system will be
 Quality of Service (QoS) incorrect.
 Time-dependent information – known as “continuous media”
communications.

 Stream Synchronization  Discrete (representation) media

 E.g. text, still images, object code
 Continuous (representation) media
 Temporal relationships between data are crucial

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Transmission Modes Two Types of Streams

 Data streams: a sequence of data units  Simple Streams: one single sequence of data
 Asynchronous transmission mode
 The data stream is transmitted in order, but there’s no timing  E.g. voice
constraints placed on the actual delivery (e.g., file transfer).
 Synchronous transmission mode
 Complex Streams: several sequences of data
 The maximum end-to-end delay is defined (but data can travel faster). (substreams) that are “related” by time
 Isochronous transmission mode  E.g. a lip-synchronized movie, with sound and
 Data transferred “on time”
 i.e., there’s a maximum and minimum end-to-end delay, known as
pictures, together with sub-titles
“bounded jitter” (e.g., multimedia systems)  This leads to data synchronization problems,
 The type of steam discussed in this chapter
which are not at all easy to deal with

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Virtual Connections Multicasting Stream
A stream can be considered as a virtual connection between a source and a sink

Source and sink

are processes

Source and sink

are devices

Multiparty communication: a stream is multicast to several receivers

However, each receiver have different QoS requirements.
(a) Setting up a stream between two processes across a network Thus, we need a filters that adjust the quality of incoming stream
(b) Setting up a stream directly between two devices
PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Quality of Service (QoS) Specifying QoS with Flow Specs.

 Time-dependent requirements can be expressed as
QoS requirements DDDDDDDDDiDDiDDDDDDDDnDuD DDDDiDDDDDuiDDD

 Definition: “ensuring that the temporal relationships maximum data unit size (bytes) Loss sensitivity (bytes)
in the stream can be preserved”. Token bucket rate (bytes/sec) Loss interval (sec)
 QoS is all about three things: Toke bucket size (bytes) Burst loss sensitivity (data units)
Maximum transmission rate Minimum delay noticed (sec)
 (a) Timeliness, (b) Volume and (c) Reliability (bytes/sec) Maximum delay variation (sec)
 But, how is QoS actually specified? Quality of guarantee
 One approach: expressing QoS by flow specification
 Specify bandwidth requirements, transmission rates, delays..
 Ref. the next slide A flow specification – one way of specifying QoS

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 An Approach to Implementing QoS:
Token Bucket Algorithm Token Bucket Algorithm
 In flow spec. the characteristics of the input stream are
formulated by Token bucket algorithm
 Specify how the stream will shape its network traffic
 Token: a fixed number of bytes that an app is allowed
to pass to network
 Tokens are buffered in a bucket with limited capacity
 Tokens are dropped when bucket is full,
 Data are passed to network at a relatively constant rate
The principle of a token bucket algorithm – a “classic” technique for
controlling the flow of data (and implementing QoS characteristics).

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Service Required in Flow Specs Flow Specifications

 In addition to the temporal relations by token bucket  Problem
algorithm, a flow spec. also consists of service requirements
 Loss sensitivity and loss interval  An application may simple not know its own
 Specify a maximum acceptable loss rate would be requirements
 Burst loss sensitivity
 How many consecutive data units may be lost
 So, another solution
 Minimum delay noticed  Classify a stream and provide defauls for detailed
 How long the network can delay the delivery of a data unit before the flow specification
receiver notices the delay
 Maximum delay variation  For example:
 The maximum tolerated jitter  Specify only whether a stream is for audio or video
 Quality of guarantee  Specify the quality is high, medium, or low
 How serious the service requirements should be taken (hard, firm, or
soft)

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Setting up of a Stream RSVP
 After flow specification, DS should allocate resource  Resource reSerVation Protocol (RSVP)
 A transport-level protocol for enabling resources reservation in
to set up a stream, including network routers
 Bandwidth, buffers, and processing capacity  First, senders in RSVP
 Set up a path to receivers.
 Which are all needed in order to realize QoS guarantees  Then provide a flow specification and send it to each intermediate node
 However, it is not easy that a model can  RSVP is a receiver-initiated protocol
 Receiver, when ready to receive, places a reservation request along
 Specify a QoS parameters its upstream path to sender
 Generically describe resources in any communication  Reservation can be lower than sender’s specification
system  RSVP process reserves resources if enough
 By admission control module
 Translate QoS parameters to resource usage  RSVP process will translate QoS parameters into things that make
sense to the data-link layer

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

RSVP Protocol Stream Synchronization

 Synchronization
 Maintain temporal relations between streams
Check whether the receiver has
permission to make the reservation  Two synchronization forms
 Between a discrete data stream and a continuous
data stream
 E.g. a slide show with audio
 Between continuous data streams
 E.g. playing a movie (lip synchronization)

The basic organization of RSVP for resource reservation in a distributed system.

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]
 Stream Synchronization (Cont.) Receive-Side with Application-Aware
 Synchronization mechanism
 Where does the synchronization occur?
 On the sending side or receiving side?
 Application transparency?
 Application take care by itself or by middleware

The principle of explicit synchronization on the level data units for multiple streams
(sub-streams).

PDF created with FinePrint pdfFactory Pro trial version [Link] PDF created with FinePrint pdfFactory Pro trial version [Link]

Receiver-Side with Application

Transparency

The principle of synchronization as supported by high-level interfaces

built as a set of “multimedia middleware streaming services”.

PDF created with FinePrint pdfFactory Pro trial version [Link]