Chapter 5

TCP Client/Server Example

TCP Client-Server Example

TCP echo server: main and str_echo TCP echo client: main and str_cli Normal startup and termination POSIX signal handling Handling SIGCHILD, interrupted system calls, and preventing zombies Connection abort before accept returns Crash of server process

SIGPIPE signal Crash, reboot, shutdown of server host Summary of TCP example Data format: passing string or binary

TCP Echo Server and Client

To expand this example to other applications, just change what the server does with the client input. Many boundary conditions to handle: signal, interrupted system call, server crash, etc. The first version does not handle them.

TCP Echo Server main() Algorithm

A typical fork()-based concurrent server Algorithm outline :

Create socket Bind it to a designated port (supposedly to be a well-known port) Allow incoming traffic for any local network interface (wildcard address: INADDR_ANY) Convert it to a listening socket

Set up a listening queue Block in call to accept(), wait for a client to connect Spawn a child to handle each client upon successful connection

Loop around forever :

Close listening socket Execute str_echo()

Close connected socket for child. parent no wait

TCP Echo Server: main function

#include "unp.h" tcpcliserv/tcpserv01.c int main(int argc, char **argv) { int listenfd, connfd; pid_t childpid; socklen_t clilen; struct sockaddr_in cliaddr, servaddr; listenfd = Socket(AF_INET, SOCK_STREAM, 0); bzero(&servaddr, sizeof(servaddr)); 9877 servaddr.sin_family = AF_INET; servaddr.sin_addr.s_addr = htonl(INADDR_ANY); servaddr.sin_port = htons(SERV_PORT); Bind(listenfd, (SA *) &servaddr, sizeof(servaddr));

Listen(listenfd, LISTENQ); for ( ; ; ) { clilen = sizeof(cliaddr); connfd = Accept(listenfd, (SA *) &cliaddr, &clilen); if ( (childpid = Fork()) == 0) { /* child process */ Close(listenfd); /* close listening socket */ str_echo(connfd); /* process the request */ exit(0); } Close(connfd); /* parent closes connected socket*/ } }

str_echo() Algorithm

It provides very simple service for each client

It reads data from a client and echoes it back to the client

Algorithm outline :

Read a buffer from the connected socket

If n (number of characters read) > 0,

Echo back to the client (writen(): p. 89), read again

Else if n < 0 & EINTR (got interrupt), read again Else just n < 0 (error occurred), display error message (and terminate child process in err_sys()) Else if n = 0 (receipt of FIN from client, the normal scenario), return
8

TCP Echo Server: str_echo function

#include "unp.h" lib/str_echo.c void str_echo(int sockfd) { ssize_t n; char line[MAXLINE]; for ( ; ; ) { if ( (n = Readline(sockfd, line, MAXLINE)) == 0) return; /* connection closed by other end */ Writen(sockfd, line, n); } }

TCP Echo Client main() Algorithm

Algorithm outline :

Check number of commandline arguments

It must be 2 (program name and server address) Quit if not 2 (call to sys_quit())

Open socket Fill in internet socket address structure Connect to server Call str_cli() to handle the rest of the client processing Exit when no more user input

Note: All errors end up in termination of the client in this function. Real applications may need 10 to recover differently

TCP Echo Client: main function

#include "unp.h" tcpcliserv/tcpcli01.c int main(int argc, char **argv) { int sockfd; struct sockaddr_in servaddr; if (argc != 2) err_quit("usage: tcpcli <IPaddress>"); sockfd = Socket(AF_INET, SOCK_STREAM, 0); bzero(&servaddr, sizeof(servaddr)); servaddr.sin_family = AF_INET; servaddr.sin_port = htons(SERV_PORT); Inet_pton(AF_INET, argv[1], &servaddr.sin_addr); Connect(sockfd, (SA *) &servaddr, sizeof(servaddr)); str_cli(stdin, sockfd); /* do it all */ exit(0); }

TCP Echo Client: str_cli function

#include "unp.h" void str_cli(FILE *fp, int sockfd) { char sendline[MAXLINE], recvline[MAXLINE]; while (Fgets(sendline, MAXLINE, fp) != NULL) { Writen(sockfd, sendline, strlen(sendline)); if (Readline(sockfd, recvline, MAXLINE) == 0) err_quit("str_cli: server terminated prematurely"); Fputs(recvline, stdout); } lib/str_cli.c

Normal Startup (1/2)

To watch the sequence of client/server To start the server in background :

linux% tcpserv01 & [1] 17870

To check the status of all sockets on a system (-a) before the client starts :
linux% netstat a Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 *:9877 *:* LISTEN Note : The output above shows only partial results, and the output format may be different from system to system
13

Normal Startup (2/2)

To start the client on the same machine (using the loopback address) :
linux% tcpcli01 [Link]

Then, check the status of all sockets again :

linux% netstat a Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 localhost:9877 localhost:42758 ESTABLISHED tcp 0 0 localhost:42758 localhost:9877 ESTABLISHED tcp 0 0 *:9877 *:* LISTEN Note : The first tcp connection is for the server child, and the second is for the client, and the third is the server parent
14

Normal Termination (1/2)

Fgets

Clientstr_cli exitmain Clientsocketkernel ClientTCPFINserver serverTCPACK. Clientserver childconnection

Normal Termination (2/2)

server TCPFINserver child readlinereadline0 server childstr_echomain Server childexitserver TCP (server childclient) Server childzombie

Zombie Process

()

Cleaning Up Zombie Processes

Child processzombie Zombiechild parentkernel Parentzombie kernelsignal(SIGCHLD)

POSIX Signal Handling

Signal (software interrupt): sent by one process to another process (or to itself) or by the kernel to a process SIGCHLD: by the kernel to the parent when a child process is terminated Disposition of a signal:

catch the signal by a specified signal handler SIG_IGN: ignore it SIG_DFL: default: terminate or ignore

signal Function That Enables System Call Restart

#include "unp.h" lib/signal.c Sigfunc *signal(int signo, Sigfunc *func) { Posixsignal disposition struct sigaction act, oact; sigaction act.sa_handler = func; signal sigemptyset(&act.sa_mask); sigaction act.sa_flags = 0; if (signo == SIGALRM) { #ifdef SA_INTERRUPT act.sa_flags |= SA_INTERRUPT; /* SunOS 4.x */ #endif } else { #ifdef SA_RESTART act.sa_flags |= SA_RESTART; /* SVR4, 44BSD */ #endif } if (sigaction(signo, &act, &oact) < 0) return(SIG_ERR); singal return(oact.sa_handler); system callrestart }

Signal Function That Enables System Call Restart (cont.)

Sigfunc *Signal(int signo, Sigfunc *func) { Sigfunc *sigfunc; if ( (sigfunc = signal(signo, func)) == SIG_ERR) err_sys("signal error"); return(sigfunc); } Signalsignal

POSIX signal semantics: 1. Once a signal handler is installed, it remains installed. 2. The signal being delivered is blocked while a signal handler is executing. 3. By default, signals are not queued.

Handling Zombies
Whenever we fork children, we must wait for them to prevent them from becoming Zombies To do this, we establish a signal handler to cache SIGCHLD

and within the handler we call wait.

Signal Handler for SIGCHLD

tcpcliserv/sigchldwait.c #include "unp.h" void wait sig_chld(int signo) child { zombie pid_t pid; int stat; pid = wait(&stat); printf("child %d terminated\n", pid); return; } child process id

Interrupted System Calls

kernelSIGCHLD signalparent parentblockacceptsystem call system callinterrupthandler signal handler returninterrupted system callrestartsystem call (EINTR)

signalrestart system call

EINTRAccept (abort)

Handling Interrupted System Calls

Definition: Slow system call system call that can

block forever Some kernels automatically restart some interrupted system calls, while some dont We must prepare for slow system calls to return EINTR

for ( ; ; ){ clilen = sizeof (cliaddr); if ( (connfd = accept (listenfd, (SA) &cliaddr, &clilen)) < 0) { if (errno == EINTR) continue; /* back to for ( ) */ else err_sys (accept error); } restart accept system call

Weakness of Wait

Unix signals are normally not queued multiple occurrences of the same signal only cause the handler to be called once Its a problem when multiple children terminate at the same time Solution: use waitpid instead of wait in the handler to kill all zombies

Client terminates all five connections

catching all SIGCHLD signals in server parent

SIGCHLD Handler UsingWaitpid

#include "unp.h" void sig_chld(int signo) Wait for the first terminated child { Not block if there are pid_t pid; no terminated children int stat; while ( (pid = waitpid(-1, &stat, WNOHANG)) > 0) printf("child %d terminated\n", pid); return; } #include <sys/wait.h> pid_t waitpid (pid_t pid, int *statloc, int options); returns: process ID if OK, o, or -1 on error

Final (correct) Version of TCP Echo Server

handling SIGCHLD, EINTR from accept, zombies
#include "unp.h" int tcpcliserv/tcpserv04.c main(int argc, char **argv) { int listenfd, connfd; pid_t childpid; socklen_ t clilen; struct sockaddr_in cliaddr, servaddr; void sig_chld(int); listenfd = Socket(AF_INET, SOCK_STREAM, 0); bzero(&servaddr, sizeof(servaddr)); servaddr.sin_family = AF_INET; servaddr.sin_addr.s_addr = htonl(INADDR_ANY); servaddr.sin_port = htons(SERV_PORT); Bind(listenfd, (SA *) &servaddr, sizeof(servaddr)); Listen(listenfd, LISTENQ);

Final (correct) Version of TCP Echo Server (cont.)

Signal(SIGCHLD, sig_chld); /* must call waitpid() */ for ( ; ; ) { clilen = sizeof(cliaddr); if ( (connfd = accept(listenfd, (SA *) &cliaddr, &clilen)) < 0) { if (errno == EINTR) continue; /* back to for() */ else err_sys("accept error"); } if ( (childpid = Fork()) == 0) { /* child process */ Close(listenfd); /* close listening socket */ str_echo(connfd); /* process the request */ exit(0); } Close(connfd); /* parent closes connected socket */ } }

Connection Abort Before accept Returns

implementation dependent !

In BSD, kernel handles this. accept does not return. In SVR4, accept is returned with EPROTO. In POSIX.1g, accept is returned with ECONNABORTED The server can ignore the error and just call accept again

Crashing of Server Child Process

Procedure: 1. Kill the child. Server TCP sends FIN to client TCP, which responds with an ACK. (TCP half-close) (The client process is blocked in fgets when client TCP receives FIN.) 2. SIGCHLD signal is sent to the server parent. 3. User inputs. The client process calls writen to send data to server. 4. The server TCP responds with an RST. 5. The client process returns from readline, 0, when client TCP receives RST. 6. The client process terminates.
The client is blocked on the call to fgets when FIN is received. Client (it should block on input from either source) input Solution: Use select or poll to block on either socket or stdio

SIGPIPE Signal
when writing to a socket that has received an RST

Procedure: 1. The client writes to a crashed server process. An RST is received at the client TCP and readline returns 0 (EOF). 2. If the client ignores the error returned from readline and write more, SIGPIPE is sent to the client process. 3. If SIGPIPE is not caught, the client terminates with no output
Problem: Nothing is output even by the shell to indicate what has happened. (Have to use echo $?to examine the shells return value of last command.) Solution: 1. Setting the signal disposition to SIG_IGN 2. Catch the SIGPIPE signal for further processing. (handle EPIPE error returned from write).

An Example to Show SIGPIPE

To invoke tcpcli11 which has two write operations to show an example of writing to a closed socket

The first write to the closed socket is to solicit RST from the server TCP The second write is to generate SIGPIPE from the local process. An sample run :
linux% tcpcli11 [Link] Hi there # user input in bold Hi there # echoed back from server # terminate server child process then Bye # then type this line purposely Borken pipe # output by the shell because of SIGPIPE

Note: To write to a socket which has received a FIN is OK. However, it is an error to write to a socket hat has received an RST 34

str_cli() Calling writen() Twice

tcpcliserv/str_cli11.c

35

Crash of Server Host

Scenario

client TCP continuously retx data and timeout around 9 min

1. client and server run on different hosts 2. make a connection between client and server 3. client types something to transmit data to the server 4. disconnect the server host from the network (destination unreachable) 5. client types something again.

The client process will then return with the error ETIMEDOUT. If some intermediate router determined that the server host was down and responded with an ICMP destination unreachable message, the error returned will then be either EHOSTUNREACH or ENETUNREACH

To quickly detect: timeout on readline, SO_KEEPALIVE socket option, heartbeat functions

Reboot of Server Host

The client does not see the server host shut down Client sends data to server after the server reboots server TCP responds to client data with an RST because it loses all connection information readline returns ECONNRESET

Shutdown of Server Host (by Operator)

init process sends SIGTERM to all processes

We can catch this signal and close all open descriptors by ourselves

init waits 5-20 sec and sends SIGKILL to all processes

all open descriptors are closed by kernel

Summary of TCP Example

From clients perspective:

socket and connect specifies servers port and IP client port and IP chosen by TCP and IP respectively socket and bind specifies servers local port and IP listen and accept return clients port and IP

From servers perspective:

TCP Client/Server Clients Perspective

40

TCP Client/Server Clients Perspective

41

Data Format: Text Strings

server process gets two numbers (in a line of text) from client and returns their sum In str_echo: sscanf converts string to long integer, snprintf converts long back to string

str_echo() Adding 2 Numbers

tcpcliserv/str_echo08.c

43

Data Format: Binary Structure

Passing binary structure between client and server does not work

when the client and server are run on hosts with different byte orders or sizes of long integer Since different implementations can store the same C datatype differently. pass in string only explicitly define the format of data types (e.g. RPCs XDR -- external data representation)

Suggestions:

str_cli() Sending 2 Binary Ints

tcpcliserv/str_cli09.c

45

str_echo() Adding 2 Binary Ints

tcpcliserv/str_echo09.c

46

Beware of Different Byte Orders

Due to the big-endian and little-endian implementations, sending binary numbers between different machine architectures may end up with different results

An example of two big-endian SPARC machines :

solaris% tcpcli09 [Link] 11 12 # user input in bold 33 # result back from server -11 -14 -55 linus% tcpcli09 [Link] 12 # user input in bold 3 # It seems to work -22 -77 -16777314 # oops! It does not work! 47

An example of big-endian SPARC and little-endian Intel machines :