Front cover

Draft Document for Review March 21, 2011 7:59 am SG24-7896-00

IBM z/OS V1R12 Communications Server

TCP/IP Implementation Volume 1:
Base Functions, Connectivity, and Routing

Discusses important z/OS Communications

Server TCP/IP base function capabilities

Includes z/OS Communications Server

base function implementation examples

Provides useful verification

techniques

Mike Ebbers
Rama Ayyar
Octavio L. Ferreira
Gazi Karakus
Yukihiko Miyamoto
Joel Porterie
Andi Wijaya

[Link]/redbooks
Draft Document for Review March 21, 2011 7:59 am [Link]

International Technical Support Organization

TCP/IP Implementation Volume 1: Base Functions,

Connectivity, and Routing

February 2011

SG24-7896-00
[Link] Draft Document for Review March 21, 2011 7:59 am

Note: Before using this information and the product it supports, read the information in “Notices” on
page xi.

First Edition (February 2011)

This edition applies to Version 1, Release 12 of Communications Server for z/OS (product number 5694-A01).

This document created or updated on March 21, 2011.

© Copyright International Business Machines Corporation 2011. All rights reserved.

Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule
Contract with IBM Corp.
Draft Document for Review March 21, 2011 7:59 am [Link]

iii
[Link] Draft Document for Review March 21, 2011 7:59 am

iv TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am [Link]

Contents

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
The team who wrote this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv
Now you can become a published author, too! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
Stay connected to IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi

Chapter 1. Introduction to Communications Server for z/OS IP . . . . . . . . . . . . . . . . . . . 1

1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.1 Basic concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Featured functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Communications Server for z/OS IP implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3.1 Functional overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3.2 Operating environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3.3 Reusable address space ID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3.4 Protocols and devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3.5 Supported routing applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.3.6 Application programming interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.3.7 z/OS Communications Server applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.3.8 UNIX System Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.4 Additional information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Chapter 2. The resolver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.1 Basic concepts of the resolver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2 The resolver address space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.2.1 The resolver SETUP data set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.2.2 The resolver configuration file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.2.3 Local hosts file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2.4 Resolver DNS cache. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.2.5 Criteria for indicating an unresponsive DNS name server . . . . . . . . . . . . . . . . . . 33
2.2.6 Unresponsive DNS name servers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.2.7 Affinity servers and generic servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.2.8 Resolving an IPv6 address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.2.9 Resolver support for EDNS0. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.2.10 Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.3 Implementing the resolver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.3.1 Implementation tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.3.2 Activation and verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
2.4 Problem determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.5 Additional information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Chapter 3. Base functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

3.1 The base functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.1.1 Basic concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.2 Common design scenarios for base functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.2.1 Single stack environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.2.2 Multiple stack environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

© Copyright IBM Corp. 2011. All rights reserved. v

[Link] Draft Document for Review March 21, 2011 7:30 am

3.2.3 Recommendation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.2.4 Recommendations for MTU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
3.3 z/OS UNIX System Services setup for TCP/IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.3.1 RACF actions for UNIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.3.2 APF authorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.3.3 Changes to [Link] members. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.3.4 Changes to [Link] members. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.3.5 Additional z/OS customization for z/OS UNIX. . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.3.6 TCP/IP server functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.3.7 TCP/IP client functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.3.8 UNIX client functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.3.9 Verification checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.4 Configuring z/OS TCP/IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.4.1 TCP/IP configuration data set names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.4.2 [Link] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.4.3 VTAM Resource . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
3.4.4 [Link] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3.4.5 Configuring the local hosts file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3.5 Implementing the TCP/IP stack. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.5.1 Create [Link] file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.5.2 Create the [Link] file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.5.3 Check BPXPRMxx . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.5.4 Create TCP/IP cataloged procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.5.5 Add RACF definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.5.6 Create a VTAM TRL major node for MPCIPA OSA . . . . . . . . . . . . . . . . . . . . . . . 93
3.6 Activating the TCP/IP stack. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.6.1 UNIX System Services verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.6.2 Verifying TCP/IP configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.7 Reconfiguring the system with z/OS commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
3.7.1 Deleting a device and adding or changing a device . . . . . . . . . . . . . . . . . . . . . . 111
3.7.2 Modifying a device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
3.8 Job log versus syslog as diagnosis tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
3.9 Message types: Where to find them . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
3.10 Additional information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

Chapter 4. Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

4.1 What is connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
4.1.1 System z network connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
4.2 Recommended interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
4.2.1 High-bandwidth and high-speed networking technologies . . . . . . . . . . . . . . . . . 120
4.2.2 OSA-Express (MPCIPA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
4.2.3 OSA-Express for zEnterprise (z196). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
4.2.4 HiperSockets (MPCIPA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
4.2.5 Dynamic XCF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
4.3 Connectivity for the z/OS environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
4.3.1 IOCP definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
4.3.2 VTAM definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
4.4 OSA-Express QDIO connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
4.4.1 Dependencies: CHPID, IOCDS, port numbers, portnames, and port sharing . . 140
4.4.2 Considerations for isolating traffic across a shared OSA port. . . . . . . . . . . . . . . 147
4.4.3 Configuring OSA-Express with VLAN ID. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
4.4.4 Verifying the connectivity status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
4.5 OSA-Express QDIO connectivity with Connection Isolation . . . . . . . . . . . . . . . . . . . . 156

vi TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am [Link]

4.5.1 Description of Connection Isolation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

4.5.2 Dependencies for Connection Isolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
4.5.3 Considerations for Connection Isolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
4.5.4 Configuring OSA-Express with Connection Isolation . . . . . . . . . . . . . . . . . . . . . 162
4.5.5 Verifying Connection Isolation on OSA2080X. . . . . . . . . . . . . . . . . . . . . . . . . . . 163
4.5.6 Conclusions and recommendations: best practices for isolating traffic. . . . . . . . 180
4.6 HiperSockets connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
4.6.1 Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
4.6.2 Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
4.6.3 Configuring HiperSockets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
4.6.4 Verifying the connectivity status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
4.7 Dynamic XCF connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
4.7.1 Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
4.7.2 Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
4.7.3 Configuring DYNAMICXCF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
4.7.4 Verifying connectivity status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
4.8 Controlling and activating devices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
4.8.1 Starting a device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
4.8.2 Stopping a device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
4.8.3 Activating modified device definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
4.9 Problem determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
4.10 Additional information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

Chapter 5. Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

5.1 Basic concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
5.1.1 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
5.1.2 Direct routes, indirect routes, and default route . . . . . . . . . . . . . . . . . . . . . . . . . 207
5.1.3 Route selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
5.1.4 Static routing and dynamic routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
5.1.5 Choosing the routing method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
5.2 Routing in the z/OS environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
5.2.1 Static routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
5.2.2 Dynamic routing using OMPROUTE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
5.2.3 Policy-based routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
5.3 Dynamic routing protocols. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
5.3.1 Open Shortest Path First . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
5.3.2 Routing Information Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
5.3.3 IPv6 dynamic routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
5.4 Implementing static routing in z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
5.4.1 Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
5.4.2 Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
5.4.3 Implementation tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
5.4.4 Activation and verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
5.5 Implementing OSPF routing in z/OS with OMPROUTE . . . . . . . . . . . . . . . . . . . . . . . 233
5.5.1 Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
5.5.2 Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
5.5.3 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
5.5.4 Implementation tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
5.5.5 Configure routers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
5.5.6 Activation and verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
5.5.7 Managing OMPROUTE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
5.6 Problem determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
5.6.1 Commands to diagnose networking connectivity problems . . . . . . . . . . . . . . . . 254

Contents vii
[Link] Draft Document for Review March 21, 2011 7:30 am

5.6.2 Diagnosing an OMPROUTE problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256

5.7 Additional information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264

Chapter 6. VLAN and Virtual MAC support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265

6.1 Virtual MAC overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
6.1.1 Why use virtual MACs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
6.1.2 Virtual MAC concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
6.1.3 Virtual MAC address assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
6.2 Virtual MAC implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
6.2.1 IP routing when using VMAC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
6.2.2 Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
6.3 Virtual LAN overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
6.3.1 Types of connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
6.4 VLAN implementation on z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
6.4.1 Single VLAN per OSA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
6.4.2 Multiple VLAN support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
6.4.3 Multiple VLANs configuration guidelines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
6.4.4 Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
6.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

Chapter 7. Sysplex subplexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
7.2 Subplex environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
7.3 Load Balancing Advisor and subplexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
7.4 Subplex implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
7.4.1 XCF group names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
7.4.2 TCP/IP structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
7.4.3 Subplex 11: Internal subplex. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
7.4.4 Subplex 22: External subplex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
7.4.5 Access verifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
7.4.6 LBA connected to a subplex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
7.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298

Chapter 8. Diagnosis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

8.1 Debugging a problem in a z/OS TCP/IP environment. . . . . . . . . . . . . . . . . . . . . . . . . 300
8.1.1 An approach to problem analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
8.2 Logs to diagnose CS for z/OS IP problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302
8.3 Useful commands to diagnose CS for z/OS IP problems . . . . . . . . . . . . . . . . . . . . . . 303
8.3.1 The ping command (TSO or z/OS UNIX) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303
8.3.2 traceroute command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
8.3.3 The netstat command (console, TSO, or z/OS UNIX) . . . . . . . . . . . . . . . . . . . . 307
8.3.4 NETSTAT Catalog validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
8.3.5 Timestamp validation for NETSTAT catalogs . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
8.4 Gathering traces in CS for z/OS IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
8.4.1 Taking a component trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
8.4.2 Event trace for TCP/IP stacks (SYSTCPIP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
8.4.3 Packet trace (SYSTCPDA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
8.4.4 OMPROUTE trace (SYSTCPRT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
8.4.5 Resolver trace (SYSTCPRE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
8.4.6 IKE daemon trace (SYSTCPIK) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
8.4.7 Intrusion detection services trace (SYSTCPIS) . . . . . . . . . . . . . . . . . . . . . . . . . 330
8.4.8 OSAENTA trace (SYSTCPOT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
8.4.9 Queued Direct I/O Diagnostic Synchronization. . . . . . . . . . . . . . . . . . . . . . . . . . 331
8.4.10 Network security services server trace (SYSTCPNS). . . . . . . . . . . . . . . . . . . . 331

viii TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am [Link]

8.4.11 Obtaining component trace data with a dump. . . . . . . . . . . . . . . . . . . . . . . . . . 332

8.4.12 Analyzing a trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
8.4.13 Configuration profile trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
8.5 OSA-Express3 Network Traffic Analyzer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
8.5.1 Determining the microcode level for OSA-Express3. . . . . . . . . . . . . . . . . . . . . . 333
8.5.2 Defining TRLE definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
8.5.3 Checking TCPIP definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
8.5.4 Customizing OSA-Express Network Traffic Analyzer . . . . . . . . . . . . . . . . . . . . . 336
8.5.5 Defining a resource profile in RACF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
8.5.6 Allocating a VSAM linear data set. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
8.5.7 Starting the OSAENTA trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
8.5.8 Operator command to query and display OSA information. . . . . . . . . . . . . . . . . 352
8.5.9 OSM and OSX information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354
8.6 Additional tools for diagnosing CS for z/OS IP problems . . . . . . . . . . . . . . . . . . . . . . 355
8.6.1 Network Management Interface API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
8.6.2 Systems Management Facilities accounting records . . . . . . . . . . . . . . . . . . . . . 357
8.7 MVS console support for selected TCP/IP commands . . . . . . . . . . . . . . . . . . . . . . . . 360
8.7.1 Concept. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360
8.7.2 Commands and environments supported by EZACMD . . . . . . . . . . . . . . . . . . . 361
8.7.3 When to use EZACMD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
8.7.4 How to use the EZACMD command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
8.7.5 Configuring z/OS for using the EZACMD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
8.7.6 Using the EZACMD command in the z/OS console . . . . . . . . . . . . . . . . . . . . . . 363
8.7.7 Preparing the EZACMD command in z/OS TSO and z/OS NetView . . . . . . . . . 364
8.7.8 Using EZACMD command from z/OS TSO . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364
8.7.9 Integrating EZACMD into REXX programs in TSO and NetView . . . . . . . . . . . . 366
8.7.10 Protecting the EZACMD command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366
8.7.11 Diagnosis: diagnosing the EZACMD command . . . . . . . . . . . . . . . . . . . . . . . . 368
8.8 Additional information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369

Chapter 9. z/OS in an ensemble. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371

9.1 Basic concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372
9.2 Connectivity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372
9.2.1 Intranode management network (INMN). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372
9.2.2 Intraensemble data network (IEDN) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
9.3 Enabling z/OS as a member of the ensemble. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
9.3.1 Enabling z/OS for IPv6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
9.3.2 Enabling VTAM for the ensemble . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
9.3.3 Validating the ensemble interfaces in z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
9.3.4 Displaying information about the OSM interfaces . . . . . . . . . . . . . . . . . . . . . . . . 378
9.4 Defining and activating the z/OS ensemble interfaces . . . . . . . . . . . . . . . . . . . . . . . . 380
9.4.1 Displaying information about the OSX interfaces . . . . . . . . . . . . . . . . . . . . . . . . 382
9.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384

Appendix A. IPv6 support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385

Overview of IPv6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
Importance of IPv6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
Common design scenarios for IPv6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
Tunneling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
Dedicated data links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
MPLS backbones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388
Dual-stack backbones. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388
Dual-mode stack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389

Contents ix
[Link] Draft Document for Review March 21, 2011 7:30 am

Recommendation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
How IPv6 is implemented in z/OS Communications Server. . . . . . . . . . . . . . . . . . . . . . . . 389
IPv6 addressing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
Stateless address autoconfiguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390
IPv6 TCP/IP Network part (prefix). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
IPv6 implementation in z/OS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405

Appendix B. Additional parameters and functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 411

MVS System symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412
Reusable Address Space ID (REUSASID) function examples . . . . . . . . . . . . . . . . . . . . . 415
[Link] statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418
IPCONFIG statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418
GLOBALCONFIG statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420
PORT statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424
IDYNAMICXCF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
SACONFIG (SNMP subagent) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
SMFCONFIG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428
Netmonitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429
INTERFACE statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430
DEVICE and LINK statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438
SRCIP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
TCP/IP built-in security functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441

Appendix C. Examples used in our environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443

Resolver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
TCP/IP stack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
OMPROUTE dynamic routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452

Appendix D. Our implementation environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457

The environment used for all four books . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458
Our focus for this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460

Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461

IBM Redbooks publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461
Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461
Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462
How to get IBM Redbooks publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463
Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465

x TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am [Link]

Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area. Any
reference to an IBM product, program, or service is not intended to state or imply that only that IBM product,
program, or service may be used. Any functionally equivalent product, program, or service that does not
infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to
evaluate and verify the operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter described in this document. The
furnishing of this document does not give you any license to these patents. You can send license inquiries, in
writing, to:
IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any other country where such
provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION
PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR
IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of
express or implied warranties in certain transactions, therefore, this statement may not apply to you.

This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may make
improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time
without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any
manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the
materials for this IBM product and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring
any obligation to you.

Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm the
accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the
capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.

COPYRIGHT LICENSE:

This information contains sample application programs in source language, which illustrate programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the sample
programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore,
cannot guarantee or imply reliability, serviceability, or function of these programs.

© Copyright IBM Corp. 2011. All rights reserved. xi

[Link] Draft Document for Review March 21, 2011 7:59 am

Trademarks
IBM, the IBM logo, and [Link] are trademarks or registered trademarks of International Business Machines
Corporation in the United States, other countries, or both. These and other IBM trademarked terms are
marked on their first occurrence in this information with the appropriate symbol (® or ™), indicating US
registered or common law trademarks owned by IBM at the time this information was published. Such
trademarks may also be registered or common law trademarks in other countries. A current list of IBM
trademarks is available on the Web at [Link]

The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:
AIX® NetView® System z®
BladeCenter® OMEGAMON® TDMF®
CICS® Parallel Sysplex® Tivoli®
ESCON® POWER® VTAM®
FICON® RACF® WebSphere®
HiperSockets™ Redbooks® z/OS®
IBM® Redbooks (logo) ® z/VM®
IMS™ RMF™ z/VSE™
Language Environment® System p® z10™
Lotus® System z10® z9®
MVS™ System z9®

The following terms are trademarks of other companies:

Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other
countries, or both.

UNIX is a registered trademark of The Open Group in the United States and other countries.

Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

Other company, product, or service names may be trademarks or service marks of others.

xii TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am [Link]

Preface

For more than 40 years, IBM® mainframes have supported an extraordinary portion of the
world’s computing work, providing centralized corporate databases and mission-critical
enterprise-wide applications. The IBM System z®, the latest generation of the IBM
distinguished family of mainframe systems, has come a long way from its IBM System/360
heritage. Likewise, its IBM z/OS® operating system is far superior to its predecessors in
providing, among many other capabilities, world class and state-of-the-art support for the
TCP/IP Internet protocol suite.

TCP/IP is a large and evolving collection of communication protocols managed by the Internet
Engineering Task Force (IETF), an open, volunteer organization. Because of its openness,
the TCP/IP protocol suite has become the foundation for the set of technologies that form the
basis of the Internet. The convergence of IBM mainframe capabilities with Internet
technology, connectivity, and standards (particularly TCP/IP) is dramatically changing the
face of information technology and driving requirements for even more secure, scalable, and
highly available mainframe TCP/IP implementations.

The z/OS Communications Server TCP/IP Implementation series provides understandable,

step-by-step guidance about how to enable the most commonly used and important functions
of z/OS Communications Server TCP/IP.

In this IBM Redbooks® publication, we provide an introduction to z/OS Communications

Server TCP/IP. We then discuss the system resolver, showing the implementation of global
and local settings for single and multi-stack environments. Next, we present implementation
scenarios for TCP/IP Base functions, Connectivity, Routing, Virtual MAC support, and sysplex
subplexing.

For more specific information about z/OS Communications Server standard applications, high
availability, and security, refer to the other volumes in the series:
 IBM z/OS Communications Server TCP/IP Implementation Volume 2: Standard
Applications, SG24-7897
 IBM z/OS V1R11 Communications Server TCP/IP Implementation Volume 3: High
Availability, Scalability, and Performance, SG24-7800
 Communications Server for z/OS TCP/IP Implementation Volume 4: Security and
Policy-Based Networking, SG24-7899

For comprehensive descriptions of the individual parameters for setting up and using the
functions described in this book, along with step-by-step checklists and supporting examples,
refer to the following publications:
 z/OS Communications Server: IP Configuration Guide, SC31-8775
 z/OS Communications Server: IP Configuration Reference, SC31-8776
 z/OS Communications Server: IP User’s Guide and Commands, SC31-8780

This book does not duplicate the information in those publications. Instead, it complements
them with practical implementation scenarios that can be useful in your environment. To
determine at what level a specific function was introduced, refer to z/OS Communications
Server: New Function Summary, GC31-8771. For complete details, we encourage you to
review the documents referred to in the additional resources section at the end of each
chapter.

© Copyright IBM Corp. 2011. All rights reserved. xiii

[Link] Draft Document for Review March 21, 2011 7:59 am

The team who wrote this book

This book was produced by a team of specialists from around the world working at the
International Technical Support Organization, Poughkeepsie Center.

Mike Ebbers is a Consulting IT Specialist and Project Leader at the International Technical
Support Organization, Poughkeepsie Center. He has worked with IBM mainframe hardware
and software products since 1974 in the field, in education, and in the ITSO.

Rama Ayyar is an independent IT Consultant based in Sydney, Australia. He is an exIBMer

with IBM Australia. He has also worked for CSC Australia in senior technical roles. He is one
of the founding members of HCL India. Rama has over 25 years of experience with the
MVS™ Operating System. His areas of expertise include TCP/IP, RACF®, DFSMS, z/OS,
HCD & Configuration Management, Dump Analysis, and Disaster Recovery. Rama has
co-authored seven IBM Redbooks. He holds a Master's Degree in Computer Science from
the Indian Institute of Technology, Kanpur. Rama has been in the computer industry for more
than 35 years.

Octavio L. Ferreira is a Senior IT Specialist in IBM Brazil. He has 28 years of experience in

IBM software support. His areas of expertise include z/OS Communications Server, SNA and
TCP/IP, and Communications Server on all platforms. For the last 10 years, Octavio has
worked at the Area Program Support Group, providing guidance and support to clients and
designing networking solutions such as SNA-TCP/IP Integration, z/OS Connectivity,
Enterprise Extender design and implementation, and SNA-to-APPN migration. He has also
co-authored other IBM Redbooks publications.

Gazi Karakus is a Network Specialist who has worked for Garanti Technology for four years.
He has six years of experience in the networking field. He has a [Link]. degree in Electronics
and Telecommunication Engineering from Istanbul Technical University. His areas of
expertise include routing and switching technologies, z/OS Communications Server, SNA and
TCP/IP, and Communications Server on other platforms.

Yukihiko Miyamoto is a Senior IT Specialist who has been with IBM Japan for over 14 years.
His areas of expertise are WAN, LAN, TCP/IP, and SNA, with a primary focus on router and
switch networking technologies. For more than five years, Yukihiko has been working with
z/OS Communications Server as a Technical Support member, providing consultation,
design, and implementation services for enterprise networking solutions to clients.

Joel Porterie is a Senior IT Specialist who has been with IBM France for over 33 years. He
works for Network and Channel Connectivity Services in the PSSC Product Support Group.
His areas of expertise include z/OS, TCP/IP, VTAM®, OSA-Express, and Parallel Sysplex®.
Joel has taught OSA-Express and FICON® problem determination classes and has provided
on site assistance in these areas in numerous countries. He has co-authored many other IBM
Redbooks publications.

Andi Wijaya is a Senior Systems Engineer in IBM-JTI Indonesia. His areas of expertise
include IT infrastructure management, networking, security, and open source based systems.
He is a trainer, consultant, and subject matter expert in Indonesia and also a public quality
assurance reviewer for other international books. For more than 10 years, Andi has been
working with networking solutions such as fault tolerant infrastructure, high performance
enterprise network, and end-to-end integrated security in network infrastructure.

Thanks to the following people for their contributions to this project:

Richard Conway
Robert Haimowitz

xiv TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am [Link]

Bill White
International Technical Support Organization, Poughkeepsie Center

Roy Brabson
IBM Communications Server Development, Raleigh

Thanks to the authors of the previous editions of this book:

Finally, we want to thank the authors of the previous z/OS Communications Server TCP/IP
Implementation series for creating the groundwork for this series: Rama Ayyar, Valirio Braga,
WenHong Chen, Demerson Cilloti, Sandra Elisa Freitag, Gwen Dente, Gilson Cesar de
Oliveira, Mike Ebbers, Octavio Ferreira, Marco Giudici, Adi Horowitz, Michael Jensen,
Shizuka Katoh, Sherwin Lake, Bob Louden, Garth Madella, Yukihiko Miyamoto, Hajime
Nagao, Shuo Ni, Carlos Bento Nonato, Yohko Ojima, Roland Peschke, Joel Porterie, Marc
Price, Frederick James Rathweg, Micky Reichenberg, Larry Templeton, Rudi van Niekerk, Bill
White, Thomas Wienert, Andi Wijaya, and Maulide Xavier.

Now you can become a published author, too!

Here's an opportunity to spotlight your skills, grow your career, and become a published
author—all at the same time! Join an ITSO residency project and help write a book in your
area of expertise, while honing your experience using leading-edge technologies. Your efforts
will help to increase product acceptance and customer satisfaction, as you expand your
network of technical contacts and relationships. Residencies run from two to six weeks in
length, and you can participate either in person or as a remote resident working from your
home base.

Find out more about the residency program, browse the residency index, and apply online at:
[Link]/redbooks/[Link]

Comments welcome
Your comments are important to us!

We want our books to be as helpful as possible. Send us your comments about this book or
other IBM Redbooks publications in one of the following ways:
 Use the online Contact us review Redbooks form found at:
[Link]/redbooks
 Send your comments in an email to:
redbooks@[Link]
 Mail your comments to:
IBM Corporation, International Technical Support Organization
Dept. HYTD Mail Station P099
2455 South Road
Poughkeepsie, NY 12601-5400

Preface xv
[Link] Draft Document for Review March 21, 2011 7:59 am

Stay connected to IBM Redbooks

 Find us on Facebook:
[Link]
 Follow us on Twitter:
[Link]
 Look for us on LinkedIn:
[Link]
 Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks
weekly newsletter:
[Link]
 Stay current on recent Redbooks publications with RSS Feeds:
[Link]

xvi TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:28 am 7896 [Link]

Chapter 1. Introduction to Communications

Server for z/OS IP
z/OS Communications Server is the IBM implementation of the standard TCP/IP protocol
suite on the z/OS platform. TCP/IP is a component product of the z/OS Communications
Server, and it provides a multitude of technologies. Collectively, those technologies provide
an Open Systems environment for the development, establishment, and maintenance of
applications and systems.

The z/OS Communications Server product includes ACF/VTAM, in addition to TCP/IP.

This chapter presents a basic overview of z/OS Communications Server IP as it is

implemented in the z/OS environment. You can find a more complete and comprehensive
explanation of z/OS Communications Server IP from the publications that are listed in 1.4,
“Additional information” on page 17.

This chapter discusses the following topics.

Section Topic

1.1, “Overview” on page 2 Basic concepts of Communications Server for z/OS IP

1.2, “Featured functions” on page 3 Key characteristics of Communications Server for z/OS
IP and

1.3, “Communications Server for z/OS IP Functional overview of how Communications Server for
implementation” on page 4 z/OS IP is implemented

1.4, “Additional information” on page 17 Lists IBM publications that provide further details for
implementing Communications Server for z/OS IP

© Copyright IBM Corp. 2011. All rights reserved. 1

7896 [Link] Draft Document for Review March 21, 2011 7:28 am

1.1 Overview
z/OS Communications Server provides the industry-standard TCP/IP protocol suite, allowing
z/OS environments to share data and computing resources with other TCP/IP computing
environments, when authorized. Communications Server for z/OS IP enables anyone in a
non-z/OS TCP/IP environment to access resources in the z/OS environment and perform
tasks and functions provided by the TCP/IP protocol suite.

It provides the computer platform with the freedom desired by organizations to distribute
workload to environments best suited to their needs. Communications Server for z/OS IP,
therefore, adds the z/OS environment to the list of environments in which an organization can
share data and computer processing resources in a TCP/IP network.

Communications Server for z/OS IP supports two environments:

 It provides a native MVS (z/OS) environment in which users can exploit the TCP/IP
protocols in the z/OS applications environment, including batch jobs, started tasks, TSO,
CICS® applications, and IMS™ applications.
 It also provides native TCP/IP support in the UNIX® Systems Services environment in
which users can create and use applications that conform to the POSIX or XPG4 standard
(a UNIX specification). The UNIX environment and services can also be exploited from the
z/OS environment, and vice versa.

1.1.1 Basic concepts

The TCP/IP address space is where the TCP/IP protocol suite is implemented for CS for z/OS
IP. The TCP/IP address space is commonly referred to as a stack.

Communications Server for z/OS IP has highly efficient direct communication between the
UNIX System Services address space (OMVS) and a TCP/IP stack that was integrated in
UNIX System Services. This communication path includes the UNIX System Services
Physical File System (PFS) component for AF_INET and AF_INET6 (Addressing
Family-Internet) sockets communication.

The z/OS Communications Server has the following features:

 A process model that provides a full multiprocessing capability. It includes full duplex data
paths of reduced lengths.
 An I/O process model that allows VTAM to provide the I/O device drivers. MultiPath
Channel (MPC) Data Link Control (DLC) is shared between VTAM and TCP/IP. It executes
multiple dispatchable units of work and is tightly integrated with the Common Storage
Manager support.
 A storage management model handles expansion and contraction of storage resources,
as well as requests of varying sizes and types of buffers. Common Storage Manager
(CSM) manages communication between the Sockets PFS through the transport provider
and network protocols to the network interface layer of Communications Server for z/OS
IP stack. The data that is placed in the buffers can be accessed by any function all the way
down to the protocol stack.

Communications Server for z/OS IP runs as a single stack that serves both the traditional
MVS (z/OS) environment and the z/OS UNIX (UNIX System Services) environment, and IP
offers two variants of the UNIX shell environment:

2 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:28 am 7896 [Link]

 The OMVS shell, which is much like a native UNIX environment

 The ISHELL, which is an ISPF interface with access to menu-driven command interfaces

The TCP/IP protocol suite is implemented by an MVS started task within the TCP/IP address
space in conjunction with z/OS UNIX (UNIX System Services).

A Communications Server for z/OS IP environment requires a Data Facility Storage

Management Subsystem (DFSMS), a z/OS UNIX file system, and a security product such as
Resource Access Control Facility (RACF). These resources must be defined and functional
before the z/OS Communications Server can be started successfully and establish the
TCP/IP environment. We later mention the manner in which these products impact this
environment.

1.2 Featured functions

z/OS Communications Server provides a high-performance, highly secure, scalable, and
reliable platform on which to build and deploy networking applications.

Communications Server for z/OS IP offers an environment that is accessible to the enterprise
IP network and the Internet if so desired. It defines the z/OS environment as a viable platform
by making z/OS applications and systems available to the non-z/OS environment, which are
typically UNIX/Windows®-centric. Consequently, it eliminates the issues and challenges of
many large corporations to migrate or integrate with a more accessible platform and newer
technologies.

The following list includes many of the technologies that have been implemented in the z/OS
environment to complement TCP/IP.
 High-speed connectivity, such as:
– OSA-Express3 10 Gigabit Ethernet in QDIO mode
– High-speed communication between TCP/IP stacks running in logical partitions using
HiperSockets™ in IQDIO mode.
 High availability for applications using Parallel Sysplex technology in conjunction with:
– Dynamic Virtual IP Address (VIPA), which provides TCP/IP application availability
across z/OS systems in a sysplex and allows participating TCP/IP stacks to provide
backup and recovery for each other, for planned and unplanned TCP/IP outages
– Sysplex Distributor, which provides intelligent load balancing for TCP/IP application
servers in a sysplex, and along with Dynamic VIPA provides a single system image for
client applications connecting to those servers
– The Load Balancing Advisor (LBA), which provides z/OS Sysplex server application
availability and performance data to outboard load balancers through the Server
Application State Protocol (SASP)
 Enterprise connectivity support is offered through many features, such as:
– TN3270 Server, which provides workstation connectivity over TCP/IP networks to
access z/OS and enterprise SNA applications.
– Enterprise Extender, which allows SNA Enterprise applications to communicate
reliably over an IP network, using SNA HPR over UDP transport layer protocol.
– IPv4 and IPv6 networking functions are provided by the TCP/IP stack operating in a
standard dual-mode setup where IPv4 and IPv6 connectivity and applications are
supported concurrently by a single TCP/IP stack instance.

Chapter 1. Introduction to Communications Server for z/OS IP 3

7896 [Link] Draft Document for Review March 21, 2011 7:28 am

– Sockets programming interface support for traditional z/OS workloads provide IP

connectivity to applications written in REXX, COBOL, PL/I. Sockets programming
interfaces are supported in various environments, such as TSO, batch, CICS, and IMS.
 Network Security protects sensitive data and the operation of the TCP/IP stack on z/OS by
using:
– IPsec/VPN functions that enable the secure transfer of data over a network using
standards for encryption, authentication, and data integrity.
– Intrusion Detection Services (IDS), which evaluates the stack for attacks that would
undermine the integrity of its operation. Events to examine and actions to take (such as
logging) at event occurrence are defined by the IDS policy.
– Transport Layer Security (TLS) enablement ensures data is protected as it flows across
the network.
– Kerberos and GSSAPI support is provided for selected applications.
– Defensive filtering provides an infrastructure to add, delete and modify short-term
TCP/IP filters in real time to counter specific attacks.
– Network Security Services provides a centralized security infrastructure to extend
System z security to NSS clients, such as IKE daemons and XML appliances.
 Network Management support collects network topology, status, and performance
information and makes it available to network management tools, including:
– Local management applications that can access management data using a specialized
high-performing network management programming interface that is known as NMI.
– Support of remote management applications through the SNMP protocol.
Communications Server z/OS supports the latest SNMP standard, SNMPv3.
Communications Server z/OS also supports standard TCP/IP-based Management
Information Base (MIB) data.
– Additional MIB support is also provided by Enterprise-specific MIB, which supports
management data for Communications Server TCP/IP stack-specific functions.

1.3 Communications Server for z/OS IP implementation

Communications Server for z/OS IP provides TCP/IP support for the native MVS and UNIX
System Services environment. It is implemented within a z/OS address space and runs within
the native MVS environment, and consequently it has RACF, DFSMS, and z/OS UNIX file
system dependencies.

1.3.1 Functional overview

CS for z/OS IP takes advantage of Communications Storage Manager (CSM) and of VTAM’s
Multipath Channel (MPC) and Queued Direct I/O (QDIO) capabilities in its TCP/IP protocol
implementation. This tight coupling with VTAM provides enhanced performance and
serviceability.

4 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:28 am 7896 [Link]

As illustrated in Figure 1-1, many data link control (DLC) protocols are provided with the z/OS
Communications Server by the VTAM component.

LPD client, NDB, NICS, RPC, Kerberos, Bind 4.9.3 (DNS/WLM server), Bind 9 (DNS server), DHCP
LPD server, MISC server, NCPRoute, server, TN3270 server, FTP server, FTP client, Telnet
SMTP server, Portmapper, NPF, SNMP query, server, X-Windows client, SNMP Agent, OMPROUTE,
Telnet client X-Windows client, DPI library DPI library and SNMP Command: Netstat, Ping, Tracerte,
R-commands, RPC, REXEC, RSH, Sendmail

IMS CICS REXX

Sockets z/OS UNIX Sockets
Sockets Extended
Pascal Callable ASM, COBOL, PL/1
API BPX
Assembler ASM
C-Sockets Callable C-Sockets
API

Logical File System

Physical File System

TCP, UDP, and Raw Sockets (Transport Protocol Layer)

C
S
IP and ICMP (Network Protocols and Interface Layer)
M

SAMEHOST
(SNALINK, CDLC CLAW CTC HYPER LCS MPC+ XCF ATM
X.25)

TCP/IP Exclusive Shared DLCs

DLCs

Figure 1-1 Functional overview

With CS for z/OS IP, two worlds converge, providing access to the z/OS UNIX environment
and the traditional MVS environment.

1.3.2 Operating environment

Because the z/OS UNIX environment is supported in the MVS environment, there is no need
to discuss the creation of an MVS environment here. However, there are customization
requirements on the UNIX System Services side of the environment that are needed in order
to start Communications Server for z/OS IP successfully. This dependence on UNIX, of
course, implies that z/OS UNIX administrators must also be familiar with both traditional MVS
commands and interfaces.

I/O flow process

Another feature of the operating environment is the storage and I/O designs. The operating
environment design features a tightly integrated storage and I/O model, known as Common
Storage Manager (CSM). The CSM facility is used by authorized programs to manage
subsystem storage pools. It provides a flat storage model that is accessible by multiple layers
of the process model, as Figure 1-1 illustrates. It is also accessible across z/OS address
space boundaries, thereby reducing the data moves between processes and tasks that
exchange data as they perform work. VTAM and TCP/IP tasks are typical examples.

Chapter 1. Introduction to Communications Server for z/OS IP 5

7896 [Link] Draft Document for Review March 21, 2011 7:28 am

The CSM facility also manages storage as it automates the addition and subtraction of the
different types and sizes of storage requests.

1.3.3 Reusable address space ID

The z/OS system assigns an address space identifier (ASID) to an address space when the
address space is created. A limited number of ASIDs are available for the system to assign.
When all ASIDs are assigned to existing address spaces, the system cannot start a new
address space. This condition can cause lost ASIDs in the system, which are address spaces
that have terminated but which the system does not reuse because of the address space’s
residual cross memory connections.

ASIDs used for the TCP/IP stack, the resolver, VTAM, and TN3270 are non-reusable,
because they provide PC-entered services that must be accessible to other address spaces.
If these address spaces are terminated enough times, all available ASIDs can be exhausted,
preventing the creation of a new address space on the system. That situation might require
an IPL.

With the reusable address space support, ASIDs that would otherwise be unusable after
termination of the started task are made available for reuse. The reusable ASID function is
available for the TCP/IP, resolver, VTAM, and TN3270 started tasks; however, it is not
available to TNF, VMCF, and all the products and applications that use their services (such as
TSO TELNET).

To enable the reuse ASID function, you must:

 Specify REUSASID(YES) in member DIAGxx of your PARMLIB
 Specify REUSASID=YES on the start command when starting the address space

The REUSASID parameter cannot be coded in the JCL of the started task because the
Master Scheduler needs to know this information before the JCL is read and the ASID is
assigned.

The resolver started task will always use a reusable ASID when started during z/OS UNIX
initialization through the BPXRMMxx statement RESOLVER_PROC but will use a
non-reusable ASID if stopped and started. You should, therefore, restart resolver with the
REUSASID=YES parameter specified on the start command.

The REUSASID parameter is to be used only by address spaces such as TCP/IP, resolver,
and TN3270 that are usually non-reusable when terminated, because unnecessary use of
REUSASID=YES can reduce the number of ASIDs that are available for satisfying ordinary
address space requests.

We include examples of REUSASID coding and its results in Appendix B, “Additional

parameters and functions” on page 411.

1.3.4 Protocols and devices

As illustrated in Figure 1-1 on page 5, the DLC is a protocol layer that manages and provides
communication between the file I/O subsystem and the I/O device driver of the particular
device. The figure also shows two categories of DLCs:
 TCP/IP exclusive DLCs
 Shared DLCs

6 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:28 am 7896 [Link]

TCP/IP exclusive DLCs

TCP/IP exclusive DLCs are those only available for TCP/IP usage and are not shared with
ACF/VTAM. Here are examples of TCP/IP exclusive DLCs that are supported by
Communications Server for z/OS IP:
 LAN Channel Station (LCS), which is a protocol is used by OSA 1000BASE-T feature,
some routers, and the 3746-9x0 MAE.
 SAMEHOST, which is another TCP/IP exclusive DLC protocol that exists, although it does
not make use of System z channels. In the past, this communication was provided by
IUCV. Currently, SNALINK LU0, SNALINK LU6.2, and X.25 exploit the SAMEHOST DLC.

Shared DLCs
Shared DLCs are those that can be simultaneously used by TCP/IP and ACF/VTAM.
Figure 1-1 on page 5 indicates the shared DLCs. The most commonly used DLCs include
those that we describe here.

Multipath Channel+
Multipath Channel+ (MPC+) is an enhanced version of VTAM’s MPC protocol. The MPC I/O
process defines the implementation of the MPC protocols and allows for the efficient use of
multiple read and write channels.

MPC handles protocol headers and data separately and executes multiple I/O dispatchable
units of work. This process, when used in conjunction with Communication Storage
Management, creates efficient I/O throughput. High Performance Data Transfer uses MPC+
together with CSM to decrease the number of data copies that are required to transmit data.

This type of connection can be used in two ways:

 MPCPTP allows a CS for z/OS IP environment to connect to a peer IP stack in a
point-to-point configuration. With MPCPTP, a CS for z/OS IP stack can be connected to:
– Another CS for z/OS IP stack
– An IP router with corresponding support
– A non-z/OS server
– 3746-9x0 MAE
PTP Samehost (MPCPTP), sometimes referred to IUTSAMEH: This connection type is
used to connect two or more CS for z/OS IP stacks running on the same z/OS LPAR. In
addition, it can be used to connect these CS for z/OS IP stacks to z/OS VTAM for the use
of Enterprise Extender.
 MPCIPA allows an Open Systems Adapter-Express (OSA-Express) port to act as an
extension of the z/OS Communications Server TCP/IP stack and not as a peer TCP/IP
stack, as with MPCPTP.
– OSA-Express provides a mechanism for communication called Queued Direct I/O
(QDIO). Although it uses the MPC+ protocol for its control signals, the QDIO interface
is quite different from channel protocols. It uses Direct Memory Access (DMA) to avoid
the overhead associated with channel programs. A partnership between CS for z/OS
IP and the OSA-Express adapter provides compute-intensive functions from the
System z server to the adapter.

Chapter 1. Introduction to Communications Server for z/OS IP 7

7896 [Link] Draft Document for Review March 21, 2011 7:28 am

OSA-Express collaborates with z/OS Communications Server TCP/IP to support up to

10 Gigabit Ethernet, 1000BASE-T, Fast Ethernet, High Speed Token Ring (HSRP), and
ATM LAN emulation. TCP/IP hosts supports OSA-Express, OSA-Express2, and
OSA-Express3 features.
– HiperSockets (Internal Queued Direct I/O, iQDIO) provides high-speed, low-latency IP
message passing between logical partitions (LPARs) within a single System z server.
The communication is through processor system memory through Direct Memory
Access (DMA). The virtual servers that are connected through HiperSockets form a
virtual LAN. HiperSockets uses internal QDIO at memory speeds to pass traffic
between virtual servers.

Cross-System Coupling Facility

Cross-System Coupling Facility (XCF) allows communication between multiple CS for z/OS
IP stacks within a Parallel Sysplex. The XCF DLC can be defined as with traditional DLCs, but
it also supports XCF Dynamics, in which the XCF links are established automatically.

If DYNAMICXCF is coded, z/OS images within the same server will use the HiperSockets
DYNAMICXCF connectivity instead of the standard XCF connectivity for data transfer.

For more information about devices and connectivity options, refer to Chapter 4,
“Connectivity” on page 117.

1.3.5 Supported routing applications

z/OS Communications Server ships only one routing application, called OMPROUTE.
OMPROUTE implements the Open Shortest Path First protocols (OSPF and OSPFv4) and
Routing Information Protocols (RIPv1, RIPv2, RIPng). It enables the Communications Server
for z/OS IP to function as an OSPF/RIP-capable router in a TCP/IP network. Either (or both)
of these two routing protocols can be used to dynamically maintain the host routing table.

Additionally, Communications Server for z/OS IP provides an OMPROUTE subagent that

implements the OSPF MIB variable containing OSPF protocol and state information for
SNMP. This MIB variable is defined in RFC 1850. Refer to Chapter 5, “Routing” on page 205,
for a detailed discussion about OMPROUTE and its function.

1.3.6 Application programming interfaces

As Figure 1-1 on page 5 illustrates, all of the APIs provided by Communications Server for
z/OS IP, with the exception of the Pascal API, interface with the Logical File System (LFS)
layer. The APIs are divided into the following categories:
 Pascal
 TCP/IP socket APIs
 z/OS UNIX APIs
 REXX sockets

We describe these items in more detail in the following sections.

Pascal API
The Pascal application programming interface enables you to develop TCP/IP applications in
Pascal language. Supported environments are normal MVS address spaces. Unlike the other
APIs, the Pascal API does not interface directly with the LFS. It uses an internal interface to
communicate with the TCP/IP protocol stack. The Pascal API only supports AF_INET.

8 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:28 am 7896 [Link]

TCP/IP socket APIs

The z/OS Communications Server provides several APIs to access TCP/IP sockets. These
APIs can be used in either or both integrated and common INET PFS configurations.

In a common INET PFS configuration, however, they function differently from z/OS UNIX
APIs. In this type of configuration, the z/OS Communications Server APIs always bind to a
single PFS transport provider, and the transport provider must be the TCP/IP stack provided
by the z/OS Communications Server.

The following TCP/IP socket APIs are included in the z/OS Communications Server:
 The CICS socket interface enables you to write CICS applications that act as clients or
servers in a TCP/IP-based network. CICS sockets only support AF_INET.
 The C sockets interface supports socket function calls that can be invoked from C
programs. However, note that for C application development, IBM recommends the use of
the UNIX C sockets interface. These programs can be ported between MVS and most
UNIX environments relatively easily if the program does not use any other MVS-specific
services. C sockets only support AF_INET.
 The Information Management System (IMS) IPv4 socket interface supports client/server
applications in which one part of the application executes on a TCP/IP-connected host
and the other part executes as an IMS application program. The IMS sockets API supports
AF_INET.
 The Sockets Extended macro API is a generalized assembler macro-based interface to
sockets programming. The Sockets Extended macro API supports AF_INET and
AF_INET6.
 The Sockets Extended Call Instruction API is a generalized call-based, high-level
language interface to sockets programming. The Sockets Extended Call Instruction API
supports AF_INET and AF_INET6.

z/OS UNIX APIs

The following APIs are provided by the z/OS UNIX element of z/OS and are supported by the
TCP/IP stack in the z/OS Communications Server:
 z/OS UNIX C sockets is used in the z/OS UNIX environment. It is the z/OS UNIX version
of the native MVS C sockets programming interface. Programmers use this API to create
applications that conform to the POSIX or XPG4 standard (a UNIX specification). The
z/OS UNIX C sockets support AF_INET and AF_INET6.
 z/OS UNIX assembler callable services is a generalized call-based, high-level language
interface to z/OS UNIX sockets programming. The z/OS UNIX assembler callable services
support AF_INET and AF_INET6.

Refer to z/OS XL C/C++ Compiler and Run-Time Migration Guide for the Application
Programmer, GC09-4913, for complete documentation of the z/OS UNIX C sockets APIs. You
can also find further guidance in z/OS UNIX System Services Programming Tools,
SA22-7805.

REXX sockets
The REXX sockets programming interface implements facilities for socket communication
directly from REXX programs by using an address rxsocket function. REXX socket programs
can execute in TSO, online, or batch. The REXX sockets programming interface supports
AF_INET and AF_INET6.

Refer to z/OS Communications Server: IP Sockets Application Programming Interface Guide

and Reference, SC31-8788, for complete documentation of the TCP/IP Services APIs.

Chapter 1. Introduction to Communications Server for z/OS IP 9

7896 [Link] Draft Document for Review March 21, 2011 7:28 am

1.3.7 z/OS Communications Server applications

z/OS Communications Server TCP/IP provides a number of standard client and server
applications, including:
 SNA 3270 Logon Services (TN3270)
 z/OS UNIX logging services (syslogd)
 File Transfer Services (FTP)
 Network Management Services (SNMP Agents, Subagents, Trap forwarding)
 IP Printing (LPR, LPD, Infoprint Server)
 Internet Daemon Listener (INETD)
 Mail Services (SMTP and sendmail)
 Client-based mail forwarding Simple Network Management Protocol (CSSMTP)
 z/OS UNIX logon services (otelnetd)
 Remote Execution (REXECD, RSHD, REXEC, RSH, orexecd, orshd, orexec, and orsh)
 Domain Name Services (Caching DNS BIND9 server)

These applications are discussed in detail in IBM z/OS Communications Server TCP/IP
Implementation Volume 2: Standard Applications ,SG24-7897, and z/OS Communications
Server: IP Configuration Guide, SC31-8775.

z/OS Communications Server also provides a number of specialized services, including:

 Policy Agent for implementing networking and security policies in a z/OS environment
 Centralized or Distributed Policy Services
 Network Security Services (NSS)
 Defense Manager
 Integrated Services policies using Resource ReSrVation Protocol (RSVP)
 Differentiated Services using Quality of Service (QoS) policies.

These applications are discussed in detail in the following publications:

 Communications Server for z/OS TCP/IP Implementation Volume 4: Security and
Policy-Based Networking, SG24-7899.
 z/OS Communications Server: IP Configuration Guide, SC31-8775

1.3.8 UNIX System Services

UNIX System Services is the z/OS Communications Server implementation of UNIX as
defined by X/Open in XPG 4.2. UNIX System Services coexists with traditional MVS functions
and traditional MVS file types (partitioned data sets, sequential files, and so on). It
concurrently allows access to z/OS UNIX file system files and to UNIX utilities and commands
by means of application programming interfaces and the interactive shell environment.

Communications Server for z/OS IP offers two variants of the UNIX shell environment:
 The z/OS shell, which is the default shell
 The tcsh shell (Ishell), which is an enhanced version of the Berkeley UNIX C shell

The Communications Server for z/OS IP requires that UNIX System Services be customized
in full-function mode before the TCP/IP stack will successfully initialize. For this reason we
present an overview of UNIX System Services to provide an overview of the coding and
security considerations that are involved with UNIX System Services.

10 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:28 am 7896 [Link]

Customization levels of UNIX System Services

There are two levels of z/OS UNIX services:
 Minimum mode, indicating that although OMVS initializes, it provides few z/OS UNIX
services, and there is no support for TCP/IP and the z/OS shell. In this mode there is no
need for DFSMS or for a security product such as RACF.
 Full-function mode, indicating that the complete array of z/OS UNIX services is available.
In this mode DFSMS, RACF, and the z/OS UNIX file system are required. TCP/IP and
z/OS UNIX file system interaction with UNIX System Services is defined within the
BPXPRMxx member of [Link].

See z/OS UNIX System Services Planning, SA22-7800 for a useful description of the UNIX
System Services customization process and TCP/IP.

UNIX System Services concepts

z/OS UNIX enables two open systems interfaces on the z/OS operating system:
 An application program interface (API)
 An interactive shell interface

With the APIs, programs can run in any environment (including batch jobs, in jobs submitted
by TSO/E interactive users, and in most other started tasks) or in any other MVS application
task environment. The programs can request:
 Only MVS services
 Only z/OS UNIX services
 Both MVS and z/OS UNIX services

The shell interface is an execution environment similar to TSO/E, with a programming

language of shell commands like those in the Restructured Extended Executor (REXX)
language. The shell work consists of:
 Programs that are run interactively by shell users
 Shell commands and scripts that are run interactively by shell users
 Shell commands and scripts that are run as batch jobs

In z/OS UNIX Systems Services, address spaces are provided by the fork() or spawn()
functions of the Open Edition callable services.
 For a fork() function, the system copies one process, called the parent process, into a
new process, called the child process, and places the child process in a new address
space, the forked address space.
 A spawn() functions also starts a new process in a new address space. Unlike a fork(), in
a spawn() call the parent process specifies a name of a program to be run in the child
process.

The types of processes can be:

 User processes, which are associated with a user
 Daemon processes, which perform continuous or periodic system-wide functions, such as
a Web server
Daemons (a UNIX concept) are programs that are typically started when the operating
system is initialized and remain active to perform standard services. Some programs are
considered daemons that initialize processes for users even though these daemons are
not long-running processes. Examples of daemons provided by z/OS UNIX are cron,
which starts applications at specific times, and inetd, which provides service management
for a network.

Chapter 1. Introduction to Communications Server for z/OS IP 11

7896 [Link] Draft Document for Review March 21, 2011 7:28 am

A process can have one or more threads. A thread is a single flow of control within a process.
Application programmers create multiple threads to structure an application in independent
sections that can run in parallel for more efficient use of system resources.

UNIX Hierarchical File System

Data sets and files are comparable terms. If you are familiar with MVS, you probably use the
term data set to describe a unit of data storage. If you are familiar with AIX® or UNIX, you
probably use the term file to describe a named set of records stored or processed as a unit.
In the UNIX System Services environment, the files are arranged in a z/OS UNIX file system.

The Hierarchical File System allows you to set up a file hierarchy that consists of:
 Directories, which contain files, other directories, or both. Directories are arranged
hierarchically, in a structure that resembles an upside-down tree, with the root directory at
the top and branches at the bottom.
 z/OS UNIX file system files, which contain data or programs. A file containing a load
module, shell script, or REXX program is called an executable file. Files are kept in
directories.
 Additional local or remote file systems, which are mounted on directories of the root file
system or of additional file systems.

To the z/OS system, the UNIX file hierarchy appears as a collection of System z File System
data sets. Each z/OS UNIX file system data set is a mountable file system. The root file
system is the first file system mounted. Subsequent file systems can be mounted logically on
a directory within the root file system or on a directory within any mounted file system.

Each mountable file system resides in a z/OS UNIX file system data set on direct access
storage. DFSMS/MVS manages the z/OS UNIX file system data sets and the physical files.

For more information about the z/OS UNIX file system, refer to z/OS CS: IP Migration,
GC31-8773, and z/OS UNIX System Services Planning, SA22-7800.

z/OS UNIX file system definitions in BPXPRMxx

To get UNIX System Services active in full-function mode, you need to define the root file
system in the BPXPRMxx member of [Link]. The root file system is usually loaded
or copied at z/OS installation time. The BPXPRMxx definition is detailed in z/OS UNIX
System Services Planning, SA22-7800.

An important part of your z/OS UNIX file system is located in the /etc directory. The /etc
directory contains some basic configuration files of UNIX System Services, and most
applications keep their configuration files in there as well. To avoid losing all of your
configuration when you upgrade your operating system, it is recommended that you put the
/etc directory in a separate z/OS UNIX file system data set and mount it at the /etc
mountpoint. Refer to z/OS UNIX System Services Planning, SA22-7800, for more information
about the /etc directory.

z/OS UNIX user identification

All users of an MVS system, including users of z/OS UNIX functions, must have a valid MVS
user ID and password. To use standard MVS functions, the user must have the standard MVS
identity based on the RACF user ID and group name.

If a unit of work in MVS uses z/OS UNIX functions, this unit of work must have, in addition to
a valid MVS identity, a z/OS UNIX identity. A z/OS UNIX identity is based on a UNIX user ID
(UID) and a UNIX group ID (GID). Both UID and GID are numeric values ranging from 0 to
2147483647 (231-1).

12 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:28 am 7896 [Link]

In a z/OS UNIX system, the UID is defined in the OMVS segment in the user's RACF user
profile, and the GID is defined in an OMVS segment in the group's RACF group profile. What
we in an MVS environment call the user ID is in a UNIX environment normally termed the
user name or the login name. It is the name that users use to present themselves to the
operating system. In both a z/OS UNIX system and other UNIX systems, this user name is
correlated to a numeric user identification, the UID, which is used to represent this user
wherever such information has to be stored in the z/OS UNIX environment. One example of
this is in the Hierarchical File System, where the UID of the owning user is stored in the file
security portion of each individual file.

Access to resources in the traditional MVS environment is based on the MVS user ID, group
ID, and individual resource profiles that are stored in the RACF database.

Access to z/OS UNIX resources is granted only if the MVS user ID has a valid OMVS
segment with an OMVS UID, or if a default user is configured as explained next. Access to
resources in the Hierarchical File System is based on the UID, the GID, and file access
permission bits that are stored with each file. The permission bits are three groups of three
bits each. The groups describe:
 The owner of the file itself
 The users with the same GID as the owner
 The rest of the world

The three bits are:

 Read access
 Write access
 Search access if it is a directory or if it is a file that is executable

The superuser UID has a special meaning in all UNIX environments, including the z/OS UNIX
environment. This user has a UID of zero and can access every resource.

In lieu of or in addition to RACF definitions for individual users, you can define a default user.
The default user will be used to allow users without an OMVS segment defined to access
UNIX System Services. The default user concept should be used with caution, because it
could become a security exposure.

You will also find more information about the RACF security aspects of implementing the
Communications Server for z/OS IP in Communications Server for z/OS TCP/IP
Implementation Volume 4: Security and Policy-Based Networking, SG24-7899.

Accessing the z/OS UNIX shells

You can access z/OS UNIX shells in the following ways:
 The TSO/E OMVS command provides a 3270 interface in the z/OS UNIX shell.
 The TSO/E ISHELL or ISH command provides a 3270 interface that uses ISPF dialogs.
 The rlogin command provides an ASCII interface.
 The Telnet command provides an ASCII interface. This Telnet is into the UNIX Telnet
daemon and not the TN3270 server in the z/OS system space.
 From a TCP/IP network, the TN3270 command can be used, which provides a full-screen
3270 interface for executing the OMVS or ISHELL commands.

There are two shells, the z/OS shell and the Ishell. The login shell is determined by the
PROGRAM parameter in the RACF OMVS segment for each user. The default is the z/OS
shell.

Chapter 1. Introduction to Communications Server for z/OS IP 13

7896 [Link] Draft Document for Review March 21, 2011 7:28 am

You can find further information about the z/OS UNIX shells in z/OS UNIX System Services
User’s Guide, SA22-7801.

Operating mode
When a user first logs on to the z/OS UNIX shell, the user is operating in line mode.
Depending on the method of accessing the shell, the user can then use utilities that require
raw mode (such as vi) or run an X Window System application.

The different workstation operating modes are:

 Line mode
Input is processed after you press Enter. This is also called canonical mode.
 Raw mode
Each character is processed as it is typed. This is also called non-canonical mode.
 Graphical mode
This is a graphical user interface for X Window System applications.

UNIX System Services communication

A socket is the endpoint of a communication path; it identifies the address of a specific
process at a specific computer using a specific transport protocol. The exact syntax of a
socket address depends on the protocol being used, that is, on its addressing family.

When you obtain a socket using the socket() system call, you pass a parameter that tells the
socket library to which addressing family the socket should belong. All socket addresses
within one addressing family use the same syntax to identify sockets.

Socket addressing families in UNIX System Services

In a z/OS UNIX environment, the most widely used addressing families are AF_INET and
AF_UNIX. There is IPv6 support (AF_INET6 addressing family) in Communications Server
for z/OS IP in a single transport driver environment configured in Dual-mode. Socket
applications written to the IPv6 APIs can use the z/OS TCP/IP stack for IPv6 network
connectivity.

Note: Throughout this discussion, information regarding AF_INET (IPv4) also applies to
AF_INET6 (IPv6).

The z/OS UNIX Systems Services implements support for a given addressing family through
different physical file systems. There is one physical file system for the AF_INET addressing
family, and there is another for the AF_UNIX addressing family. A PFS is the part of the z/OS
UNIX operating system that handles the storage of data and its manipulation on a storage
medium.

AF_UNIX addressing family

The UNIX addressing family is also referred to as the UNIX domain. If two socket applications
on the same MVS image want to communicate with each other, they can open a socket as an
AF_UNIX family socket. In that case, the z/OS UNIX address space will handle the full
communication between the two applications (see Figure 1-2). That is, the AF_UNIX physical
file system is self-contained within z/OS UNIX and does not rely on other products to
implement the required functions.

14 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:28 am 7896 [Link]

UNIX Application UNIX Application

LFS

PFS = AF_UNIX = UDS

Figure 1-2 AF_UNIX sockets

AF_INET addressing family

This is the Internet addressing family, also referred to as the Internet domain. Socket
programs communicate with socket programs on other hosts in the IP network using
AF_INET family sockets which, in turn, use the AF_INET physical file system.

You can configure either AF_INET or both AF_INET and AF_INET6. You cannot define the
stack as IPv6 only. Although coding AF_INET6 alone is not prohibited, TCP/IP will not start
because the master socket is AF_INET and the call to open it will fail.

For more on this subject, refer to Chapter 3, “Base functions” on page 59 or z/OS UNIX
System Services Planning, SA22-7800.

The AF_INET physical file system relies on other products to provide the AF_INET transport
services to interact with UNIX System Services and its sockets programs.

For AF_INET/AF_INET6 sockets, the z/OS UNIX address space routes the socket request to
the TCP/IP address space directly. As shown in Figure 1-3, the sockets/Physical File System
layer is a transform layer between z/OS UNIX and the TCP/IP stack.

UNIX Application

LFS

PFS = AF_INET=Type(INET)

CS for z/OS IP

Figure 1-3 AF_INET sockets

The sockets/PFS effectively transforms the sockets calls from the z/OS UNIX interface to the
TCP/IP stack regardless of the version of MVS or TCP/IP. The sockets/PFS handles the
communication between the TCP/IP address space and the z/OS UNIX address space in
much the same manner as High Performance Native Socket (HPNS) handles the
communication between the TCP/IP address space and the TCP/IP client and server address
spaces.

Chapter 1. Introduction to Communications Server for z/OS IP 15

7896 [Link] Draft Document for Review March 21, 2011 7:28 am

Physical File System transport providers

TCP/IP requires the use of the Physical File System (AF_INET) configured in two ways:
 The Integrated Sockets File System type (INET)
 The Common INET Physical File System type (CINET)

INET is used in a single-stack environment and CINET is used in a multiple-stack

environment.

A single Physical File System transport provider

If your background is in a UNIX environment, it might seem strange to have a choice of using
INET or CINET, because you are familiar with the TCP/IP protocol stack being an integral part
of the UNIX operating system. However, this is not the case in a z/OS environment; it is very
versatile. In this environment you can start multiple instances of a TCP/IP protocol stack,
each stack running on the same operating system, but each stack having a unique TCP/IP
identity in terms of network interfaces, IP addresses, host name, and sockets applications.

A simple example of a situation where you have more TCP/IP stacks running in your z/OS
system is if you have two separate IP networks, one production and one test (or one secure
and one not). You do not want routing between them, but you do want to give hosts on both IP
networks access to your z/OS environment. In this situation you could implement two TCP/IP
stacks, one connected to the production IP network and another connected to the test
network.

This multi-stack implementation in which you share the UNIX System Services across
multiple TCP/IP stacks provides challenges. Sockets applications that need to have an affinity
to a particular stack need special considerations, in some cases including the coordination of
port number assignments to avoid conflicts. For more information, see Chapter 3, “Base
functions” on page 59.

If a single AF_INET(6) transport provider is sufficient, then use the Integrated Sockets
physical file system (INET). If you need more than one AF_INET(6) transport provider
(multiple TCP/IP stacks), then you must use the Common INET physical file system (CINET).

You can customize z/OS to use the Common INET physical file system with just a single
transport provider (AF_INET(6), but it is generally not recommended due to a slight
performance decrease as compared to the Integrated Sockets Physical File System (INET).
However, you might consider doing this if you expect to run multiple stacks in the future.

The PFS is also known under the name INET, and this appears in UNIX System Services
definitions when a FILESYSTYPE and NETWORK TYPE need to be defined in the
BPXPRMxx member of [Link].

Common INET Physical File System (CINET)

If you have two or more AF_INET transport providers on an MVS image, such as a production
TCP/IP stack together with a test TCP/IP stack, you must use the Common INET Physical
File System. Figure 1-4 shows a multiple stack environment with Common INET (CINET).

16 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:28 am 7896 [Link]

OE Application

OE LFS

C-INET PFS

Stack TCPIPA Stack TCPIPB

TCP and UDP TCP and UDP

IP and ICMP IP and ICMP

Network Interfaces Network Interfaces

CS for z/OS CS for z/OS

IP Network

Figure 1-4 Multiple INET transport providers: CINET PFS

Recommendation: Although there are situations where multiple stacks per LPAR can
provide value, in general we recommend that you implement only one TCP/IP stack per
LPAR for the following reasons:
 A TCP/IP stack is capable of exploiting all available resources defined to the LPAR in
which it is running. Therefore, starting multiple stacks will not yield any increase in
throughput.
 When running multiple TCP/IP stacks, additional system resources, such as memory,
CPU cycles, and storage, are required.
 Multiple TCP/IP stacks add a significant level of complexity to TCP/IP system
administration tasks.
 It is not necessary to start multiple stacks to support multiple instances of an application
on a given port number, such as a test HTTP server on port 80 and a production HTTP
server also on port 80. This type of support can instead be implemented using
BIND-specific support where the two HTTP server instances are each associated to
port 80 with their own IP address, using the BIND option on the PORT reservation
statement.

1.4 Additional information

The following IBM publications provide further details for implementing a z/OS environment
that supports the TCP/IP protocol suite:
 z/OS Communications Server: IP Configuration Guide, SC31-8775
This document explains the major concepts of, and provides implementation guidance for,
z/OS Communications Server functions.

Chapter 1. Introduction to Communications Server for z/OS IP 17

7896 [Link] Draft Document for Review March 21, 2011 7:28 am

 z/OS Communications Server: IP Configuration Reference, SC31-8776

This document details the parameters or statements that can be used to implement z/OS
Communications Server functions.
 z/OS Communications Server: IP Programmer’s Guide and Reference, SC31-8787
This document provides the guidelines for programming the IP applications on the z/OS.
 z/OS Communications Server: IP Sockets Application Programming Interface Guide and
Reference, SC31-8788
This document provides detailed information about the socket API for programming the IP
applications on the z/OS.

For migration, the following publications are also helpful:

 z/OS Communications Server: New Function Summary, GC31-8771
This document includes function summary topics to describe all the functional
enhancements for the IP and SNA components of Communications Server, including task
tables that identify the actions necessary to exploit new function. Use this document as a
reference to using all the enhancements of z/OS Communications Server.
 z/OS Planning for Installation, GA22-7504
This document helps you prepare to install z/OS by providing the information you need to
write an installation plan.
 z/OS Migration, GA22-7499
This document describes how to migrate (convert) from release to release. Use this
document as a reference for keeping all z/OS applications working as they did in previous
releases.
 z/OS Introduction and Release Guide, GA22-7502
This document provides an overview of z/OS and lists the enhancements in each release.
Use this document to determine whether to obtain a new release, and to decide which
new functions to implement.
 z/OS Summary of Message and Interface Changes, SA22-7505
This document describes the changes to interfaces for individual elements and features of
z/OS. Use this document as a reference to the new and changed commands, macros,
panels, exit routines, data areas, messages, and other interfaces of individual elements
and features of z/OS.

18 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

Chapter 2. The resolver

TCP/IP protocols rely upon IP addressing in order to reach other hosts in a network to
communicate. For ease of use, instead of using the IP addresses represented by numbers,
they are sometimes mapped to easy-to-remember names. Typically, the names are assigned
to the IP address of the servers that many users can access, such as Web servers, FTP
servers, and TN3270 servers.

The resolver function allows applications to use names instead of IP addresses to connect to
other partners. The mapping of IP addresses and names is managed by name servers or
local definitions. The resolver queries those name servers, or searches local definitions, in
order to convert the name to an IP address or the IP address to a name. Using the resolver
relieves users of having to remember the decimal or hexadecimal IP addresses.

The resolver is important for enabling TCP/IP stacks or TCP/IP applications to establish
connections to other hosts.

This chapter discusses the following topics.

Section Topic

2.1, “Basic concepts of the resolver” on Basic concepts of the resolver

page 20

2.2, “The resolver address space” on Key characteristics of the resolver address space
page 21

2.3, “Implementing the resolver” on The configuration and verification tasks of resolver
page 42 implementation

2.4, “Problem determination” on page 50 Problem determination techniques

© Copyright IBM Corp. 2011. All rights reserved. 19

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

2.1 Basic concepts of the resolver

A resolver is a set of routines that acts as a client on behalf of an application. It reads a local
host file or accesses one or more Domain Name System (DNS) servers for name-to-IP
address or IP address-to-name resolution.

In most systems, in order for an application to reach a remote partner, it uses two commands
to ask the resolver what the IP address is for a host name, or vice versa. The commands are
gethostbyname(nnnnn) and gethostbyaddress([Link]). The IPv6-enabled
equivalent calls are getaddrinfo(nnnn) and getnameinfo(IPaddress).

Figure 2-1 illustrates the information request and response flows. The resolver gets a request
and, based on its own configuration file, will either look at a local hosts file or send a request
to a DNS server. After the relationship between the host name and IP address is established,
the resolver returns the response to the application.

Client Application Resolver Routines

-- [Link]
-- Get resolver info --
gethostbyname(hostx) send UDP query DomainOrigin [Link]
-- receive UDP reply NSInterAddr [Link]
connect([Link]) return IP address --
--

Give me IP address
for [Link]

DNS server [Link]

IP address Find an IP address that

corresponds to the name
is [Link]

Figure 2-1 How the resolver works

As mentioned, the resolver function allows applications to use names instead of IP addresses
to connect to other partners. Although using an IP address might seem to be an easy way to
establish such a connection, for applications that need to connect to numerous partners, or
for applications that are accessed by thousands of clients, using names is a much easier and
more reliable form of establishing access.

Another important reason to use names instead of IP addressing is that a user or an

application is not affected by the IP address changes to the underlying network.

20 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

Table 2-1 compares the benefits and drawbacks of the use of hard-coded IP addresses and
the two name resolution methods:
 The local hosts file
 The name server (DNS)

Table 2-1 Comparing the use of direct addressing with name resolution
Hard-coded IP Local hosts file Domain Name System
addresses (DNS)

Technology None - Use the entered Use gethostbyname() and Use gethostbyname() and
IP address directly on the let the resolver find an let the resolver contact the
connect() or sendto() IP address in the locally configured name server
socket call. configured hosts file. for an IP address.

Benefits Fast (no name resolution). Fast (local name IP address changes can
Good in some debugging resolution). be done without any local
situations (you know changes. All host names
exactly which IP address (in the entire network) can
is being used). be resolved.
A hierarchical name
space.

Drawbacks Difficult to remember If an IP addressing Additional packets

IP addresses. Very change is needed, all the (requests) flow to resolve
inconvenient if an local hosts files have to be a host name before a
IP address change updated. Only locally destination can be
occurs. Just think about configured host names reached.a
IPv6. can be resolved.
a. Resolver caching (discussed in 2.2.4, “Resolver DNS cache” on page 28) alleviates some of
the need for these flows by saving previously obtained information locally.

2.2 The resolver address space

In z/OS systems, the resolver works as a procedure. The resolver must be started before
TCP/IP stacks or any TCP/IP applications issue the resolver calls. It can be started in one of
the following ways:
 Default z/OS UNIX resolver
If no customized resolver address space is configured, the z/OS UNIX System Services
starts the default resolver. The default resolver is named RESOLVER. To use the default
RESOLVER address space, specify the RESOLVER_PROC(DEFAULT) statement or do
not specify any RESOLVER_PROC statements in BPXPRMxx.
 Customized resolver address space
The customized resolver address space can specify additional options to control the use
of the resolver configuration file. To create the customized resolver address space, create
a resolver started procedure and a SETUP data set to specify the additional options. The
customized resolver address space can be started automatically with the
RESOLVER_PROC(procname) statement in BPXPRMxx.

Although the resolver address space can be started manually, we recommend that you start
the resolver address space automatically during initialization of the UNIX System Services by
defining the RESOLVER_PROC() statement within BPXPRMxx.

Chapter 2. The resolver 21

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

After the resolver address space is activated, the global [Link] statements cannot be
overridden unless the MODIFY command is issued.

2.2.1 The resolver SETUP data set

The resolver SETUP data set is used by the customized resolver address space. The default
z/OS UNIX resolver does not read this file. The SETUP data set can include the following
statements:
 GLOBALTCPIPDATA
 DEFAULTTCPIPDATA
 GLOBALIPNODES
 DEFAULTIPNODES
 COMMONSEARCH or NOCOMMONSEARCH
 CACHE or NOCACHE
 CACHESIZE
 MAXTTL
 UNRESPONSIVETHRESHOLD

The use of each statement is discussed in later sections.

2.2.2 The resolver configuration file

The resolver configuration file is often called [Link]. In this file you can define how the
resolver should query the name-to-address or address-to-name resolution to the name
servers or search the local hosts file.

The configuration file can be an MVS data set or a z/OS UNIX Hierarchical File System (HFS)
file.

Note: The publication z/OS Communications Server: IP Configuration Guide, SC31-8775,

contains useful information about the characteristics that are required for the z/OS data
sets or file system files that contain resolver SETUP and configuration statements. The
guide also points out the security characteristics and file system permission settings that
are needed.

[Link] configuration statements

The following basic statements should be defined in the [Link] file.
 TCPIPJOBNAME (equivalent to TCPIPUSERID)
The name of the procedure used to start the TCP/IP address space. The default is TCPIP.
 DOMAIN (equivalent to DOMAINORIGIN)
The domain origin that is appended to the host name to form the fully qualified domain
name of a host.
 HOSTNAME
The TCP host name of the z/OS Communications Server server.
 LOOKUP
The order in which the DNS or local host files are to be searched for name resolution. By
default, DNS is looked up first. If caching is in effect, the resolver cache is considered to be
part of the “DNS” lookup step, and resolver will examine its cache data prior to actually

22 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

contacting any name server. Then if the resolution is unsuccessful, the local host files are
searched.
 NSINTERADDR (equivalent to NAMESERVER)
The IP address of a name server the resolver should query to.
 DATASETPREFIX
The high-level qualifier for the dynamic allocation of data sets. DATASETPREFIX is
referred to as the hlq of the TCP/IP stacks.
 NOCACHE
You must specify NOCACHE in the [Link] data set if you want to prevent
applications using this data set from either querying the cache or adding records to the
cache.

[Link] search order

On z/OS, the configuration file is located based on the search order. You must be mindful of
this search order, to ensure that the resolver works in the way you expect.

The TCP/IP applications execute a set of commands in the Sockets API Library to initiate a
request to the resolver in z/OS. The Sockets API Library uses one of the following socket
environments:
 Native MVS environment
 z/OS UNIX environment

Table 2-2 lists some of the APIs, z/OS applications, and user commands that use the active
MVS environment and the z/OS UNIX environment.

Table 2-2 Socket APIs, applications, and commands in Native MVS or z/OS UNIX environment
Native MVS environment z/OS UNIX environment

Socket APIs TCP/IP C Sockets Language Environment® C Sockets

TCP/IP Pascal Sockets UNIX System Services
TCP/IP REXX Sockets
TCP/IP Sockets Extended
IMS Sockets
CICS Sockets

z/OS Applications TN3270 Telnet server FTP

SMTP OMPROUTE
CICS Listener CSSMTP
LPD SNMP
Miscellaneous server z/OS UNIX OPORTMAP
PORTMAP z/OS UNIX OREXECD
RSHD z/OS UNIX ORSHD

Chapter 2. The resolver 23

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

Native MVS environment z/OS UNIX environment

User commands TSO FTP (batch) TSO FTP (command)

TSO NETSTAT netstat
TSO NSLOOKUP nslookup
TSO PING ping
TSO TRACERTE traceroute
TSO DIG ftp
TSO LPR host
TSO REXEC hostname
TSO RPCINFO dnsdomainname
TSO RSH dig
rexec
rpcinfo
sendmail
snmp

Each socket environment uses a different search order of the resolver configuration file, as
shown in Figure 2-2.

Native MVS environment z/OS UNIX environment

1. GLOBALTCPIPDATA 1. GLOBALTCPIPDATA
2. //SYSTCPD DD statement 2. RESOLVER_CONFIG
3. userid/[Link] environment variable
4. [Link](TCPDATA) 3. /etc/[Link]
5. DEFAULTTCPIPDATA 4. //SYSTCPD DD statement
6. [Link] 5. userid/[Link]
6. [Link](TCPDATA)
7. DEFAULTTCPIPDATA
8. [Link]

Figure 2-2 The resolver configuration file search order for each socket environment

Note: UNIX System Services Callable sockets use the z/OS UNIX environment search
order but cannot use the RESOLVER_CONFIG environment variable.

This provides the flexibility to control the resolver lookup differently, depending on which
socket API the application uses. However, because of the difference in search orders, it could
sometimes cause an unexpected result in the address resolution.

For example, if you set up /etc/[Link] as your resolver configuration file, the FTP server
application that uses the z/OS UNIX search order can resolve the name-to-address or
address-to-name successfully. However, the TN3270 server, which uses the native MVS
search order, would fail because /etc/[Link] is not included in its search list.

Using GLOBALTCPIPDATA
In order to deal with the complexity of the different search orders in the environments, the
GLOBALTCPIPDATA statement was introduced. Using the GLOBALTCPIPDATA statement,
you can use the same resolver configuration file throughout the z/OS system, because it is
the first choice in all socket search orders. This consolidation allows for consistent name
resolution processing across all TCP/IP applications.

To specify the GLOBALTCPIPDATA statement, you need to create a resolver started

procedure and its SETUP data set, instead of using the z/OS UNIX System Services default

24 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

RESOLVER address space. The use of the resolver address space and GLOBALTCPIPDATA
statement simplifies the resolver configuration on z/OS.

The [Link] file specified by the GLOBALTCPIPDATA statement is often called the
global [Link] file. If you define GLOBALTCPIPDATA, the following statements can be
included only in the global [Link] file:
 DomainOrigin/Domain or Search
 NSInterAddr/NameServer
 NSPortAddr
 ResolveVia
 ResolverTimeOut
 ResolverUDPRetries
 SortList

Other [Link] statements can be optionally included in the global [Link], and
the definition in the global [Link] always has precedence. If TCPIPJobname is specified
in both the global [Link] file and the local (non-global) [Link] file, then the one in
the global [Link] file is used.

If other [Link] statements, such as HostName and TCPIPJobname, cannot be found in

the global [Link] file, then the resolver continues its search according to the search
order of the each socket environment. The search stops when the file is found.

If statements such as HostName and TCPIPJobname cannot be found in that file either, the
defaults are applied. Note that it does not continue searching in the list. In other words, a
maximum of two files can be used (global [Link] file and one [Link] file in the
search order list).

Using GLOBALTCPIPDATA, the administrators can specify which statements should be

applied throughout the z/OS image, and decide which statements can be customized by each
socket environment by omitting those statements in the global [Link] file.

Note: In the Common INET (CINET) multi-stack environment, you should omit the
TCPIPJobname statement from the global [Link] file so that each TCP/IP stack, or
the applications that have affinity to a stack, can specify a local [Link] with its own
TCPIPJobname statement.

When using GLOBALTCPIPDATA in the CINET environment, the name server specified by
NSInterAddr or NameServer in the global [Link] file must be accessible from all
TCP/IP stacks that issue resolver calls.

Figure 2-3 on page 26 depicts the relationship between global [Link] and local
[Link].

Chapter 2. The resolver 25

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

SC30
TN3270
TCPIPA
server

Global [Link]
Local [Link]
DOMAINORIGIN [Link]
TCPIPJOBNAME TCPIPA
NSINTERADDR [Link]
HOSTNAME WTSC30A
NSPORTADDR 53
DATASETPREFIX TCPIPA
RESOLVEVIA UDP
MESSAGECASE MIXED
RESOLVERTIMEOUT 10
RESOLVERUDPRETRIES
LOOKUP LOCAL DNS

FTPDZ
TCPIPZ
server

Local [Link]
TCPIPJOBNAME TCPIPZ
HOSTNAME WTSC30Z
DATASETPREFIX TCPIPZ
MESSAGECASE MIXED

Figure 2-3 Using global [Link] and Local [Link]

Using DEFAULTTCPIPDATA
DEFAULTTCPIPDATA can be specified in the resolver SETUP data set to define the last
choice of the [Link] in the search order. The file specified by DEFAULTTCPIPDATA is
used when the application does not specify the local (non-global) [Link].

2.2.3 Local hosts file

The local hosts file lists the mapping of the IP addresses and the names just like the name
servers, but held locally on the server. The LOOKUP statement in the [Link]
configuration file defines whether the resolver address space performs the name resolution
only in the local files, or using the defined name server (including resolver cache, if active), or
both, in any specified order.

Using COMMONSEARCH
When the local hosts file is searched, the search order for the native MVS environment and
the z/OS UNIX environment are different. The difference in the search orders adds complexity
to configuration tasks and can lead unexpected results of the name resolution.

The simpler approach is to utilize the COMMONSEARCH statement in the resolver SETUP
data set. By specifying COMMONSEARCH, native MVS and z/OS UNIX environments use
the same search order as shown in Figure 2-4 (except the RESOLVER_IPNODES
environment variable, which is only supported by the z/OS UNIX environment). In both
environments, the first choice is the file specified by GLOBALIPNODES statement, which is
defined in the resolver SETUP data set.

The local hosts files looked up in this search order are typically called [Link] files.
When the COMMONSEARCH is specified in the resolver SETUP data set, it uses the same
search order for both IPv4 and IPv6 queries. You can list both IPv4 and IPv6 addresses in the
[Link] file.

26 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

Native MVS environment z/OS UNIX environment

(COMMONSEARCH specified) (COMMONSEARCH specified)
IPv4 or IPv6 host name or address IPv4 or IPv6 host name or address
search: search:

1. GLOBALIPNODES 1. GLOBALIPNODES
2. userid/[Link] 2. RESOLVER_IPNODES
3. [Link] environment variable
4. DEFAULTIPNODES 3. userid/[Link]
5. /etc/ipnodes 4. [Link]
5. DEFAULTIPNODES
6. /etc/ipnodes

Figure 2-4 Local hosts file search order with COMMONSEARCH specified

To determine which environment is used for a particular socket’s APIs, applications, or

commands, refer to Table 2-2 on page 23.

If COMMONSEARCH is not specified in the resolver SETUP data set, then the default is
NOCOMMONSEARCH and the default search order shown in Figure 2-5 on page 27 is used.

Using GLOBALIPNODES
The GLOBALIPNODES statement specifies the global local host file that is to be used in the
entire z/OS image, regardless of which environment (native MVS or z/OS UNIX) that the
applications or sockets API use. To put the GLOBALIPNODES statement into effect for the
name resolution of IPv4 addresses, also specify COMMONSEARCH in the resolver SETUP
data set.

Using DEFAULTIPNODES
The DEFAULTIPNODES statement specifies the last candidate of the local host file search.
To put the DEFAULTIPNODES statement into effect for the name resolution of IPv4
addresses, also specify COMMONSEARCH in the resolver SETUP data set.

Default local hosts file search order

If NOCOMMONSEARCH (the default) is specified in the resolver SETUP data set or default
z/OS UNIX resolver is used, the default local hosts file search order shown in Figure 2-5 is
used. The default local hosts file search order only applies to the query of IPv4 addresses.
The query for IPv6 addresses always uses the search order listed in Figure 2-4.

Native MVS environment z/OS UNIX environment

(NOCOMMONSEARCH specified) (NOCOMMONSEARCH specified)
IPv4 host name or address search: IPv4 host name or address search:

1. [Link] or 1. X_SITE or X_ADDR environment

[Link] variable
2. [Link] or 2. /etc/hosts
[Link] 3. [Link] or
[Link]
4. [Link] or
[Link]

Figure 2-5 Local hosts file search order with NOCOMMONSEARCH specified (default)

Chapter 2. The resolver 27

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

2.2.4 Resolver DNS cache

In order to provide better system performance, consider eliminating redundant network flows
to DNS servers. You can accomplish this goal by exploiting the resolver DNS cache. This
uses the resolver for system-wide caching of Domain Name System (DNS) responses.

Using CACHE or NOCACHE

The resolver cache is enabled by default and is shared across the entire z/OS system image.
If you are currently running a caching-only DNS name server, you might be able to use the
resolver DNS cache instead; the resolver DNS cache provides the same function with better
system performance.

Two of the new resolver setup file statements are CACHE and NOCACHE. The CACHE
statement, which is the default, explicitly indicates that resolver caching is active across the
entire system. The NOCACHE statement explicitly indicates that resolver caching is not
active across the entire system. You must code NOCACHE if you want to maintain the current
level of resolver processing. The setting of CACHE or NOCACHE can be changed
dynamically using the MODIFY RESOLVER,REFRESH,SETUP command. If you change
from a setting of CACHE to a setting of NOCACHE dynamically, any existing cache records
are immediately deleted.

Figure 2-6 shows how the resolver DNS cache works.

query for
[Link] z/OS LPAR
4,6
1,5 2

RESOLVER Name Server

[Link]
CACHE
2

3
[Link]
Name Server
NSINTERADDR [Link] [Link]
NSINTERADDR [Link]

Figure 2-6 How the cache works

In step 1, an application delivers a request to translate the host name [Link]

into an IP address. The resolver, in step 2, forwards the request to the first DNS name server
specified in the list of name servers in the [Link] data set. The response from the name
server in step 3 is returned to the application in step 4. If the first DNS name server ([Link])
does not respond in time, the resolver forwards the request to the second name server
([Link]) in the list. In this point, the resolver now saves the information into the local resolver
cache. At step 5, when the second request for the host name translation is received, the
resolver first queries the local cache for data about the host name. In this example, the
information is there, and is still valid, so the resolver returns the response data immediately to
the application (step 6).

28 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

Using CACHESIZE(size)
CACHESIZE indicates how much storage the cache function can use to hold resolver cache
information. The valid range for size is 1 MB to 999 MB. The default is 200 MB. For planning
purposes, assume a megabyte of data holds slightly more than 400 entries and consider
coding a CACHESIZE at least 50% greater than your expected needs.

Tip: When CACHESIZE is specified with NOCACHE, the value is ignored.

Important: You can modify the CACHESIZE using the MODIFY REFRESH,SETUP
command, but you can only increment the storage amount or keep it the same. To
decrease the value of CACHESIZE M, you must stop and restart the resolver.

Using MAXTTL(time)
MAXTTL indicates the longest amount of time that the resolver cache can use saved
information. The valid range for time is 1 to 2147483647 (seconds). The default is
2,147,483,647, which is the largest TTL a name server can return.

You can dynamically change MAXTTL using the MODIFY REFRESH,SETUP command.
Changing the value of MAXTTL has no impact on any records currently in the cache. The
value can be increased or decreased, but the new value only affects records created after the
MODIFY RESOLVER command completes.

Tip: When MAXTTL is specified with NOCACHE, the value is ignored.

Using the cached information

The resolver caching function does not impact the data that is presented to the application
across the resolver APIs. The same control block structures are used for returning the
information. Applications invoking the resolver should not detect any difference between data
supplied from the cache and data that had to be retrieved from a name server.

Furthermore, the cache function is designed to allow resource information to be re-used by

compatible API calls. For example, if Getaddrinfo is used to obtain IPv4 addresses for a host
name, that same cached information can be retrieved later using Gethostbyname. The same
capability exists for Getnameinfo and Gethostbyaddr calls in terms of host names obtained
from an IPv4 address. IPv6 processing is only available using Getaddrinfo and Getnameinfo,
so IPv6 information cannot be shared in this manner. In addition, the resolver caching
function will handle translating the cache information from EBCDIC to ASCII, or vice versa, so
cached information is available using either protocol.

One function not provided by the resolver caching function is the ability to return the cached
IP addresses in a different, or “round robin”, order than they were received from the name
server. The resolver returns the addresses in the same order all the time. It should be noted
that Getaddrinfo processing re-orders the list of addresses already; refer to “Default
destination address selection” on page 38 for details. Similarly, if you have SORTLIST
configuration statements coded, they will also re-order the list of addresses into a more
predictable pattern.

Testing the resolver DNS cache

Verify if the resolver is able to cache the expected name-to-address resolution. First, we
delete all information from the resolver cache using the modify resolver,flush,all
command, as shown in Example 2-1.

Chapter 2. The resolver 29

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

Example 2-1 MODIFY RESOLVER,FLUSH,ALL command display

F RESOLV30,FLUSH,ALL
EZZ9305I 1 CACHE ENTRIES DELETED
EZZ9293I FLUSH COMMAND PROCESSED

Display cache entry data using netstat RESCache command

You can use the netstat RESCache command in the console or TSO command line to display
information regarding the resolver cache. In the Unix System Services (now called z/OS
UNIX) environment the same command is netstat -q. Two main types of information can be
displayed: statistical information and actual resource information.

You can specify additional modifiers or filters to influence the amount of cache data that is
displayed. For statistical information, you can add the DNS modifier to have the overall
statistics broken into statistical information on a name server IP address basis. You have even
more options for detailed entry information reports. You can filter the information by the IP
address of the name server that provided the information. You can filter the information so
that only entries related to a specific host name or IP address value are displayed. You can
display only negative cache information from the cache, either all entries or subsets of entries
based on name server IP address, host name value, or IP address value.

We then verify if the name [Link] is in the cache by using the netstat -q
command, as shown in Example 2-2.

Example 2-2 netstat -q DETAIL command display

CS02 @ SC30:/u/cs02>netstat -p tcpipa -q DETAIL -H [Link]

MVS TCP/IP NETSTAT CS V1R12 TCPIP Name: TCPIPA [Link]

Now we can resolve [Link] using a ping command, and reenter the netstat -q
command, as shown in Example 2-3 and Example 2-4.

Example 2-3 UNIX ping command display

CS02 @ SC30:/u/cs02>ping -p tcpipa [Link]
CS V1R12: Pinging host [Link] ([Link])
Ping #1 response took 0.000 seconds.

Example 2-4 netstat -q DETAIL command display

CS02 @ SC30:/u/cs02>netstat -p tcpipa -q DETAIL -H [Link]

MVS TCP/IP NETSTAT CS V1R12 TCPIP Name: TCPIPA [Link]
HostName to IPAddress translation
---------------------------------
HostName: [Link] 1
DNS IPAddress: [Link] 2
DNS Record Type: T_A
Canonical Name: [Link]
Cache Time: 09/27/2010 [Link]
Expired Time: 09/27/2010 [Link] 3
Hits: 1 4
IPAddress: [Link] 5

30 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

In Example 2-4 on page 30, the numbers correspond to the following information:
1. The entry-name is already in the cache.
2. The DNS Server IP address where the resolver found the entry-name [Link].
3. The expiration time, which is 600 seconds. This is the time in the MAXTTL statement or in
the TTL value for this entry, as supplied by the name server at [Link].
4. How many times this entry-name ([Link]) was used by the resolver while
remaining in the cache.
5. The IP address of the entry-name [Link].

In Example 2-2 on page 30, the resolver had not yet cached [Link]. However, after a
ping command, [Link] was cached, as shown in Example 2-4 on page 30.

Now we display the cache statistics by using the netstat -q SUMMARY DNS command, as
shown in Example 2-5.

Example 2-5 netstat -q SUMMARY DNS command display

CS02 @ SC30:/u/cs02>netstat -p tcpipa -q SUMMARY DNS

MVS TCP/IP NETSTAT CS V1R12 TCPIP Name: TCPIPA [Link]
Storage Usage:
Maximum: 20M 1
Current: 19K MaxUsed: 21K 2
Cache Usage:
Total number of entries: 1
Non-NX entries: 1
A: 1 AAAA: 0 PTR: 0
NX entries: 0
A: 0 AAAA: 1 PTR: 0
Queries: 1 Hits: 0
SuccessRatio: 0% 3

DNS address: [Link] 4

Total number of entries: 1 5
Non-NX entries: 1
A: 1 AAAA: 0 PTR: 0
NX entries: 0
A: 0 AAAA: 0 PTR: 0
References: 1 Hits: 0

In this example, the numbers correspond to the following information:

1. The maximum amount of storage permitted, or CACHESIZE
2. The current amount of storage in use and maximum amount the resolver has ever used for
caching since the resolver was started
3. Percentage of queries satisfied by information in the cache
4. IP address of the name server providing cache data
5. Number of entries in cache, grouped by negative (NX) entries and other (Non-NX) entries

Cache usage statistics include the total number of entries in the cache and the volume of
activity involving the cache. The number of entries is differentiated between negative cache
entries and non-negative cache entries. Within each of these main categories, the number of

Chapter 2. The resolver 31

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

DNS A, AAAA, and PTR records is indicated. These same subsets of entries are displayed for
individual name servers.

The number of resolver cache requests and how often usable data was returned by the cache
gives you a sense of the efficiency of your cache operations. Note that a single resolver API
call can generate multiple cache queries. For example, a Getaddrinfo request for both IPv6
and IPv4 addresses generates two cache queries. On an individual name server level, the
“References” value indicates the number of times the set of cache information provided by
this name server was examined. Typically, the sum of the name server “References” values is
greater than the number of cache queries, because multiple name server information sets
can be examined as part of one cache query.

Now, we display the cache statistics by using the netstat -q DETAIL command, to display
information about a specific cache entry, as shown in Example 2-6 on page 32.

Example 2-6 netstat -q DETAIL command display

CS02 @ SC30:/u/cs02>netstat -p tcpipa -q DETAIL -H [Link]

MVS TCP/IP NETSTAT CS V1R12 TCPIP Name: TCPIPA [Link]
HostName to IPAddress translation
---------------------------------
HostName: [Link]
DNS IPAddress: [Link] 1
DNS Record Type: T_A
Canonical Name: [Link]
Cache Time: 09/27/2010 [Link] 2
Expired Time: 09/27/2010 [Link] 3
Hits: 1 4
IPAddress: [Link] 5

In this example, the numbers correspond to the following information:

1. The DNS that provided the entry
2. The time and the date of cache entry creation
3. The time and date when the entry will expire
4. The number of times this entry has been re-used
5. IP addresses provided by the specified name server

This is a partial example of a netstat report showing a detailed cache entry. The reports are
formatted so that DNS A and AAAA records are displayed as one group, and DNS PTR
records are displayed as a second group. Negative cache entries can appear in either group,
in any order, and are identified using the notation “***NA***”.

For each record, the cache entry key, or the target resource that was searched for to acquire
this cache information, is the first line of the entry. After that, the two types of entries are very
similar. The IP address of the DNS name server that supplied this particular information is
displayed, allowing you to see what values were provided by what name servers. In the case
of DNS A and AAAA record entries, the host name used to create the record might really be
an alias or nickname for the official name of the resource. For that reason, the display
includes the official, or canonical, name, regardless of whether the names match or not.
There is no canonical name concept for DNS PTR records.

Two time values are displayed: one is the time and the date of cache entry creation. The other
is the time and date when the entry will expire, based on name server TTL or MAXTTL

32 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

setting. The netstat RESCACHE report will not include any resources that are in the cache that
represent expired information. The number of times this entry has been re-used is displayed
as the “Hits” value. Finally, for DNS A and AAAA entries, up to 35 IP addresses provided by
the specified name server for the host name value are included. For DNS PTR entries, the
one host name associated with the input IP address (either IPv4 or IPv6) is included.

2.2.5 Criteria for indicating an unresponsive DNS name server

A DNS name server is considered unresponsive for a specific query when:
 The resolver sends a UDP or TCP query to a name server and never receives a response.
 The resolver sends a UDP query to a name server and receives a response after the
RESOLVERTIMEOUT value has expired.
 The resolver attempts to send data to a name server using UDP, but the data cannot be
sent to the target IP address (for example, because of an error in the route configuration).
 The resolver attempts to connect to a name server using TCP, but the connection attempt
times out.

The unresponsive DNS notification function is enabled by default. It can be turned off by
specifying the UNRESPONSIVETHRESHOLD configuration statement with a value of zero
(0).

2.2.6 Unresponsive DNS name servers

Communications Server for z/OS IP has the capability of notifying the operator console when
a defined Domain Name System (DNS) name server does not respond to resolver queries
during the most recent sliding 5-minute interval.

Resolver also provides statistics for each currently unresponsive name server, regarding the
number of queries attempted and the number of queries which received no response during a
sliding 5-minute interval.

CS for z/OS IP considers a DNS name server to be unresponsive when the number of
unsuccessful queries exceeds a percentage threshold of the total queries sent during a
5-minute interval. By default, the percentage threshold is 25% of the total queries. This
percentage can be customized using the UNRESPONSIVETHRESHOLD configuration
statement in the resolver setup file.

The percentage threshold value can also be changed while the resolver is active, by changing
the UNRESPONSIVETHRESHOLD configuration statement in the resolver setup file and
issuing the MODIFY resolver,REFRESH,SETUP=setup_file_name command.

Resolver notifications for DNS name server responsiveness

When the resolver detects that a name server is not being responsive, based on the provided
failure threshold, network operator messages are issued to report the problem, as shown in
Example 2-7:

Example 2-7 Notifying DNS responsiveness

*EZZ9308E UNRESPONSIVE NAME SERVER DETECTED AT IP ADDRESS [Link]
EZZ9310I NAME SERVER [Link]
TOTAL NUMBER OF QUERIES SENT 132
TOTAL NUMBER OF FAILURES 132
PERCENTAGE 100%

Chapter 2. The resolver 33

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

The error message EZZ9308E is issued only once. It remains on the operator console for as
long as the resolver considers the name server to be unresponsive. During that time of
unresponsiveness, the informational message EZZ9310I is reissued every 5 minutes, giving
updated statistics for the unresponsive name server for that sliding 5-minute interval.

If by the end of a subsequent monitor interval, the resolver determines that the name servers
failure rate has dropped below the threshold value, the resolver considers this name server to
be responsive again, clears message EZZ9308E from the operator console and issue a
message indicating the DNS is responsive again, as shown in Example 2-8:

Example 2-8 Notifying DNS is now responsive

EZZ9309I NAME SERVER IS NOW RESPONSIVE AT IP ADDRESS [Link]
EZZ9310I NAME SERVER [Link]
TOTAL NUMBER OF QUERIES SENT 190
TOTAL NUMBER OF FAILURES 19
PERCENTAGE 10%

Considerations about UNRESPONSIVETHRESHOLD usage

When you specify the UNRESPONSIVETHRESHOLD value, consider the following factors
that have an impact on the network environment:
 Specifying a small value may generate a large number of console messages.
 Specifying a value that is too large may result in intermittent problems with the DNS name
server or the IP network not being detected.
 Consider using a higher value for UNRESPONSIVETHRESHOLD if you use a small value
for RESOLVERTIMEOUT. If you set a very short timeout value, even temporary problems
in the network might generate unnecessary unresponsive name server messages to the
operator.
 The values specified on the RESOLVERUDPRETRIES, SEARCH, and NAMESERVER
statements in the [Link] file can affect the number of messages generated by the
system resolver.

2.2.7 Affinity servers and generic servers

In the multiple stack environment, a TCP/IP application might have an affinity to a specific
TCP/IP stack. When designing a multiple stack system, it is important to check each
application that will be used and how it will be implemented in the environment.

Affinity server
An affinity server is an application that has affinity to a specific TCP/IP stack; it provides
service to the clients that are connected through the TCP/IP stack to the applications.

In this case, you need to code a TCP/IPJobname statement that represents the application in
order to direct traffic to a specific stack. So, when designing the global definitions in the
resolver address space, do not code a TCPIPJobname statement in GLOBALTCPIPDATA.
Instead, allow it to be coded in the local [Link].

A native TCP/IP sockets program will always use one stack only, and by default, it will be the
stack that is identified in the TCPIPJOBNAME option in the chosen resolver configuration file.
However, the stack can also be chosen through the program configuration and API calls to
associate the program with a chosen stack, as shown in Figure 2-7 on page 35.

34 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

Native MVS
TCPIP Jobname Socket
TCPB Program

Inbound Outbound
BPX Callable Sockets

Pre-routing Table
C-INET

TCPA TCPB TCPC

X Y Z V

Figure 2-7 Native TCP/IP applications in a multiple stack environment

Applications using UNIX System Services callable APIs or Language Environment C/C++
sockets APIs can also use a specific bind to open a socket. A bind-specific server socket will
only receive connections from the stack that owns the IP address to which the socket is
bound. Outbound connections or UDP datagrams will be handled by the stack that offers the
best route to the destination IP address, as shown in Figure 2-8.

socket( )
Application-specific bind(8001, Y)
Configuration Data listen( )

Inbound Outbound
BPX Callable Sockets

Pre-routing Table
C-INET

TCPA TCPB TCPC

X Y Z V

Figure 2-8 APIs, bind-specific

Chapter 2. The resolver 35

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

Generic server
A generic server is a server without an affinity to a specific stack, and it provides service to
any clients that are connected to any TCP/IP stacks on the system.

When using the generic bind, it does not matter if the chosen resolver configuration file has a
TCPIPJobname; it is not used when the server is a pure generic server.

36 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

Applications using UNIX System Services callable APIs or Language Environment C/C++
sockets APIs can be implemented using a generic bind to open the same port in all TCP/IP
stacks. By doing so, the application will accept incoming connections or UDP datagrams over
any interface of all connected stacks, as shown in Figure 2-9.

socket( )
Application-specific bind(8001, Y)
Configuration Data listen( )

Inbound Outbound
BPX Callable Sockets

Pre-routing Table
C-INET

TCPA TCPB TCPC

X Y Z V

Figure 2-9 APIs, generic bind

Outbound connections or UDP datagrams are processed by the C-INET pre-router, and the
stack with the best route to the destination is chosen.

When using a generic bind, the server port number must be reserved in all stacks. If one
stack has it reserved to another address space, the bind() call fails.

2.2.8 Resolving an IPv6 address

IPv6 support introduces several changes to how host name and IP address resolution is
performed. These changes affect several areas of resolver processing, including:
 Resolver APIs were introduced for IPv6-enabled applications.
 An algorithm is defined to describe how a resolver needs to sort a list of IP addresses
returned for a multihomed host.
 DNS resource records are defined to represent hosts with IPv6 addresses, and therefore
network flows between resolvers and name servers (instead of DNS IPv4 A records).

Resolver support for IPv6 connections to DNS name servers

Communications Server for z/OS IP allows the system resolver to send requests to the
Domain Name System (DNS) name servers using IPv6 communication. To implement IPv6
communication with a DNS name server, specify the IPv6 address of the server on the
existing NSINTERADDR and NAMESERVER resolver configuration statements in the
[Link] data set.

Chapter 2. The resolver 37

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

Restriction: The res_state structure (nsaddr_list) contains only the IPv4 addresses coded
on the NSINTERADDR or NAMESERVER statements. Applications that examine or
update the nsaddr_list cannot manipulate the IPv6 addresses.

IPv6 resolver statements

[Link] is a local host file (in the style of /etc/hosts), which can contain both IPv4 and
IPv6 addresses. IPv6 addresses can only be defined in [Link]. This file allows the
administration of local host files to more closely resemble that of other TCP/IP platforms, and
eliminates the requirement of post-processing the files (specifically, MAKESITE).

The IPv6 search order is same as the COMMONSEARCH search order, as shown in
Figure 2-4 on page 27. If you do not want to use the COMMONSEARCH search order for
existing IPv4 local hosts files, you might need to maintain two different local host files (for
example, IPv4 addresses in [Link], and IPv6 and IPv4 addresses in
[Link]).

Name and address resolution functions

The APIs such as getaddrinfo, getnameinfo, and freeaddrinfo allow applications to resolve
host names to IP addresses and vice versa for IPv6. The APIs are designed to work with both
IPv4 and IPv6 addressing. The use of these APIs should be considered if an application is
being designed for eventual use in an IPv6 environment.

The manner in which host name (getaddrinfo) or IP address (getnameinfo) resolution is

performed is dependent upon resolver specifications contained in the resolver SETUP data
sets and [Link] configuration data sets, just like IPv4 address resolution. These
specifications determine whether the APIs will query a name server first and then search the
local host files, or whether the order will be reversed—or even if one of the steps will be
eliminated completely. The specifications also control whether local host files have to be
searched, and which local host file will be accessed.

Default destination address selection

Resolver APIs have the capability to return multiple IP addresses as a result of a host name
query. However, many applications only use the first address returned to attempt a
connection or to send a UDP datagram. Therefore, the sorting of these IP addresses is
performed by the default destination address selection algorithm.

Establishing connectivity might depend on whether an IPv6 address or an IPv4 address is

selected, thus making this sorting function even more important. Default destination address
selection only occurs when the system is enabled for IPv6 and the application is using the
getaddrinfo() API to retrieve IPv6 or IPv4 addresses.

The default destination address selection algorithm takes a list of destination addresses and
sorts them to generate a new list. The algorithm sorts together both IPv6 and IPv4 addresses
by a set of rules.

The following rules are applied, in order, to the first and second address, choosing a best
address. Rules are then applied to this best address and the third address. This process
continues until rules are applied to the entire list of addresses.
Rule 1 Avoid unusable destinations. If one address is reachable (the stack has a route
to the particular address) and the other is unreachable, then place the
reachable destination address prior to the unreachable address.

38 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

Rule 2 Prefer matching scope. If the scope of one address matches the scope of its
source address and the other address does not meet this criteria, then the
address with the matching scope is placed before the other destination
address.
Rule 3 Avoid deprecated addresses. If one address is deprecated and the other is
non-deprecated, then the non-deprecated address is placed prior to the other
address.

Terminology: Deprecated, in this context, means that the state of an IPv6 address has
changed from preferred state (the address was leased to an interface for a fixed, possibly
infinite, length of time) to deprecated state. (When a lifetime expires, the binding and
address can become invalid, and the address can be reassigned to another interface
elsewhere on the Internet.) While in a deprecated state, the use of an address is
discouraged but not strictly forbidden.

Rule 4 Prefer matching address formats. If one address format matches its associated
source address format and the other destination does not meet this criteria,
then place the destination with the matching format prior to the other address.
Rule 5 Prefer higher precedence. If the precedence of one address is higher than the
precedence of the other address, then the address with the higher precedence
is placed before the other destination address.
Rule 6 Use the longest matching prefix. If one destination address has a longer
CommonPrefixLength with its associated source address than the other
destination address has with its source address, then the address with the
longer CommonPrefixLength is placed before the other address.
Rule 7 Leave the order unchanged. No rule selected a better address of these two;
they are equally good. Choose the first address as the better address of these
two and the order is not changed.

2.2.9 Resolver support for EDNS0

An early implementation of DNS, which is discussed in RFC 1035, allows only a maximum of
512 bytes for any DNS packet sent through UDP. This limitation inhibits DNS performance
because, when a DNS server or client needs to communicate with a large amount of data, it
will have to use the bulky TCP protocol (higher performance cost) instead of the simple UDP
protocol (lower performance cost).

Extension Mechanism for DNS (EDNS0) was introduced in RFC 2671 to address the
performance improvement limitation imposed by the traditional DNS implementation. The IBM
implementation of the EDNS0 standard allows DNS communication of up to 3072 bytes using
UDP. This implementation improves DNS’s ability to communicate a large amount of data,
such as IP version 6 (IPv6).

z/OS Communications Server resolver supports the EDNS0 standard by default. No

additional steps are needed to enabled this feature. However, the following dependencies are
required for the resolver to support EDNS0 properly:
 The DNS name server must also support EDNS0 protocols in order to use UDP packets
larger than 512 bytes.
 Firewalls that exist between the DNS name server and the z/OS resolver must be
configured to accept DNS messages sent as UDP packets of greater than 512 bytes in
order to use EDNS0 protocols.

Chapter 2. The resolver 39

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

In rare situations where the DNS server was just upgraded to support EDNS0, a refresh of
the z/OS resolver is required so that it can relearn the DNS server EDNS0 capabilities. Issue
MODIFY RESOLVER,REFRESH to the resolver address space to refresh.

2.2.10 Considerations
To implement the resolver address space, it is important to first determine whether your
environment requires a single TCP/IP stack or multiple TCP/IP stacks. In both cases the
resolver is an independent address space and has to be up and running before the TCP/IP
stack is started.

The statements defined in the global [Link] cannot be overridden by the local
[Link] file of the each TCP/IP stack. The local [Link] file can only specify the
statement if it is not already defined in the global [Link] file.

Important: In some resolver environments, the use of the trace functions (such as
SockDebug or TraceResolver) might affect performance. Therefore, we recommend using
the method that we describe in “CTRACE: RESOLVER (SYSTCPRE)” on page 55.

The resolver in a single stack environment

We recommend that you create a global [Link] file for a single stack environment. The
TCPIPJobname statement can be coded in a global [Link] file or in the local
(non-global) [Link] file, because there is only one stack on the system. If some
applications have requirements to specify their own [Link] statements, then omit them
from the global [Link] so the applications can point to the local [Link] file to be
used.

The resolver in a multiple stack environment

When implementing for a multiple stack environment, each TCP/IP stack should use a local
[Link] file specifying stack-specific statements, such as TCPIPJobname and
HostName. Optionally, you can merge some statements that can be applied to all TCP/IP
stacks and all TCP/IP applications to a global [Link] file. You need to determine which
statements should be defined in the global [Link] and used in the entire z/OS image.
This will depend on how much you want to allow each stack or application to define its own
definitions.

In the multiple stack environment, we recommend that you create a global [Link] if all
the statements needed in the global [Link] (see “Using GLOBALTCPIPDATA” on
page 24) can be applied to all the stacks as shown in Figure 2-3 on page 26. If not, do not use
the global [Link] and only use local [Link] for each stack.

40 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

Figure 2-10 depicts the multiple stack environment without the use of a global [Link].

SC30
TN3270 FTPDZ
TCPIPA TCPIPZ
server server

Local [Link] Local [Link]

TCPIPJOBNAME TCPIPA TCPIPJOBNAME TCPIPZ
HOSTNAME WTSC30A HOSTNAME WTSC30Z
DATASETPREFIX TCPIPA DATASETPREFIX TCPIPZ
MESSAGECASE MIXED MESSAGECASE MIXED
DOMAINORIGIN [Link] DOMAINORIGIN [Link]
NSINTERADDR [Link] NSINTERADDR 10.2001.2
NSPORTADDR 53 NSPORTADDR 53
RESOLVEVIA UDP RESOLVEVIA UDP
RESOLVERTIMEOUT 10 RESOLVERTIMEOUT 10
RESOLVERUDPRETRIES RESOLVERUDPRETRIES
LOOKUP LOCAL DNS LOOKUP DNS LOCAL

Figure 2-10 The multiple stack environment without global [Link]

Recommendation: Although there are specialized cases where multiple stacks per LPAR
can provide value, generally we recommend implementing only one TCP/IP stack per
LPAR. The reasons for this recommendation are as follows:
 A TCP/IP stack is capable of exploiting all available resources defined to the LPAR in
which it is running. Therefore, starting multiple stacks will not yield any increase in
throughput.
 When running multiple TCP/IP stacks, additional system resources, such as memory,
CPU cycles, and storage, are required.
 Multiple TCP/IP stacks add a significant level of complexity to TCP/IP system
administration tasks.
 It is not necessary to start multiple stacks to support multiple instances of an application
on a given port number, such as a test HTTP server on port 80 and a production HTTP
server also on port 80. This type of support can instead be implemented using
BIND-specific support where the two HTTP server instances are each associated with
port 80 with their own IP address, using the BIND option on the PORT reservation
statement.

One example where multiple stacks can have value is when an LPAR needs to be
connected to multiple isolated security zones in such a way that there is no network level
connectivity between the security zones. In this case, a TCP/IP stack per security zone can
be used to provide that level of isolation, without any network connectivity between the
stacks.

Chapter 2. The resolver 41

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

2.3 Implementing the resolver

In this scenario, we use the customized resolver address space and specify
GLOBALTCPIPDATA, DEFAULTTCPIPDATA, and GLOBALIPNODES in the resolver SETUP
data set. We define a global [Link] file and define a common set of parameters for
entire z/OS image. We omit some statements in the global [Link] file so that the
applications or TCP/IP stack can use their own local [Link] file for the statements
undefined in the global [Link] file.

Figure 2-11 depicts the environment that we use for this implementation.

RESOLV30 TCPIP stack

Global [Link] Local [Link]

DOMAINORIGIN [Link] TCPIPJOBNAME TCPIPA
NSINTERADDR [Link] HOSTNAME WTSC30A
NSPORTADDR 53 DATASETPREFIX TCPIPA
RESOLVEVIA UDP MESSAGECASE MIXED
RESOLVERTIMEOUT 10
RESOLVERUDPRETRIES
LOOKUP LOCAL DNS

Global [Link]
[Link] WTSC30A
[Link] WTSC31B
[Link] WTSC32C
[Link] router1
[Link] router2 ...

Figure 2-11 Our resolver environment on SC30

2.3.1 Implementation tasks

To implement the resolver address space in our test environment, we performed these steps:
1. Set up the resolver started procedure.
2. Customize BPXPRMxx.
3. Configure the resolver SETUP data set.
4. Create the global [Link] file.
5. Create the default [Link] file.
6. Create the global IPNODES data set.
7. Create a [Link] file for TCPIPA stack.
8. Create the TCPIPA stack started procedure.

We describe these steps in the sections that follow.

Set up the resolver started procedure

We created the resolver procedure so that it would start during the UNIX System Services
initialization.

To create the procedure, we copied the sample procedure [Link](EZBREPRC) and

customized it to our environment, as shown in Example 2-9 on page 43. The procedure has
only one DD card that must be configured, the SETUP DD card 1, which describes where the
SETUP data set is located.

42 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

Example 2-9 The resolver started procedure

/*****************************************
/* [Link](RESOLV30)
/*****************************************
//RESOLV30 PROC PARMS='CTRACE(CTIRES00)' 1
//EZBREINI EXEC PGM=EZBREINI,REGION=0M,TIME=1440,PARM=&PARMS
//* SETUP contains resolver setup parameters.
//* See the section on "Understanding Resolvers" in the
//* IP Configuration Guide for more information. A sample of
//* resolver setup parameters is included in member RESSETUP
//* of the SEZAINST data set.
//*
//SETUP DD DSN=[Link](RESOLV&SYSCLONE),DISP=SHR,FREE=CLOSE 2

In this example, the numbers correspond to the following information:

1. The name of the resolver procedure is RESOLV30.
2. Specifies the resolver SETUP data set. The &SYSCLONE MVS system symbol value on
this system is 30.

Customize BPXPRMxx
We customized the RESOLVER_PROC statement in BPXPRMxx, to specify the procedure
name that we used, which causes the resolver to start automatically the next time z/OS UNIX
System Services initializes. Example 2-10 shows the partial contents of BPXPRMxx.

Example 2-10 Specifying the resolver procedure to be started

/*****************************************
/* [Link](BPXPRM00)
/*****************************************
/* RESOLVER_PROC is used to specify how the resolver address space */
/* is processed during Unix System Services initialization. */
/* The resolver address space is used by Tcp/Ip applications */
/* for name-to-address or address-to-name resolution. */
/* In order to create a resolver address space, a system must be */
/* configured with an AF_INET or AF_INET6 domain. */
/* RESOLVER_PROC(procname|DEFAULT|NONE) */
/* procname - The name of the address space for the resolver. */
/* In this case, this is the name of the address */
/* space as well as the procedure member name */
/* in [Link]. procname is 1 to 8 characters */
/* long. */
/* DEFAULT - An address space with the name RESOLVER will */
/* be started. This is the same result that will */
/* occur if the RESOLVER_PROC statement is not */
/* specified in the BPXPRMxx profile. */
/* */
/* NONE - Specifies that a RESOLVER address space is */
/* not to be started. */
/* @DAA*/
/********************************************************************/
RESOLVER_PROC(RESOLV&SYSCLONE.) 1

Chapter 2. The resolver 43

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

In this example, the numbers correspond to the following information:

1. Specifies the name of the resolver procedure we created in the previous step. The
&SYSCLONE MVS system symbol value on this system is 30.

Important: When the resolver is started by UNIX System Services, you must pay attention
to the following information:
 The resolver address space is started by SUB=MSTR. This means that JES services
are not available to the resolver address space. Therefore, no DD cards with SYSOUT
can be used.
 The resolver start procedure needs to reside in a data set that is specified by the
MSTJCLxx PARMLIB member’s IEFPDSI DD card specification. Otherwise, the
procedure will not be found and the resolver will not start. [Link] is usually
one of the libraries specified there.

Configure the resolver SETUP data set

We configured the resolver SETUP data set which is specified with the SETUP DD definition
in the resolver started procedure. This data set defines the location of the global and default
[Link] files containing the parameters we wanted to be defined in the z/OS
environment.

In our test environment, we copied the SETUP sample data set and changed its contents to
meet our requirements, as shown in Example 2-11.

Example 2-11 Resolver address space SETUP data set

; ****************************************
; [Link](RESOLV30)
; ****************************************
GLOBALTCPIPDATA('[Link](GLOBAL)') 1
DEFAULTTCPIPDATA('[Link](DEFAULT)') 2
GLOBALIPNODES('[Link](IPNODES)') 3
COMMONSEARCH 4
CACHE 5
CACHESIZE(20M) 6
MAXTTL(600) 7
UNRESPONSIVETHRESHOLD(25) 8

In this example, the numbers correspond to the following information:

1. Specifies the first choice of the [Link] file.
2. Specifies the last choice of the [Link] file.
3. Specifies the first choice of the local hosts file.
4. The COMMONSEARCH search order is used. This statement is needed to have
GLOBALIPNODES to be applied.
5. Indicates that system-wide caching is enabled for the resolver.
6. Specifies the maximum amount of storage that can be allocated by the resolver to manage
cached records. A value of at least 50M should be considered in a production
environment.
7. Specifies the maximum amount of time the resolver can use resource information
obtained from a Domain Name System (DNS) server as part of resource resolution.

44 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

8. Define the percentage threshold value to be used to calculate when a name server should
be declared to be unresponsive to resolver queries.
.

Important: Be careful when creating these global parameters. The definitions in the
resolver SETUP data set is applied to all TCP/IP stacks or applications.

Create the global [Link] file

In this step, we provide the global statements that all TCP/IP stacks and applications used in
our z/OS environment. To define these statements, we copied the sample [Link] file
provided in [Link](TCPDATA) and customized the statements, as shown in
Example 2-12.

Example 2-12 Global [Link] file

; *****************************************
; [Link](GLOBAL)
; *****************************************
DOMAINORIGIN [Link] 1
NSINTERADDR [Link] 2
NSPORTADDR 53
RESOLVEVIA UDP
RESOLVERTIMEOUT 10
RESOLVERUDPRETRIES 1
LOOKUP DNS LOCAL 3

In this example, the numbers correspond to the following information:

1. Specifies the list of domain names appended to the host name when the search is
performed.
2. Specifies the IP address of the DNS server.
3. To take advantage of the caching that we enabled in the Global Resolver SETUP file
(Example 2-11 on page 44), we changed our previously used LOOKUP sequence to favor
the DNS over the LOCAL file. If neither a cache entry or a DNS entry is found for a lookup,
then the resolver searches the local file.

Important: If GLOBALTCPIPDATA is specified:

 Any statements contained in the global [Link] file will take precedence over any
statements in local [Link] file found by way of the appropriate environment’s
(Native z/OS or z/OS UNIX) search order.
 The [Link] statements in Example 2-12 (with the exception of LOOKUP) can only
be specified in GLOBALTCPIPDATA. If the resolver statements are found in any of the
other search locations for [Link], they are ignored. If the resolver statements are
not found in GLOBALTCPIPDATA, their default value will be used.

Chapter 2. The resolver 45

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

Create the default [Link] file

We created a default [Link] file, as shown in Example 2-13, to be the last choice of the
local [Link] search order. It is used when the application does not specify the local
[Link] explicitly.

Example 2-13 Default [Link] file

; *****************************************
; [Link](DEFAULT)
; *****************************************
TCPIPJOBNAME TCPIP 1
HOSTNAME WTSC30 2

In this example, the numbers correspond to the following information:

1. Specifies the default TCP/IP procedure name.
2. Specifies the default host name.

Important: Applications that use Language Environment services without a

TCPIPJOBNAME statement cause applications that issue __iptcpn() to receive a job
name of NULL, and some of these applications will use INET instead of TCP/IP. Although
this presents no problem when running in a single-stack environment, it can potentially
cause errors in a multi-stack environment.

Create the global IPNODES data set

We created the global IPNODES data set, which is referred as GLOBALIPNODES in the
resolver SETUP data set. It contains name-to-address mappings. This data set is used for the
local search to resolve a name into an IP address or vice versa.

We chose to use the COMMONSEARCH, because it allowed us to have a common local

search environment with IPv4 or IPv6 hosts. Example 2-14 shows the contents of the
GLOBALIPNODES data set. When using COMMONSEARCH, only the IPNODES data set is
used.

Example 2-14 GLOBALIPNODES data set

; *****************************************
; [Link](IPNODES)
; *****************************************
[Link] OURDNS 1
[Link] WTSC30A 1
[Link] WTSC31B 1
[Link] WTSC32C 1
[Link] router1 1
[Link] router2 1
1::2 TESTIPV6ADDRESS1 2
[Link] TESTIPV6ADDRESS2 2

In this example, the numbers correspond to the following information:

1. The mapping of a name and a IPv4 address is listed.
2. The mapping of a name and a IPv6 address is listed.

46 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

Create a [Link] file for TCPIPA stack

We created a local [Link] file for the TCPIPA stack with file name DATAA30, as shown
in Example 2-15.

Example 2-15 [Link] file DATAA30

; *****************************************
; [Link](DATAA30)
; *****************************************
TCPIPJOBNAME TCPIPA 1
HOSTNAME WTSC30A 2
DATASETPREFIX TCPIPA
MESSAGECASE MIXED

In this example, the numbers correspond to the following information:

1. Specifies the procedure name of TCPIPA stack.
2. Specifies the host name of the TCPIPA stack.

Create the TCPIPA stack started procedure

We created the TCPIPA stack procedure (RESOLVER_CONFIG) and pointed to
[Link](DATAA30), using the &sysclone variable to simplify our implementation
to allow for a single procedure to be used by any z/OS image in our sysplex environment, as
shown in Example 2-16.

Example 2-16 TCPIPA procedure

/*****************************************
/* [Link](TCPIPA)
/*****************************************
//TCPIPA PROC PARMS='CTRACE(CTIEZB00),IDS=00', 1
// PROFILE=PROFA&SYSCLONE.,TCPDATA=DATAA&SYSCLONE
//TCPIPA EXEC PGM=EZBTCPIP,REGION=0M,TIME=1440,
// PARM=('&PARMS',
// 'ENVAR("RESOLVER_CONFIG=//''[Link](&TCPDATA)''")') 2
//SYSPRINT DD SYSOUT=*,DCB=(RECFM=VB,LRECL=132,BLKSIZE=136)
//SYSTCPT DD SYSOUT=*,DCB=(RECFM=VB,LRECL=132,BLKSIZE=136)
//ALGPRINT DD SYSOUT=*,DCB=(RECFM=VB,LRECL=132,BLKSIZE=136)
//CFGPRINT DD SYSOUT=*,DCB=(RECFM=VB,LRECL=132,BLKSIZE=136)
//SYSOUT DD SYSOUT=*,DCB=(RECFM=VB,LRECL=132,BLKSIZE=136)
//CEEDUMP DD SYSOUT=*,DCB=(RECFM=VB,LRECL=132,BLKSIZE=136)
//SYSERROR DD SYSOUT=*
//PROFILE DD DISP=SHR,DSN=[Link](&PROFILE.) 3

In this example, the numbers correspond to the following information:

1. The TCP/IP procedure name is TCPIPA.
2. The local [Link] is specified.
3. The TCP/IP profile is specified (the TCP/IP configuration file example is not shown in this
chapter).

Chapter 2. The resolver 47

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

2.3.2 Activation and verification

To verify that the resolver address space was working as expected, we performed these
steps:
1. Stop the default z/OS UNIX resolver.
2. Start the resolver address space.
3. Display the resolver address space configuration.
4. Use the ping command to verify the name resolution.

To implement our resolver address space, we halt the running resolver using the STOP
command, as shown in Example 2-17.

Important: Stop and restart the resolver only if you install a new level of the resolver code.

Stop the default z/OS UNIX resolver

In our current environment, the default z/OS UNIX resolver was running. We stopped this
default resolver, as shown in Example 2-17, to run the customized resolver.

Example 2-17 Stopping the resolver address space

P RESOLV30
EZZ9292I RESOLVER ENDING
IEF352I ADDRESS SPACE UNAVAILABLE
$HASP395 RESOLV30 ENDED

Start the resolver address space

We started the customized resolver address space using the procedure we created in the
previous step, as shown in Example 2-18.

Example 2-18 Starting a configured resolver address space

S RESOLV30
IRR812I PROFILE ** (G) IN THE STARTED CLASS WAS USED 646
TO START RESOLV30 WITH JOBNAME RESOLV30.
$HASP100 RESOLV30 ON STCINRDR
IEF695I START RESOLV30 WITH JOBNAME RESOLV30 IS ASSIGNED TO USER
IBMUSER , GROUP SYS1
$HASP373 RESOLV30 STARTED
IEE252I MEMBER CTIRES00 FOUND IN [Link]
EZZ9298I DEFAULTTCPIPDATA - [Link](DEFAULT) 1
EZZ9298I GLOBALTCPIPDATA - [Link](GLOBAL) 2
EZZ9298I DEFAULTIPNODES - None
EZZ9298I GLOBALIPNODES - [Link](IPNODES) 3
EZZ9304I COMMONSEARCH
EZZ9304I CACHE 4
EZZ9298I CACHESIZE - 20M 5
EZZ9298I MAXTTL - 600 6
EZZ9298I UNRESPONSIVETHRESHOLD - 25 7
EZZ9291I RESOLVER INITIALIZATION COMPLETE

In this example, the numbers correspond to the following information:

1. The correct DEFAULTTCPIPDATA file is applied.
2. The correct GLOBALTCPIPDATA file is applied.

48 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

3. The correct GLOBALIPNODES file is applied.

4. Indicates that system-wide caching is enable for the resolver.
5. Indicates the maximum amount of storage that can be allocated by the resolver.
6. Indicates the maximum amount of time the resolver can use resource information obtained
from a Domain Name System (DNS) server as part of resource resolution.
7. Indicates the percentage threshold value to calculate when a name server should be
declared to be unresponsive to resolver queries.

Note: If you want to start the default z/OS UNIX resolver, use the following command
instead:
START [Link],PROG=EZBREINI,SUB=MSTR

Note: The resolver utilizes non-reusable address spaces. To start resolver using a
reusable address space ID (REUSASID), see 1.3.3, “Reusable address space ID” on
page 6.

If you want to reload the SETUP data set content changes, use the MODIFY command to
refresh the resolver. To show how this is done, we created a new SETUP data set named
NEWSETUP, with the same configuration as the RESOLV30 setup file and changed
UNRESPONSIVETHRESHOLD statement changed to 35%, and refreshed the resolver to
reflect the changes, as shown in Example 2-19.

Example 2-19 Modifying the resolver address space

F RESOLV30,REFRESH,SETUP=[Link](NEWSETUP)
EZZ9298I DEFAULTTCPIPDATA - [Link](DEFAULT)
EZZ9298I GLOBALTCPIPDATA - [Link](GLOBAL)
EZZ9298I DEFAULTIPNODES - None
EZZ9298I GLOBALIPNODES - [Link](IPNODES)
EZZ9304I COMMONSEARCH
EZZ9304I CACHE
EZZ9298I CACHESIZE - 30M
EZZ9298I MAXTTL - 600
EZZ9298I UNRESPONSIVETHRESHOLD - 35
EZZ9293I REFRESH COMMAND PROCESSED

Display the resolver address space configuration

To verify that the correct configuration file is applied to the resolver address space, use the
MODIFY command with the display option, as shown in Example 2-20.

Example 2-20 Modify resolver with display option

F RESOLV30,DISPLAY
EZZ9298I DEFAULTTCPIPDATA - [Link](DEFAULT)
EZZ9298I GLOBALTCPIPDATA - [Link](GLOBAL)
EZZ9298I DEFAULTIPNODES - None
EZZ9298I GLOBALIPNODES - [Link](IPNODES)
EZZ9304I COMMONSEARCH
EZZ9304I CACHE
EZZ9298I CACHESIZE - 30M
EZZ9298I MAXTTL - 600

Chapter 2. The resolver 49

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

EZZ9298I UNRESPONSIVETHRESHOLD - 35
EZZ9293I DISPLAY COMMAND PROCESSED

Use the ping command to verify the name resolution

Verify that the resolver is able to perform the expected name-to-address resolution by using
the ping command, as shown in Example 2-21. As you can see, the name router1 has
resolved to address [Link]. Refer to 8.3.1, “The ping command (TSO or z/OS UNIX)” on
page 303 for more details about issuing the ping command.

Example 2-21 UNIX ping command display

CS02 @ SC30:/u/cs02>ping router1
CS V1R12: Pinging host router1 ([Link])
Ping #1 response took 0.001 seconds.

The TSO PING command was also successful, as shown in Example 2-22.

Example 2-22 TSO PING command display

TSO PING ROUTER1
CS V1R12: Pinging host router1 ([Link])
Ping #1 response took 0.001 seconds.
***

It is also possible to verify where the resolver is looking by using the TRACE RESOLVER
parameter in the stack’s or application’s [Link] file. For an explanation of how this is
done and what the contents of this trace will be, refer to 2.4, “Problem determination” on
page 50.

2.4 Problem determination

To diagnose resolver problems, you can use two kinds of trace tools:
 Trace Resolver
This provides information that can be helpful in debugging problems that an application
program could have with using resolver facilities (for example, GetHostByName or
GetHostByAddr).
 Component Trace RESOLVER (SYSTCPRE)
This is useful for diagnosing resolver problems that cannot be isolated to one particular
application.

In this section we provide a brief explanation of when to debug, which trace has to be used,
and how to use these trace facilities. For more information about resolver diagnosis, refer to
z/OS Communications Server: IP Diagnosis Guide, GC31-8782.

50 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

Deciding which tool to use to diagnose a resolver problem

The first thing to do when diagnosing a possible resolver problem is to check the symptoms to
verify if it is indeed a resolver problem (see Table 2-3).

Table 2-3 What to do if the host name cannot be reached

When we ping a host name, What is the problem? Solution
the ping command:

Succeeds, but another The problem is with the resolver Use the Trace Resolver
application fails when configuration for the application in statement on the local
resolving the same host the users environment. [Link] used by the
name. application that has the
problem.

Fails, but the host name is The resolution is successful but This problem is related to
converted to an IP address. the host is not reachable or active. connectivity, not a resolver
problem.

Fails to convert the name to The problem might be with the Use Trace Resolver to solve the
an IP address. resolver configuration, searching problem.
local host files, or using DNS.

Tip: If the problem seems to be related to the DNS, use the LOOKUP LOCAL DNS
statement to check the local files first.

Trace Resolver
The Trace Resolver informs us what the resolver looked for (the questions) and where it
looked (name server’s cache and IP addresses or local host file names).

The following situations can be checked in the trace output:

 Check whether the correct resolver data sets is in use. If an unexpected [Link] file is
used, check the search orders of the data set.
 Check whether the data sets defined to be used are authorized by RACF and can be read
by the application, TCP/IP stack, or user.
 Check the [Link] parameter values, especially Search, NameServer,
NSINTERADDR, and NsPortAddr.
 Check the questions posed by the resolver to DNS or in searching the local host files. Are
these the queries you expected?
 Look for errors or failures in the trace.
 Was the information obtained from the resolver cache? If so, use netstat Rescache/-Q
commands to determine if the cache information is correct.
 Did DNS respond (if you expected it to)? If not, see whether DNS is active at the IP
address you specified for NameServer and NSINTERADDR and what port it is listening
on. Also, DNS logs can be helpful, so ask the DNS administrator for help.

Trace Resolver can be activated in the following ways, in its precedence order:
1. The RESOLVER_TRACE environment variable (z/OS UNIX environment only).
2. SYSTCPT DD allocation.
3. TRACE RESOLVER or OPTIONS DEBUG statements (you must allocate STDOUT or
SYSPRINT to generate trace data).

Chapter 2. The resolver 51

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

4. The resDebug bit set to on in the _res structure option field (you must allocate STDOUT or
SYSPRINT to generate trace data).

Next, we illustrate using Trace Resolver in a z/OS UNIX environment, and in a TSO
environment.

Using Trace Resolver in z/OS UNIX environment

Example 2-23 shows how to enable and disable the Trace Resolver in z/OS UNIX
environment.

Example 2-23 Using Trace Resolver in a z/OS UNIX environment

CS02 @ SC30:/u/cs02>export RESOLVER_TRACE=/u/cs02/[Link] 1
CS02 @ SC30:/u/cs02>ping admin 2
CS V1R12: Pinging host [Link] ([Link])
Ping #1 response took 0.000 seconds.
CS02 @ SC30:/u/cs02>set -A RESOLVER_TRACE 3
CS01 @ SC30:/u/cs01>obrowse /u/cs02/[Link]

In this example, the numbers correspond to the following information:

1. To enable the Trace Resolver, set the RESOLVER_TRACE environment variable. This
command directs the output to the /u/cs06/[Link] HFS file. You can also direct the
output to STDOUT by specifying RESOLVER_TRACE=STDOUT. If you want to direct it to a new
MVS data set, specify the following command:
RESOLVER_TRACE="//’[Link]’"
2. After enabling a Trace Resolver, perform a z/OS UNIX shell command that invokes a
resolver call.
3. This command disables the Trace Resolver.

Using Trace Resolver in a TSO environment with SYSTCPT DD

Example 2-24 shows how to enable and disable the Trace Resolver in a TSO environment
environment.

Example 2-24 Using Trace Resolver in a TSO environment

alloc dd(systcpt) da(*) 1
ping router1 2
free dd(systcpt) 3

In this example, the numbers correspond to the following information:

1. To enable the Trace Resolver, allocate a SYSTCPT data set. If you specify da(*), the
Trace Resolver output to a TSO terminal. If you want to direct the output to a specific data
set, specify da(‘[Link]’).
2. After enabling the Trace Resolver, perform a TSO command that invokes a resolver call.
3. To disable theTrace Resolver, free a SYSTCPT data set.

Tip: When directing Trace Resolver output to a TSO terminal, define the screen size to
be only 80 columns wide. Otherwise, trace output is difficult to read.

52 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

Using Trace Resolver for applications with [Link] statements

Allocate STDOUT or SYSPRINT (as a DD statement in the procedure) as an output data set,
and define the statement TRACE RESOLVER or OPTIONS DEBUG in the first line of the
[Link] file that is being used by the application, as shown in Example 2-25. Start the
application that invokes a resolver call.

Example 2-25 Using the OPTIONS DEBUG to get a trace of the resolver
OPTIONS DEBUG 1
TCPIPJOBNAME TCPIPA
HOSTNAME WTSC30A
DOMAINORIGIN [Link]
DATASETPREFIX TCPIPA
MESSAGECASE MIXED
NSINTERADDR [Link]
NSPORTADDR 53

In this example, the numbers correspond to the following information:

1. Specify OPTIONS DEBUG or TRACE RESOLVER to enable Trace Resolver.

Displaying the output of the Trace Resolver

Example 2-26 shows the output of the Trace Resolver in the z/OS UNIX environment (which
was taken from Example 2-23 on page 52). Note that the Trace Resolver taken in the TSO
environment (Example 2-24 on page 52) is almost identical.

Example 2-26 Trace Resolver partial output: z/OS UNIX shell environment
Resolver Trace Initialization Complete -> 2010/09/27 [Link].709930
res_init Resolver values:
Global Tcp/Ip Dataset = [Link](GLOBAL) 1
Default Tcp/Ip Dataset = [Link](DEFAULT)
Local Tcp/Ip Dataset = /etc/[Link] 2
...
...
(G) LookUp = LOCAL DNS 3
(*) Cache
res_init Succeeded
res_init Started: 2010/09/27 [Link].741620
res_init Ended: 2010/09/27 [Link].741624
***************************************************************************
GetAddrInfo Started: 2010/09/27 [Link].741646
GetAddrinfo Invoked with following inputs:
Host Name: admin 4
...
...
GetAddrInfo Only IPv4 Interfaces Exist
GetAddrInfo Searching Local Tables for IPv4 Address
Global IpNodes Dataset = [Link](IPNODES) 5
Default IpNodes Dataset = None
Search order = CommonSearch
...
...
- Lookup for [Link]
- Lookup for admin
res_search(admin, C_IN, T_A)
res_search Host Alias Search found no alias 6
res_querydomain(admin, [Link], C_IN, T_A)
res_querydomain resolving name: [Link]

Chapter 2. The resolver 53

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

res_query([Link], C_IN, T_A)

Querying resolver cache for [Link]
...
No cache information was available 7
res_mkquery(QUERY, [Link], C_IN, T_A)
res_mkquery created message:
...
res_send Name Server Capabilities
Name server [Link]
...
res_send Sending query to Name Server [Link] 8
DNS Communication Started: 2010/09/27 [Link].752519
No OPT RR record sent on request to [Link]
...
BPX1AIO RECVMSG : From [Link]
UDP Data Length: 86
res_send received data via UDP. Message received:
* * * * * Beginning of Message * * * * *
Query Id: 62855
...
Response Code: NOERROR
Number of Question RRs: 1
Question 1:
[Link]
...
Answer 1:
[Link]
Type (0X0001) T_A Class (0X0001) C_IN
TTL: 86400 (1 days, 0 hours, 0 minutes, 0 seconds)
[Link]
* * * * * End of Message * * * * *
DNS Communication Ended: 2010/09/27 [Link].753095 time used [Link].000576
Name Server Capability Updates
Name server [Link]
Queries sent = 1
Failures = 0
res_send Succeeded
Attempting to cache results for [Link]
EZBRECAR: RetVal = 0, RC = 0, Reason = 0x00000000
Cache information was saved 9
...
GetAddrInfo Succeeded: IP Address(es) found:
IP Address(1) is [Link] 10
GetAddrInfo Ended: 2010/09/27 [Link].753194
***************************************************************************
FreeAddrInfo Started: 2010/09/27 [Link].753222
FreeAddrInfo Called to free addrinfo structures
FreeAddrInfo Succeeded, Freed 1 Addrinfos
FreeAddrInfo Ended: 2010/09/27 [Link].753229
***************************************************************************

In this example, the numbers correspond to the following information:

1. Informs you that the global [Link] in use.
2. Informs you that the local [Link] in use.
3. The local hosts file is looked up first, followed by the DNS server if it fails.
4. The admin host name is looked up.
5. Informs you that the global [Link] in use.

54 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

6. No information was available in [Link].

7. The admin host entry could not be found in the cache.
8. The resolver send a query to name server.
9. The response of name server is cached.
[Link] IP Address was found in the name server.

CTRACE: RESOLVER (SYSTCPRE)

Component Trace (CTRACE) is used for the RESOLVER component (SYSTCPRE) to collect
debug information. The TRACE RESOLVER traces information about a per-application basis
and directs the output to a unique file for each application. The CTRACE shows resolver
actions for all applications (although it might be filtered).

The CTRACE support allows for JOBNAME, ASID filtering, or both. The trace buffer is located
in the resolver private storage. The trace buffer minimum size is 128 KB. The maximum size is
128 MB. The default size is 16 MB. Trace records can optionally be written to an external
writer.

The resolver CTRACE can be started any time needed by using the TRACE CT command, or
it can be activated during resolver procedure initialization.

Note: If you suspect that there is an error in the operation of the resolver cache, you must
collect CTRACE records, as there are no Trace Resolver trace entries for cache
processing.

Using CTRACE for RESOLVER

The resolver CTRACE initialization PARMLIB member can be specified at resolver start time.
To activate the resolver CTRACE during resolver initialization, follow these steps:
1. Create a CTWTR procedure in your [Link], as shown in Example 2-27.

Example 2-27 CTWTR procedure

//CTWTR PROC
//IEFPROC EXEC PGM=ITTTRCWR
//TRCOUT01 DD DSNAME=CS02.CTRACE1,VOL=SER=COMST2,UNIT=3390,
// SPACE=(CYL,10),DISP=(NEW,KEEP),DSORG=PS
//TRCOUT02 DD DSNAME=CS02.CTRACE2,VOL=SER=COMST2,UNIT=3390,
// SPACE=(CYL,10),DISP=(NEW,KEEP),DSORG=PS
//*

2. Using the sample resolver procedure shipped with the product, enter the following console
command:
S RESOLV30,PARMS='CTRACE(CTIRESxx)'
Where xx is the suffix of the CTIRESxx PARMLIB member to be used. To customize the
parameters used to initialize the trace, you can update the [Link] member
CTIRES00, as shown in Example 2-28.

Example 2-28 Trace options

/*********************************************************************/
TRACEOPTS
/* ---------------------------------------------------------------- */
/* Optionally start external writer in this file (use both */
/* WTRSTART and WTR with same wtr_procedure) */

Chapter 2. The resolver 55

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

WTRSTART(CTWTR)
/* ---------------------------------------------------------------- */
/* ON OR OFF: PICK 1 */
/* ---------------------------------------------------------------- */
ON
/* OFF */
/* BUFSIZE: A VALUE IN RANGE 128K TO 128M */
BUFSIZE(16M)
/* JOBNAME(jobname1,...) */
/* ASID(Asid1,...) */
WTR(CTWTR)
/* ---------------------------------------------------------------- */
/* OPTIONS: NAMES OF FUNCTIONS TO BE TRACED, OR "ALL" */
/* ---------------------------------------------------------------- */
/* OPTIONS( */
/* 'ALL ' */
/* ,'MINIMUM ' */
/* ) */

3. Use the TRACE CT command to define the options, as shown in Example 2-29.

Example 2-29 TRACE CT command flow

TRACE CT,ON,COMP=SYSTCPRE,SUB=(RESOLV30)
*189 ITT006A SPECIFY OPERAND(S) FOR TRACE CT COMMAND.
R 189,OPTIONS=(ALL),END
IEE600I REPLY TO 189 IS;OPTIONS=(ALL),END
ITT038I ALL OF THE TRANSACTIONS REQUESTED VIA THE TRACE CT COMMAND
WERE SUCCESSFULLY EXECUTED.
IEE839I ST=(ON,0256K,00512K) AS=ON BR=OFF EX=ON MT=(ON,024K) 497
ISSUE DISPLAY TRACE CMD FOR SYSTEM AND COMPONENT TRACE STATUS
ISSUE DISPLAY TRACE,TT CMD FOR TRANSACTION TRACE STATUS

4. Reproduce the problem.

5. Save the trace contents into the trace file created by the CTWTR procedure, executing the
the commands shown in Example 2-30.

Example 2-30 Saving the trace contents

TRACE CT,ON,COMP=SYSTCPRE,SUB=(RESOLV30)
*190 ITT006A SPECIFY OPERAND(S) FOR TRACE CT COMMAND.
R 190,WTR=DISCONNECT,END
IEE600I REPLY TO 190 IS;WTR=DISCONNECT,END
ITT038I ALL OF THE TRANSACTIONS REQUESTED VIA THE TRACE CT COMMAND
WERE SUCCESSFULLY EXECUTED.
IEE839I ST=(ON,0256K,00512K) AS=ON BR=OFF EX=ON MT=(ON,024K) 503
ISSUE DISPLAY TRACE CMD FOR SYSTEM AND COMPONENT TRACE STATUS
ISSUE DISPLAY TRACE,TT CMD FOR TRANSACTION TRACE STATUS

6. Stop the CTRACE by issuing the command shown in Example 2-31.

Example 2-31 Stopping CTRACE

TRACE CT,OFF,COMP=SYSTCPRE,SUB=(RESOLV30)
ITT038I ALL OF THE TRANSACTIONS REQUESTED VIA THE TRACE CT COMMAND
WERE SUCCESSFULLY EXECUTED.
IEE839I ST=(ON,0256K,00512K) AS=ON BR=OFF EX=ON MT=(ON,024K) 506

56 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:59 am 7896 [Link]

ISSUE DISPLAY TRACE CMD FOR SYSTEM AND COMPONENT TRACE STATUS
ISSUE DISPLAY TRACE,TT CMD FOR TRANSACTION TRACE STATUS

After these steps, we will have a trace file to be formatted using the IPCS command:
CTRACE COMP(SYSTCPRE) TALLY

Displaying the CTRACE result

The resulting display will show the resolver process entries, as shown in Example 2-32.

Example 2-32 Resolver formatted trace entries

COMPONENT TRACE TALLY REPORT
SYSNAME(SC30)
COMP(SYSTCPRE)
TRACE ENTRY COUNTS AND AVERAGE INTERVALS (IN MICROSECONDS)

FMTID COUNT Interval MNEMONIC DESCRIBE

-------- ----------- ------------ -------- --------------------------------
00000001 0 CTRACE CTrace Initialized
00000002 0 CTRACE Status changed or displayed
00000003 0 CTRACE CTrace Terminated
00000004 0 CTRACE !CTrace has abended
00000005 0 CTRACE CTrace Stopped - Buffers Retain
00010001 0 API GetHostByAddr Entry Parameters
00010002 0 API GetHostByAddr Stack Affinity
00010003 0 API GetHostByAddr Failure
00010004 0 API GetHostByAddr Success
00010005 0 API GetHostByAddr GetLocalHostName
00010006 0 API GetHostByName Entry Parameters
00010007 0 API GetHostByName Stack Affinity
00010008 0 API GetHostByName Failure
00010009 0 API GetHostByName Success

2.5 Additional information

For more specific information regarding the resolver address space, refer to z/OS
Communications Server: IP Configuration Guide, SC31-8775 and z/OS Communications
Server: IP Configuration Reference, SC31-8776.

For more information about resolver diagnosis, refer to z/OS Communications Server: IP
Diagnosis Guide, GC31-8782.

Chapter 2. The resolver 57

7896 [Link] Draft Document for Review March 21, 2011 7:59 am

58 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

Chapter 3. Base functions

The term base functions in this case implies the minimum configuration required for the
proper operation of a z/OS TCP/IP environment. The base functions that we describe in this
chapter are considered necessary for any useful deployment of the TCP/IP stack and
commonly used applications.

This chapter discusses the following topics.

SectioncSectionSection Topic

3.1, “The base functions” on page 60 Basic concepts of base functions

3.2, “Common design scenarios for base Key characteristics of base functions and why they might
functions” on page 60 be important in your environment

3.3, “z/OS UNIX System Services setup Selected implementation scenarios, tasks, configuration
for TCP/IP” on page 65 examples, and problem determination suggestions

3.4, “Configuring z/OS TCP/IP” on Configuration details for the z/OS TCP/IP environment
page 79

3.5, “Implementing the TCP/IP stack” on Implementation tasks for the TCP/IP stack
page 88

3.6, “Activating the TCP/IP stack” on Messages used to verify the accuracy of the current
page 94 environment customization data sets used in z/OS UNIX
and TCP/IP initialization

3.7, “Reconfiguring the system with z/OS z/OS commands used to reconfigure the system
commands” on page 110

3.8, “Job log versus syslog as diagnosis Information about using job log versus syslog when
tool” on page 115 diagnosing issues

3.9, “Message types: Where to find Listing of message types

them” on page 115

© Copyright IBM Corp. 2011. All rights reserved. 59

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

3.1 The base functions

Base functions are those functions considered to be standard in TCP/IP environments
regardless of the implementation. Base functions establish a functional working environment
that can be exploited by other features, or upon which many other functions can be
implemented or validated. When the base functions are implemented, they exercise the most
commonly used features of a TCP/IP environment, providing an effective way to perform
integrity tests and validate the TCP/IP environment before embarking on the more complex
features, configurations, and implementations of the stack.

Most of these functions are implemented at the lower layers. There are some base functions
that are implemented at the application layer (such as Telnet and FTP). The details of the
standard applications can be found in Communications Server for z/OS V1R12 TCP/IP
Implementation Volume 3: Standard Applications, SG24-7897. Here, we discuss the
configuration that provides the infrastructure of the TCP/IP protocol suite in the z/OS
Communications Server environment.

3.1.1 Basic concepts

The z/OS TCP/IP stack (a TCP/IP instance) is a full functional implementation of the standard
RFC protocols that are fully integrated and tightly coupled between z/OS and UNIX System
Services. It provides the environment that supports the base functions, as well as the many
traditional TCP/IP applications. The two environments that need to be created and
customized to support the z/OS Communications Server for TCP/IP are:
 A native z/OS environment in which users can exploit the TCP/IP protocols in a standard
z/OS application environment such as batch jobs (with JES interface), started tasks, TSO,
CICS, and IMS applications.
 A z/OS UNIX System Services environment that lets you develop and use applications
and services that conform to the POSIX or XPG4 standards (UNIX specifications). The
z/OS UNIX environment also provides some of the base functions to support the z/OS
environment and vice versa.

Because the z/OS Communications Server exploits z/OS UNIX services even for traditional
z/OS environments and applications, a full-function mode z/OS UNIX environment, including
a Data Facility Storage Management Subsystem (DFSMS), a z/OS UNIX file system, and a
security product (such as Resource Access Control Facility, or RACF), are required before the
z/OS Communications Server can be started successfully and the TCP/IP environment
initialized.

3.2 Common design scenarios for base functions

Because base functions are primarily setting up the primitives in the TCP/IP environment, we
deal with very basic scenarios, which can be built upon at a later time. For the base functions
we consider two scenarios:
 Single stack environment
 Multiple stack environment

Important: Although there are specialized cases where multiple stacks per LPAR can
provide value, in general we recommend implementing only one TCP/IP stack per LPAR.

60 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

3.2.1 Single stack environment

A single stack environment refers to the existence of one TCP/IP system address space in a
single z/OS image (LPAR) providing support for the functions and features of the TCP/IP
protocol suite.

Dependencies
In order to achieve a successful implementation of the z/OS Communications Server - TCP/IP
component, we identified certain dependencies, as explained here:
 Implement a full-function UNIX System Services system on z/OS. Detailed information
about this topic is available in z/OS UNIX System Services Planning, GA22-7800, and in
and z/OS MVS Initialization and Tuning Reference, SA22-7592. Also refer to z/OS
Program Directory, GI10-0670, which is available at the following address:
[Link]
 Define a RACF environment for the z/OS Communications Server - TCP/IP component.
This includes defining RACF groups to z/OS UNIX groups to manage resources, profiles,
user groups, and user IDs.
An OMVS UID must be defined with UID (0) and assigned to the started task name of the
CS for z/OS IP system address space. Detailed information is available in
Communications Server for z/OS TCP/IP Implementation Volume 4: Security and
Policy-Based Networking, SG24-7899, z/OS Security Server RACF Security
Administrator's Guide, SA22-7683, z/OS Security Server RACF System Programmer's
Guide, SA22-7681, and z/OS Security Server RACF Command Language Reference,
SA22-7687.
 Customize [Link] members with special reference to BPXPRMxx to use the
integrated sockets INET with the AF_INET and AF_INET6 physical file system. Detailed
information is available in z/OS MVS Initialization and Tuning Reference, SA22-7592,
z/OS UNIX System Services Planning, GA22-7800, and z/OS V1R7.0 Program Directory
GI10-0670.
 Customize the TCP/IP configuration data sets:
– [Link]
– [Link]
– Other configuration data sets
 Fully functional VTAM is required to support the interfaces used by TCP/IP.

Advantages
The advantages of a single stack are:
 Fewer CPU cycles are spent processing TCP/IP traffic, because there is only one logical
instance of each physical interface in a single stack environment versus a multiple stack
environment.
 Servers use fewer CPU cycles when certain periodic updates arrive (OMPROUTE
processing routing updates). Multiple stacks mean multiple copies of OMPROUTE.
 Each stack requires a certain amount of storage, the most significant being virtual storage.
 Multiple TCP/IP stacks add a significant level of complexity to TCP/IP system
administration tasks.

Chapter 3. Base functions 61

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

Considerations for a single stack environment

When creating a TCP/IP stack, you need to consider the other requirements upon which the
successful initialization of the stack depends. Very often the initial problems encountered are
related to the omission of tasks that were not performed by other disciplines such as RACF
administration.

CS for z/OS TCP/IP exploits the tightly coupled design of the z/OS Communications Server,
the integration of z/OS and UNIX System Services, and the provision of RACF services.
Coordination is the key to a successful implementation the TCP/IP stack.

3.2.2 Multiple stack environment

A multiple stack environment consists of more than one stack running concurrently in a single
LPAR. These stacks exist independent of each other, with the ability to be uniquely
configured. Each stack can support different features and provide different functions. Each
stack is configured in its own address space, and can communicate with the other stacks in
the LPAR if so desired.

Dependencies
The dependencies for the multiple stack environment are exactly the same as for the single
stack environment, as well as:
 Additional storage, especially virtual storage
 Additional CPU cycles for processing subsequent interfaces and services performing
periodic functions, such as OMPROUTE routing updates

Advantages
There are advantages for running a multiple stack environment, because it provides you with
the flexibility to partition your networking environment. Here are advantages to consider:
 You might want to establish separate stacks to separate workloads based on availability
and security. For example, you might have different requirements for a production stack, a
system test stack, and a secure stack.
This approach could, for example, be used to establish a test TCP/IP stack, where new
socket applications are tested before they are moved into the production system. You
might also want to apply maintenance to a non-production stack so it can be tested before
you apply it to the production stack.
 Your strategy might be to separate workload onto multiple stacks based on the functional
characteristics of applications, as with UNIX (OpenEdition) applications and non-UNIX
(z/OS) applications.
 You might be running z/OS servers and UNIX (OpenEdition) servers on the same
well-known port (TN3270 and otelnet on port 23). An alternative to this is approach is the
BIND for INADDR_ANY function.

Whatever the reason, the ability to configure multiple stacks and have them fully functional,
independently and concurrently, can be exploited in many different ways.

Considerations for a multiple stack environment

The considerations for a multiple stack environment are primarily the same as they are for a
single stack environment. We therefore indicate here only the differences and the additional
considerations regarding the multiple stack environment.

62 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

Sharing resolver between multiple stacks

The general recommendation is that you use separate DATASETPREFIX values per stack
and create separate copies of configuration data sets or at the very least resolver data sets.
Refer to “The resolver in a multiple stack environment” on page 40, for further details.

Selecting the correct configuration data sets

The resolver needs access to all resolver data sets if there are multiple stacks in multiple
z/OS LPARs. Refer to Chapter 2, “The resolver” on page 19, for further details.

TSO clients
TSO client functions can be directed against any number of TCP/IP stacks. Keep in mind,
though, that the client must be able to find the [Link] data set appropriate for the stack
of interest. You can modify your TSO logon procedure with a SYSTCPD DD statement, or use
a common TSO logon procedure without the SYSTCPD DD statement and allocate the
[Link] data set to the appropriate stack of interest.

Stack affinity
Any server or client needs to reference the appropriate stack if the desired stack is not the
default stack defined in the BPXPRMxx member of [Link]. Servers can use the
BPXK_SETIBMOPT_TRANSPORT environment variable to override the choice of the default
stack. There might also be applications that have affinity to the wrong stack and do not have
the option of establishing stack affinity. In those instances, you can execute BPXTCAFF prior
to the application execution step. For example:
//AFFINITY EXEC PGM=BPXTCAFF,PARM='TCPIPA'

This assumes TCPIPA is not the default stack.

Port management
When there is a single stack and the relationship of server to stack is 1:1, port management is
relatively simple. Using the PORT statement, the port number can be reserved for the server
in the [Link] for that given stack.

Port management becomes more complex, however, in an environment where there are
multiple stacks and a potential for multiple combinations of the same server (for example,
UNIX System Services TELNET and TN3270 TELNET). With use of VIPA, it is possible to use
the same "well-known" port number, in this case 23, for both services. The distinction would
be made by different names mapping to different IP addresses (VIPAs).Therefore, in a
multiple stack environment, you need to answer some questions based on the following
concepts:
 Generic server
A generic server is a server without affinity for a specific stack, and it provides service to
any client in the network. FTP is an example, because the stack is merely a connection
linking client and server. The service File Transfer is not related to the internal functioning
of the stack, and the server can communicate concurrently over any number of stacks.
 Servers with an affinity for a specific stack
There must be an explicit binding of the server application to the chosen stack when the
service (for example, UNIX System Services DNS, OSNMP, and ONETSTAT) is related to
the internal functioning of the stack.
This bind is made using the setibmopt() socket call (to specify the chosen stack) or using
the C function _iptcpn(), which allows applications to search in the [Link] file to
find the name of a specific stack.

Chapter 3. Base functions 63

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

 Ephemeral ports
In addition to synchronizing PORT reservations for specific applications across all stacks,
you have to synchronize reservations for port numbers that will be dynamically assigned
across all stacks when running with multiple stacks.
Those ports are called ephemeral ports, which are all above 1024, and are assigned by
the stack when none is specified on the application bind(). Use the PORTRANGE
statement in the [Link] to reserve a group of ports, and specify the same port
range for every stack. You also need to let CINET know which ports are guaranteed to be
available on every stack, using the BPXPRMxx parmlib member through
INADDRANYPORT and INADDRANYCOUNT statements.

CPU resources
Provisions need to be made for additional CPU cycles and storage (especially virtual
storage). These increases in resources are just for the existence of the additional stacks
running concurrently.

3.2.3 Recommendation
In general, we recommend implementing only one TCP/IP stack per LPAR, for the following
reasons:
 A TCP/IP stack is capable of exploiting all available resources defined to the LPAR in
which it is running. Therefore, starting multiple stacks will not yield any increase in
throughput.
 When running multiple TCP/IP stacks additional system resources, such as memory, CPU
cycles, and storage, are required.
 Multiple TCP/IP stacks add a significant level of complexity to TCP/IP system
administration tasks.
 It is not necessary to start multiple stacks to support multiple instances of an application
on a given port number, such as a test HTTP server on port 80 and a production HTTP
server also on port 80. This type of support can instead be implemented using
BIND-specific support where the two HTTP server instances are each associated to port
80 with their own IP address, using the BIND option on the PORT reservation statement.

3.2.4 Recommendations for MTU

The maximum transmission unit (MTU) is the largest packet size that can be sent using this
route. If the packet is larger than this size, the packet will have to be fragmented if
fragmentation is permitted. If fragmentation is not permitted, the packet is dropped and an
ICMP error is returned to the originator of the packet. If a route is inactive, the configured
MTU value that was defined using the MTU parameter in the ROUTE statement (or the
default MTU value for the specified interface type) is displayed. If a route is active, then the
actual MTU value is displayed.

For more information about MTU sizes for OSA-Express and HiperSockets, refer to
Communications Server for z/OS V1R12 TCP/IP Implementation Volume 3: High Availability,
SG24-7898.

64 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

3.3 z/OS UNIX System Services setup for TCP/IP

There are several areas that require your attention and action in order to implement a TCP/IP
stack successfully. In Chapter 1, “Introduction to Communications Server for z/OS IP” on
page 1, we review the UNIX concepts in the z/OS environment. We make specific references
to the BPXPRMxx member in [Link]. However, it is important to first understand the
security considerations for the UNIX environment.

3.3.1 RACF actions for UNIX

Security is an important consideration for most z/OS installations, and there are a few
features we need to mention here for the base functions of any TCP/IP environment. TCP/IP
has some built-in internal security mechanisms, and it relies on the services of a security
manager, such as the IBM Resource Access Control Facility (RACF).

A security manager is a requirement in the Communications Server for z/OS IP environment.

As an online application, it is important that TCP/IP undergo security checks to eliminate
possible security exposures. Some basic security concepts are included in the following
sections, but for a more detailed explanation refer to Communications Server for z/OS V1R12
TCP/IP Implementation Volume 4: Security and Policy-Based Networking, SG24-7899.

RACF environment
RACF is very flexible and can be set up and tailored to meet almost all security requirements
of large enterprises. All RACF implementations are based on the following key elements:
 User IDs
 Groups
 RACF resources
 RACF profiles
 RACF facility classes
 The hierarchical owner principle, which is applicable for all RACF definitions of user IDs,
groups, and RACF resources

RACF implementation
Each unit of work in the z/OS system that requires UNIX System Services must be
associated with a valid UNIX System Services identity. A valid identity refers to the presence
of a valid UNIX user ID (UID) and a valid UNIX group ID (GID) for each such user. The UID
and the GID are defined through the OMVS segment in the user’s RACF user profile and in
the group’s RACF group profile.

Each functional RACF access group must be authorized to access a specific TCP/IP RACF
resource with a specific access attribute. The details of this process are discussed in
Communications Server for z/OS V1R12 TCP/IP Implementation Volume 4: Security and
Policy-Based Networking, SG24-7899.

Assigning user IDs to started tasks

In some cases, the user ID and started task must be associated with the UNIX superuser. In
other cases, you can associate the user ID and started task with the default user.

Chapter 3. Base functions 65

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

RACF offers you two techniques to assign user IDs and group IDs to started tasks:
 The started procedure name table (ICHRIN03)
 The RACF STARTED resource profiles
By using the STARTED resources, you can add new started tasks to RACF, and
immediately make those new definitions active.
IEF695I START T03DNS WITH JOBNAME T03DNS IS ASSIGNED TO USER TCPIP3, GROUP
OMVSGRP

The user ID and default group must be defined in RACF, which then treats the user ID as any
other RACF user ID for its resource access checking. RACF allows multiple started procedure
names to be assigned to the same RACF user ID. We used this method to assign RACF user
IDs to all TCP/IP started tasks.

Started task user IDs

The UNIX System Services tasks OMVS and BPXOINIT need to execute in an z/OS system
space and have the special user ID OMVSKERN assigned to them. OMVSKERN has to be
defined as superuser with UID 0, program /bin/sh, and home directory.

TCP/IP tasks need RACF user IDs with the OMVS segment defined. The user ID associated
with the main TCP/IP address space must be defined as a superuser; the requirements for
the individual servers vary, but most need to be a superuser as well.

z/OS VARY TCPIP commands

Access to VARY TCPIP commands can be controlled by RACF. This places restrictions on
this command, which can be used to alter and disrupt the TCP/IP environment.

NETSTAT command
Access to the TSO NETSTAT command, the UNIX shell command onetstat, and command
options can be controlled by RACF, by defining NETSTAT resources to the RACF generic
class SERVAUTH. This command might also need to be restricted, because it can be used to
alter or drop connections or to stop the TN3270 server.

Establish RACF security environment

The notes that follow are merely an overview of the steps in the process. Consult the
instructions in z/OS Security Server RACF Callable Services, SA22-7691, to accomplish
these tasks.
1. Defining commands for CS for z/OS IP in the RACF OPERCMDS class.
2. Establishing a group ID for a default OMVS group segment:
ADDGROUP OEDFLTG OMVS(GID(9999))
3. Defining a user ID for a default OMVS group segment:
RDEFINE FACILITY [Link] APPLDATA('OEDFLTU/OEDFLTG')
ADDUSER OEDFLTU DFLTGRP(OEDFLTG) NAME('OE DEFAULT USER') PASSWORD(xg18ej)
OMVS(UID(999999) HOME('/') PROGRAM('/bin/sh'))
4. Activating or refreshing appropriate facility classes:
SETROPTS CLASSACT(FACILITY)
SETROPTS RACLIST(FACILITY)
SETROPTS RACLIST(FACILITY) REFRESH

66 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

5. Defining one or more superuser IDs to be associated with certain UNIX System Services
users and TCP/IP started tasks:
ADDGROUP OMVSGRP OMVS(GID(1))
ADDUSER TCPIP3 DFLTGRP(OMVSGRP) OMVS(UID(0) HOME('/') PROGRAM('/bin/sh'))
6. Defining other UNIX System Services users.
You might already have defined RACF groups and users. If this is the case, you can set up
a z/OS UNIX file system home directory for each user and add an OMVS identity by
altering the group to include a GID (ALTGROUP). Then, using the ISHELL utility, add OE
segments for UNIX System Services users (associating them with the altered group and
giving each user a distinct UID).
Otherwise, you will have to perform these tasks in a more painstaking manner, for
example:
ADDGROUP usergrp OMVS(GID(10))
ADDUSER user01 DFLTGRP(usrgrp) OMVS(UID(20) HOME('/u/user01')
PROGRAM('/bin/sh/'))

More information about RACF with z/OS Communications Server TCP/IP

RACF can be used to protect many TCP/IP resources, such as the TCP/IP stack itself and
ports. Further information about securing your TCP/IP implementation can be found in
Communications Server for z/OS V1R12 TCP/IP Implementation Volume 4: Security and
Policy-Based Networking, SG24-7899.

3.3.2 APF authorization

The TCP/IP system program libraries must be APF authorized. Authorized Program Facility
(APF) means that z/OS built-in security can be bypassed by programs that are executed from
such libraries. CS for z/OS IP data sets have to be protected with RACF. Special attention has
to be given to the APF authorized libraries defined in PROGxx.

We used the LNKAUTH=LNKLST specification in [Link] member IEASYSxx, which

means that all libraries in the LNKLST concatenation will be APF authorized. If these libraries
are accessed through STEPLIB or JOBLIB, they will not be APF authorized unless they have
been specifically defined in the IEAAPFxx or PROGxx member.

SEZALOAD is one library that must be made part of your LNKLST concatenation. Because of
the LNKAUTH=LNKLST specification, it will be APF authorized when it is accessed through
the LNKLST concatenation. The SEZALOAD library holds the TCP/IP system code used by
both servers and clients.

In addition to the LNKLST libraries, there are libraries that are not accessed through the
LNKLST concatenation, but have to be APF authorized. The SEZATCP library holds the
TCP/IP system code used by servers. This library is normally placed in the STEPLIB or
JOBLIB concatenation, which is part of the server JCL.

The following libraries might have to be APF authorized, depending on the choices that you
make during the installation of z/OS:
SEZALPA This library holds the TCP/IP modules that must be made part of your
system’s LPA. If you choose to add the library name to your LPALSTxx
member in [Link], you also have to make sure the library is
APF authorized. If you copy the load modules in the library to an
existing LPALSTxx data set, you do not need to authorize the
SEZALPA data set.

Chapter 3. Base functions 67

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

SEZADSIL This library holds the load modules used by the SNMP command
processor running in the NetView® address space. If you choose to
concatenate this library to STEPLIB in the NetView address space,
you might have to APF authorize it, if other libraries in the
concatenation are already APF authorized.

Every APF-authorized online application might have to be reviewed to ensure that it matches
the security standards of the installation. A program is a “well-behaved program” if:
 Logged-on users cannot access or modify system resources for which they are not
authorized.
 The program does not require any special credentials to be able to execute.
Or, in the case of RACF, the program does not need the RACF authorization attribute
OPERATIONS for execution.

Note: User IDs with the RACF attribute OPERATIONS have ALTER access to all data
sets in the system. The access authority to single data sets can be specifically lowered
or excluded.

3.3.3 Changes to [Link] members

As we noted, the z/OS environment consists of the traditional MVS and UNIX System
Services environment. Because the UNIX System Services environment is implemented
within a z/OS system space, there are definitions in the z/OS environment upon which the
UNIX System Services environment depends.

[Link] is the single most important data set in the z/OS environment. It contains
most of the parameters that define z/OS as well as many other subsystems. The
[Link] data set definition parameters are critical to the proper initialization and
functioning of UNIX System Services and, therefore, to the TCP/IP implementation. Some of
the members of interest include:
 IEASYS00
 BPXPRMxx
 Integrated Sockets PFS definitions

IEASYS00
Because the z/OS Communications Server exploits z/OS UNIX services even for traditional
MVS environments and applications, a full-function mode z/OS UNIX environment, including
a Data Facility Storage Management Subsystem (DFSMS) and z/OS File Systems (including
z/OS UNIX file system) is required before the z/OS Communications Server can be started
and the TCP/IP environment successfully established.

The IEASYS00 parmlib definitions we used that are relevant to TCP/IP are:
OMVS=7A,
SMS=00,

OMVS=7A specifies that BPXPRM7A is used to configure the z/OS UNIX environment at
system initialization time. SMS=00 specifies that IGDSMS00 is to be used for definitions of
the Data Facility Storage Management Subsystem at z/OS UNIX initialization time.

68 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

BPXPRMxx
All the parameters defined in BPXPRMxx should be reviewed and tailored to individual
installation specification and resource utilization. z/OS UNIX System Services Planning,
GA22-7800, and z/OS MVS Initialization and Tuning Guide, SA22-7591, explain the details
and significance of each parameter in the BPXPRMxx member.

z/OS UNIX System Services Planning, GA22-7800; z/OS UNIX System Services User's
Guide, SA22-7802; and z/OS Program Directory, GI10-0670, detail the structure, design,
installation, and implementation of the z/OS UNIX environment.

z/OS Program Directory, GI10-0670, is available at the following address:

[Link]

Concepts such as Logical and Physical File Systems (PFS) are design components of z/OS
UNIX and are not discussed here.

Integrated Sockets PFS definitions

We need to define the desired file systems to support the communication provided by the
stack. Example 3-1 illustrates how support for IPv4 and IPv6 (dual mode) is defined for a
single stack environment.

Specifying NETWORK definitions for both AF_NET and AF_INET6 provides dual support. If
IPv6 support is not desired, then you can omit the NETWORK DOMIAINNAME(AF_INET6)
statement and subsequent parameters.

Example 3-1 BPXPRMxx definitions for a single stack supporting dual mode
FILESYSTYPE TYPE(UDS)
ENTRYPOINT(BPXTUINT)
NETWORK DOMAINNAME(AF_UNIX)
DOMAINNUMBER(1)
MAXSOCKETS(10000)
TYPE(UDS)

/* IPv4 support
NETWORK DOMAINNAME(AF_INET) 1
DOMAINNUMBER(2)
MAXSOCKETS(25000)
TYPE(INET) 2
INADDRANYPORT(10000)
INADDRANYCOUNT(2000)

FILESYSTYPE TYPE(INET) 2
ENTRYPOINT(EZBPFINI) 3

/* IPv6 support
NETWORK DOMAINNAME(AF_INET6) 4
DOMAINNUMBER(19)
TYPE(INET)

Chapter 3. Base functions 69

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

INET specifies a single stack with TCP/IP (by default) as the stack name. In this example, the
numbers correspond to the following information:
1. AF_INET specifies IPv4 support for the physical file type for socket address used by this
stack (TCP/IP).
2. Specify TYPE(INET) for a single stack environment. If you specify INET, you cannot start
multiple TCP/IP stacks.
3. EZBPFINI identifies a TCP/IP stack (this is the only valid value).
4. AF_INET6 specifies IPv6 support for the physical file type for socket address used by this
stack (TCP/IP).

Example 3-2 shows BPXPRMxx definitions for a multiple stack environment.

Example 3-2 BPXPRMxx definitions for a multiple stack supporting dual mode
FILESYSTYPE TYPE(UDS) ENTRYPOINT(BPXTUINT)
NETWORK DOMAINNAME(AF_UNIX)
DOMAINNUMBER(1)
MAXSOCKETS(10000)
TYPE(UDS)
FILESYSTYPE TYPE(CINET)
ENTRYPOINT(BPXTCINT)
NETWORK DOMAINNAME(AF_INET) 1
DOMAINNUMBER(2)
MAXSOCKETS(10000)
TYPE(CINET) 2
INADDRANYPORT(10000)
INADDRANYCOUNT(2000)

NETWORK DOMAINNAME(AF_INET6) 3
DOMAINNUMBER(19)
MAXSOCKETS(10000)
TYPE(CINET)

SUBFILESYSTYPE NAME(TCPIPA) 4
TYPE(CINET) 2
ENTRYPOINT(EZBPFINI) 5
DEFAULT

SUBFILESYSTYPE NAME(TCPIPB) 4
TYPE(CINET) 2
ENTRYPOINT(EZBPFINI) 5
.....

In this example, the numbers correspond to the following information:

1. AF_INET specifies IPv4 support for the physical file type for socket address used by this
stack (TCP/IP).
2. Specify TYPE(CINET) for a single stack environment. If you specify INET, you cannot start
multiple TCP/IP stacks.
3. AF_INET6 specifies IPv6 support for the physical file type for socket address used by this
stack (TCP/IP).
4. Specify the name of TCP/IP stack you want to configure.
5. EZBPFINI identifies a TCP/IP stack (this is the only valid value).

70 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

Additional [Link] updates

The updates are:
1. LNKLSTxx
Add the following CS for z/OS IP link libraries to the z/OS system link list:
– [Link]
– hlq.SEZALNK2
2. LPALSTxx
Add the following CS for z/OS IP LPA modules to the LPA during IPL of z/OS:
– [Link]

Note: [Link] must be cataloged into the MVS master catalog.

[Link] and hlq.SEZALNK2 can be cataloged into the MVS master catalog.
You can omit them from the MVS master catalog if you identify them to include a
volume specification as in:
[Link](WTLTCP),
TCPIP.SEZALNK2(WTLTCP)

If the three data sets mentioned were renamed during the installation process, then
use these names instead.

3. PROGnn or IEAAPFxx
Add the following TCP/IP libraries for APF authorization:
– [Link]
– [Link]
– [Link]
– hlq.SEZALNK2
– [Link]
– [Link]
4. IEFSSNxx
TNF and VMCF may be required for some of the CS for z/OS IP facilities and components
you are using. If you need to configure TNF and VMCF, add the subsystem definitions for
the MVS address spaces of TNF and VMCF as follows:
– If you choose to use restartable VMCF and TNF, as is recommended:
• TNF
• VMCF
– If you will not be using restartable VMCF and TNF:
• TNF,MVPTSSI
• VMCF,MVPXSSI,nodename
Set the nodename to the MVS NJE node name of this MVS system. It is defined in the
JES2 parameter member of [Link]:
NJEDEF ....
OWNNODE=03,
....

N03 NAME=SC30,SNA,NETAUTH

Chapter 3. Base functions 71

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

Before you make this update, make sure that the [Link] definition has been
added to LNKLSTxx and the library itself has been APF authorized. z/OS initializes the
address spaces of the TNF and VMCF subsystems during IPL as part of the master
scheduler initialization.
5. SCHEDxx
You need to specify certain CS for z/OS IP modules as privileged modules in MVS. The
following entries are present in the IBM-supplied program properties table (PPT); however,
if your installation has a customized version of the PPT, ensure these entries are present:
– For CS for z/OS IP:
PPT PGMNAME(EZBTCPIP) KEY(6) NOCANCEL PRIV NOSWAP SYST LPREF SPREF
– If you use restartable VMCF and TNF:
PPT PGMNAME(MVPTNF) KEY(0) NOCANCEL NOSWAP PRIV SYST
PPT PGMNAME(MVPXVMCF) KEY(0) NOCANCEL NOSWAP PRIV SYST
– For NPF:
PPT PGMNAME(EZAPPFS) KEY(1) NOSWAP
PPT PGMNAME(EZAPPAAA) NOSWAP
– For SNALINK:
PPT PGMNAME(SNALINK) KEY(6) NOSWAP SYST
6. COMMNDxx
VMCF and TNF may be required for some of the CS for z/OS IP facilities and components
you are using. If you use restartable VMCF and TNF, procedure EZAZSSI must be run
during your IPL sequence (EZAZSSI starts VMCF and TNF).
Either use your operation's automation software to start EZAZSSI, or add a command to
your COMMNDxx member in [Link]:
COM='S EZAZSSI,P=your_node_name'
The value of variable P defaults to the value of the MVS symbolic &SYSNAME. If your
node name is the same as the value of &SYSNAME, then you can use the following
command instead:
COM='S EZAZSSI'
When the EZAZSSI address space starts, a series of messages is written to the MVS log
indicating the status of VMCF and TNF. Then, the EZAZSSI address space terminates.
After VMCF and TNF initialize successfully, you can start your TCP/IP system address
spaces.
7. IKJTSOxx
You also need to specify CS for z/OS IP modules as authorized for TSO commands.
Update the IKJTSOxx member by adding the following to the AUTHCMD section:
MVPXDISP, NETSTAT, TRACERTE, RSH, LPQ, LPR, and LPRM.
8. IEASYSxx
Review your CSA and SQA specifications and verify that the numbers allocated are
sufficiently large enough to prevent getmain errors.
IEASYSxx: CSA(3000,250M)
IEASYSxx: SQA(8,448)

72 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

9. IVTPRMxx
Review the computed CSM requirements to reflect ACF/VTAM and CS for z/OS IP usage:
– IVTPRMxx: FIXED MAX(120M)
– IVTPRMxx: ECSA MAX(120M)
[Link]
Copy CTIEZB00 to [Link] from [Link] for use with CTRACE.
This member can be customized to include a different size buffer. The default buffer size is
8 MB. This should be increased to 32 MB to allow the capture of debugging information.
We made a new member, CTIEZB01, with the buffer size change.
For more information about the use of component tracing (CTRACE), refer to z/OS CS: IP
Diagnosis, [Link] z/OS CS: IP Migration, GC31-8773. Also see Chapter 8,
“Diagnosis” on page 299.
[Link]
In addition to defining the UNIX Physical File Systems, you must ensure that the ports
enabled on the system are consistent with what is defined in the [Link] data
set, as shown in Example 3-3.

Example 3-3 BPXPRMxx member with port range provided by a single stack environment
/* IPv4 support
NETWORK DOMAINNAME(AF_INET)
DOMAINNUMBER(2)
MAXSOCKETS(25000)
TYPE(INET)
INADDRANYPORT(10000) 8
INADDRANYCOUNT(2000) 8
* IPv6 support
NETWORK DOMAINNAME(AF_INET6)
DOMAINNUMBER(19)
TYPE(INET)

Ensure that the INADDRANYPORT 8 assignment does not conflict with PORT
assignments in the [Link] data set.

Note: The OpenEdition ENTRYPOINT for CS for z/OS IP is EZBPFINI. If you have the
incorrect value in BPXPRMxx member, you might see messages such as EZZ4203I or
abend codes such as S806.

Review the values specified in BPXPRMxx for MAXPROCSYS, MAXPROCUSER,

MAXUIDS, MAXFILEPROC, MAXPTYS, MAXTHREADTASKS, and MAXTHREADS.
[Link] or PROGxx
Use these to add product and feature information in a z/OS environment.

Chapter 3. Base functions 73

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

3.3.4 Changes to [Link] members

This section explains changes that you can make to incorporate the new TCP/IP functions.

TCP/IP JCL procedures

If you choose to use restartable VMCF and TNF, add procedure EZAZSSI:
//EZAZSSI PROC P=''
//STARTVT EXEC PGM=EZAZSSI,PARM=&P
//STEPLIB DD DSN=[Link],DISP=SHR

Update your TCP/IP startup JCL procedure. The sample for the CS for z/OS IP procedure is
in [Link](TCPIPROC).

TSO logon procedures

Update your TSO logon procedures by adding the TCP/IP help data set [Link] to the
//SYSHELP DD concatenation. Optionally, add the //SYSTCPD DD statement to your logon
procedures.

Add [Link] to the //ISPMLIB DD concatenation and [Link] to the

//ISPPLIB DD and the //ISPTLIB DD concatenations.

3.3.5 Additional z/OS customization for z/OS UNIX

Updating the MVS system libraries must be done with great care. Follow the instructions in
z/OS Program Directory, Program Number 5694-A01, GI10-0670, and check the PSP bucket
to ensure that all required PTFs and modifications are done as required. You might need to
make changes to some or all of the following members, depending on the features you are
installing.

3.3.6 TCP/IP server functions

Each CS for z/OS IP server relies on the use of a security manager, such as RACF. Several
servers provide some built-in security functions for additional security. These servers are
described in IBM z/OS Communications Server TCP/IP Implementation Volume 2: Standard
Applications, SG24-7897.

3.3.7 TCP/IP client functions

The client functions of Communications Server for z/OS IP are executed in a TSO
environment or a UNIX shell environment. Some functions are also available in other
environments, such as batch or started task address spaces.

Any TSO user can execute any TCP/IP command and use a TCP/IP client function to access
any other TCP/IP server host through the attached TCP/IP network. If these TCP/IP servers
have not implemented adequate password protection, then any TSO client user can log on to
these servers and access all data.

74 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

3.3.8 UNIX client functions

Certain client functions executed from the UNIX shell environment require superuser
authority. The user ID accessing the shell must have an OMVS segment associated with it.
RACF considerations for UNIX Client functions in CS for z/OS IP are covered in detail in IBM
z/OS Communications Server TCP/IP Implementation Volume 2: Standard Applications,
SG24-7897, and Communications Server for z/OS TCP/IP Implementation Volume 4:
Security and Policy-Based Networking, SG24-7899.

Common errors implementing UNIX System Services

In this section, we discuss implementation problems frequently encountered.

Superuser mode
Certain commands and operations from OMVS or from the ISHELL are authorized only for
superusers. There are two alternatives for running as a superuser:
 The user ID can have permanent superuser status.
This means that the ID has been created with a UID value of zero (0). TCP/IP started
tasks and some of its servers are also defined with a UID of zero.
 The user ID can have temporary authority for the superuser tasks.
The defined UID will have been set up as a non-zero value in RACF, but the user will have
been granted READ access to the RACF facility class of [Link]. Also, RACF
provides superuser granularity enhancements to assign functions to users that need them.

If you need only temporary authority to enter superuser mode, then granting simple READ
permission to the [Link] facility class will allow the user to switch back and forth
between superuser mode and standard mode. You can enter su from the OMVS shell, or you
can select SETUP OPTIONS from the ISHELL and specify Option #7 to obtain superuser
mode.

The user is then authorized to enter commands authorized for the superuser function from the
ISHELL, or switch to an OMVS shell the user has already signed onto. The basic prompt
level, indicated by the dollar sign ($) prompt, is changed when in superuser mode to a pound
sign (#). The exit command takes the user out of superuser mode as well as the OMVS
(UNIX) shell. Use of the whoami command shows the change of user IDs.

Problems with the home directory

In Example 3-4, the TSO user attempted unsuccessfully to enter the OMVS shell interface
from ISPF. The user has an OMVS segment defined but another problem occurs. The user
entered the TSO OMVS command to enter the UNIX environment and received the response
shown in Example 3-4.

Example 3-4 Error executing the TSO OMVS command

FSUM2078I No session was started. The home directory for this TSO/E
user does not exist or cannot be accessed, +
FSUM2079I Function = sigprocmask, return value = FFFFFFFF, return code
code = 9C reason code = 0507014D

This error occurred because the home directory that is associated with the user is not defined
or authorized in to the OMVS segment. You can determine the home directory with the RACF
listuser command (if you have the RACF authorization to use the command). However, you
still have access to the z/OS file, even though the message was displayed.

Chapter 3. Base functions 75

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

A similar problem occurs when trying to access the ISHELL environment, as shown in
Example 3-5.

Example 3-5 Error executing in the ISHELL

Errno=9Cx Process Initialization error; Reason=0507014D The dub failed
due to an error with the initial home directory. Press Enter to continue.

In both cases, the user had an OMVS segment defined in RACF. However, the home directory
that was associated with the user in the user’s OMVS segment was not defined or authorized.
(You can determine the home directory with the RACF listuser command.) Authorization is
provided with the permission bits.

The same symptom shows up for users without an OMVS segment defined if the
[Link] facility has been activated with an inaccessible home directory.

UNIX permission bits

You have already read something about setting up appropriate UNIX permission bits.
Example 3-6 shows an example of incorrect permission bits set for a user.

Example 3-6 Incorrect permission bits set for a user

ICH408I USER(CS01 ) GROUP(WTCRES ) NAME(CS RESIDENT ) 703
/u/CS01 CL(DIRSRCH) FID(01E2D7D3C5E7F34E2B0F000000000003)
INSUFFICIENT AUTHORITY TO LOOKUP
ACCESSINTENT(--X) ACCESS ALLOWED(OTHER ---)
ICH408I USER(CS01 ) GROUP(WTCRES ) NAME(CS RESIDENT ) 704

In this case, although the user has the UNIX permission bit settings of 755 on the /u/cs01/
directory, the permission bits are set at 600 for the /u/ directory. Thus, you must ensure that all
directories in the entire path are authorized with suitable permission bits. After the settings
are changed to 755 for the /u/ directory, access to the subdirectory is allowed.

You can display UNIX permission bits from the ISHELL environment or by issuing the ls -alF
command from the shell.

The ls -alF options indicate that all files should be listed (including hidden files), that the long
format should be displayed, and that the flags about the type of file (link, directory, and so on)
should be given.

Default search path and symbolic links

The directory search path is specified in the environment variable $PATH. Normally this
environment variable is set system-wide in the /etc/profile and can be further customized for
individual users in $home/.profile. The sample for /etc/profile sets $PATH to:
/bin:.

It should be expanded to:

/bin:/usr/sbin:.

or (depending on whether you want the current directory searched first or last):
.:/bin:/usr/sbin

The instructions for setting up this user profile are contained in z/OS UNIX System Services
User’s Guide, SA22-7801, and z/OS UNIX System Services Planning, SA22-7800.

76 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

Note: To view the search path that has been established for you, issue echo $PATH from
the shell environment.

A user might attempt to run a simple TCP/IP command, such as oping, and receive an error
that the command is not found, as shown in Example 3-7.

Example 3-7 Command not found error

BROWSE -- /tmp/cs01
Command ===>
********************************* Top of Data ***
oping: FSUM7351 not found

In this case you must preface the command with the directory path necessary to locate it:
/usr/lpp/tcpip/bin/oping

If you experience such a problem, check that the symbolic links are correct. Part of the
installation is to run the UNIX MKDIR program to set up the symbolic links for the various
commands and programs from their real path to /bin or /usr/sbin, where they can be found
using the default search path.

3.3.9 Verification checklist

The following checklist can help ensure that all z/OS and UNIX System Services related
setup tasks are complete for the base functions:
1. If you are using TNF and VMCF, have TNF and VMCF initialized successfully?
Check the console log for a successful start of EZAZSSI, TNF, VMCF.
2. Has the TCP/IP feature of z/OS been enabled or registered in IFAPRDxx?
3. Has a full-function OMVS (DFSMS, RACF, zFS) started successfully?
– Is OMVS active when you issue D OMVS?
– Is SMS active when you issue D SMS?
– Have z/OS UNIX file systems been mounted? Verify with D OMVS,F.
– Is RACF enabled on the system?
4. Have the definitions in BPXPRMxx of [Link] been made to reflect:
– The correct stack for the stack (or stacks) you will be running?
– The support for dual-mode is defined to support IPv4 and IPv6 (AF_INET and
AF_INET6)?
– The correct CS for z/OS IP proc names?
– The correct use of INET versus CINET?
– The correct ENTRYPOINT name for Communications Server for z/OS IP versus earlier
versions of OE function in TCP/IP (z/OS IP ENTRYPOINT = EZBPFINI)?
– The mounting of file systems for users? (You can verify with D OMVS,F.)
– Appropriate values for MAXPROCSYS, MAXPROCUSER, MAXUIDS,
MAXFILEPROC, MAXPTYS, MAXTHREADTASKS, and MAXTHREADS?
5. Have z/OS UNIX file systems and directories been created and mounted for the users of
the system?

Chapter 3. Base functions 77

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

6. Have RACF definitions been put in place? For example:

– OMVS user IDs and group IDs for your CS for z/OS IP procedures
– OMVS user IDs and group IDs for your other users, for superusers, for a default user,
with definitions for appropriate Facility classes, like [Link]
– TCP/IP VARY commands
– NETSTAT commands
7. Have you placed the correct definitions in the z/OS data sets? For example:
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
8. Raw sockets require authorization; they run from SEZALOAD and are usually already
authorized. If you have moved applications and functions to another library (which is not
recommended), ensure that this library is authorized.
9. The loopback address is now [Link] for IPv4 and ::1 for IPv6. However, If you require
[Link], have you added this to the HOME list?
[Link] you computed CSA requirements to include not only ACF/VTAM, but also CS for
z/OS IP?
– IEASYSxx: CSA(3000,250M) (need to review)
– IEASYSxx: SQA(8,448) (need to review)
[Link] you computed CSM requirements to include not only ACF/VTAM, but also CS for
z/OS IP?
– IVTPRMxx: FIXED MAX(120M)
– IVTPRMxx: ECSA MAX(120M)
[Link] you modified the CTRACE initialization member (CTIEZB00) to reflect 32 MB of
buffer storage?
[Link] you created CTRACE Writer procedures for taking traces?
[Link] you updated your TCP/IP procedure?
[Link] you updated your other procedures, for example, the FTP server procedure?
[Link] you revamped your TCP/IP Profile to use the new statements and to comment out
the old?
– Have you made provisions to address device connections that are no longer
supported?
– Have you investigated all your connections to ensure to what extent they are still
supported? (In some cases, definitions will have changed.)
[Link] your applications that relied on VMCF and IUCV sockets been converted now that
those APIs are no longer supported?
[Link] you are migrating from a previous release, have you reviewed the Planning and
Migration checklist in z/OS CS: IP Migration, GC31-8773, and made appropriate plans to
use the sample data sets?
[Link] you reviewed the list and location of configuration data set samples in z/OS
Communications Server: IP Configuration Reference, SC31-8776?

78 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

3.4 Configuring z/OS TCP/IP

A z/OS TCP/IP environment can be very complex. It is controlled using a large variety of
settings, including parmlib members, and /etc files for UNIX System Services. Each of these
has a different interface and requires special knowledge to configure.

z/OS Communications Server IP continues to be enhanced with new features,

enhancements, and defaults. So if you are migrating from a previous release, consult with the
migration guide for your particular release from which you are migrating. For further details,
refer to z/OS Communications Server: New Function Summary, GC31-8771.

3.4.1 TCP/IP configuration data set names

This topic is described in z/OS CS: IP Configuration Guide, SC31-8775. We strongly
recommend that you read the information about data set names in this book, before you
decide on your data set naming conventions.

The purpose here is to give an introduction to the data set naming and allocation techniques
that z/OS Communications Server uses. If you choose, you can allocate some of the
configuration data sets either implicitly or explicitly. In addition, you need to ensure that both
the MVS and the z/OS UNIX functions can find the data sets.
 Implicit allocation
The name of the configuration data set is resolved at runtime based on a set of rules (the
search order) implemented in the various components of TCP/IP. When a data set name
has been resolved, the TCP/IP component uses the dynamic allocation services of MVS
or of UNIX System Services to allocate that configuration data set. See z/OS CS: IP
Configuration Guide, SC31-8775, for details.
These are some of the data sets (or files) that can only be implicitly allocated in an z/OS
Communications Server IP:
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
[Link]
In these data set names, hlq is determined using the following search sequence:
– User ID or jobname
– DATASETPREFIX value (or its default of TCP/IP), defined in [Link]

Chapter 3. Base functions 79

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

Dynamically allocated data sets can include a mid-level qualifier (MLQ), for example, a
node name, or a function name.
– For data sets containing a PROFILE configuration file:
[Link]
– For data sets containing a translate table used by a particular TCP/IP server:
xxxx.function_name.zzzz (for the FTP server the function_name is SRVRFTP)
Data set [Link](TCPDATA) can be dynamically allocated if it contains the
[Link] configuration file.
 Explicit allocation
For some of the configuration files, you can tell TCP/IP which files to use by coding DD
statements in JCL procedures, or by setting UNIX environment variables. The various data
sets used by TCP/IP functions and their resolution method are described in z/OS CS: IP
Configuration Guide, SC31-8775.

3.4.2 [Link]
Before you start your TCP/IP stack, you must configure the operational and address space
characteristics. These definitions are defined in the configuration data set which is often
called [Link]. The [Link] data set is read by the TCP/IP address space
during initialization.

The PROFILE data set contains the following major groups of TCP/IP configuration
parameters:
 Operating characteristics
 Port number definitions
 Network interface definitions
 Network routing definitions

A sample [Link] configuration file is provided in [Link](SAMPPROF).

You can find detailed information about TCP/IP connectivity and routing definitions in
Chapter 4, “Connectivity” on page 117, and Chapter 5, “Routing” on page 205.

[Link] statements
In this section we show some essential statements for configuring TCP/IP stack.

The syntax for the parameters in the PROFILE can be found in z/OS Communications Server:
IP Configuration Reference, SC31-8776. Additional profile statements and descriptions are
available in “[Link] statements” on page 418.

Most PROFILE parameters required in a basic configuration have default values that will allow
the stack to be initialized and ready for operation. There are, however, a few parameters that
must be modified or must be unique to the stack.

Appendix D, “Our implementation environment” on page 457, describes the environment we

used to create this book.

DEVICE and LINK

Use DEVICE and LINK statements to define the physical or virtual interfaces, such as OSA,
HiperSockets, and VIPA. z/OS Communications Server can define multiple interfaces. You
need to define a pair of DEVICE and LINK statements for each interface you want to
configure for a TCP/IP stack.

80 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

Note: You can instead define IPv4 OSA-Express devices (IPQAENET) with the
INTERFACE statement. We recommend this approach, as described in “INTERFACE” on
page 81.

Each device type has a different set of parameters that you can define. For details on each
device type and its definition, refer to Chapter 4, “Connectivity” on page 117.

The following is an example of DEVICE and LINK statements for defining one OSA in QDIO
mode.
DEVICE OSA20A0 MPCIPA
LINK OSA20A0I IPAQENET OSA20A0

The following is an example of DEVICE and LINK statements for defining one VIPA.
DEVICE VIPA1 VIRTUAL 0
LINK VIPA1L VIRTUAL 0 VIPA1

INTERFACE
The INTERFACE statement defines all IPv6 interfaces and is enhanced to define IPv4
IPAQENET devices. This statement combines the definitions of the DEVICE, LINK, and
HOME into a single statement for IPv4 and IPv6.

The INTERFACE statement is set to reference the PORTNAME that is defined in the QDIO
TRLE definition statement as per DEVICE and LINK definitions and assigns an IP address to
it using the IPADDR operand, according to the HOME definition. Optional operands include
subnetmask settings using the /subnetmask bit number value in the IPADDR statement and
MTU size with the BEGINROTES or BSDROUTINGPARMS and SOURCEVIPAINT
statements, which associates a specific VIPA with this INTARFACE only.

Note: If SOURCEVIPAINT is coded, you define the entire INTERFACE definition block in
PROFILE after the VIPA DEVICE and LINK statements are defined.

You can define the VLANID and VMAC with the LINK statement, with the additional benefit
that you can use the INTERFACE statement to set multiple VLANs on the same OSA port.
You cannot, however, define multiple VLANs on the same OSA port with the LINK statement.

The devices that are defined through the INTERFACE statement return different displays than
devices that are defined through the DEVICE/LINK statements. See examples in
“INTERFACE statement” on page 430.

Example 3-8 shows a sample definition of the INTERFACE statement.

Example 3-8 INTERFACE statement in profile TCP/IP for IPv4 IPAQENET devices
INTERFACE OSA20A0I
DEFINE IPAQENET
PORTNAME OSA20A0
IPADDR [Link]/24
MTU 1492
VLANID 20
VMAC
SOURCEVIPAINT VIPA2L

Chapter 3. Base functions 81

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

You can delete a previously defined interface from the stack, after you stop it, with the
INTERFACE DELETE command using the OBEYFILE command, as shown in Example 3-9.

Example 3-9 INTERFACE delete statement

INTERFACE OSA20A0I
DELETE

More examples and displays are available in Appendix B, “INTERFACE statement” on

page 430.

Refer to z/OS Communications Server: IP Configuration Guide, SC31-8775, and z/OS

Communications Server: IP Configuration Reference, SC31-8776, for further details.

HOME
The HOME statement is used for assigning an IP address for each interface you defined with
DEVICE and LINK statements. The following is an example of a HOME statement.
HOME
[Link] VIPA1L
[Link] OSA20A0I

Note: The HOME statement (along with DEVICE and LINK) is mutually exclusive from the
INTERFACE statement. You must use one or the other. We recommend that you use
INTERFACE, as described in “INTERFACE” on page 81.

The TCP/IP stack uses an IP address of [Link] for IPv4 and ::1 for IPv6 as the loopback
interfaces. If there is a requirement to represent the loopback IP address of [Link] for
compatibility with earlier TCP/IP versions, you must code an entry in the HOME statement.
The link label specified is LOOPBACK and you can define multiple IP addresses with the
LOOPBACK interface. For example:
HOME
[Link] LOOPBACK

You can display the HOME IP address defined in a particular TCP/IP stack with a
D TCPIP,procname,Netstat HOME command, as shown in Example 3-10. You can also use
the z/OS UNIX shell command onetstat -h. There is an additional field, called the Flag field,
that indicates which interface is the primary interface. The primary interface is the first entry in
the HOME list in the [Link] definitions unless the PRIMARYINTERFACE
parameter is specified.

The PRIMARYINTERFACE statement can be used to specify which link is to be designated

as the default local host address for the GETHOSTID() function.

Example 3-10 netstat home display

D TCPIP,TCPIPA,N,HOME
EZD0101I NETSTAT CS V1R12 TCPIPA
HOME ADDRESS LIST:
LINKNAME: VIPA3L
ADDRESS: [Link]
FLAGS:
LINKNAME: VIPA1L
ADDRESS: [Link]
FLAGS: PRIMARY
LINKNAME: VIPA2L

82 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

ADDRESS: [Link]
FLAGS:
LINKNAME: IUTIQDF4L
ADDRESS: [Link]
FLAGS:
LINKNAME: IUTIQDF5L
ADDRESS: [Link]
FLAGS:
LINKNAME: IUTIQDF6L
ADDRESS: [Link]
FLAGS:
LINKNAME: EZASAMEMVS
ADDRESS: [Link]
FLAGS:
LINKNAME: IQDIOLNK0A01070B
ADDRESS: [Link]
FLAGS:
LINKNAME: VIPL0A010817
ADDRESS: [Link]
FLAGS:
LINKNAME: LOOPBACK
ADDRESS: [Link]
FLAGS:
INTFNAME: OSA2080I
ADDRESS: [Link]
FLAGS:
INTFNAME: OSA2081I
ADDRESS: [Link]
FLAGS:
INTFNAME: OSA20A0I
ADDRESS: [Link]
FLAGS:
INTFNAME: OSA20C0I
ADDRESS: [Link]
FLAGS:
INTFNAME: OSA20E0I
ADDRESS: [Link]
FLAGS:
INTFNAME: LOOPBACK6
ADDRESS: ::1
TYPE: LOOPBACK
FLAGS:
16 OF 16 RECORDS DISPLAYED

BEGINROUTES
Use this statement to define static routes for TCP/IP routing table. This statement is optional
when you use OMPROUTE dynamic routing daemon. However, if you do not configure
OMPROUTE dynamic routing daemon, BEGINROUTES is necessary for a TCP/IP stack to
communicate with other hosts. For details on static and dynamic routing, refer to Chapter 5,
“Routing” on page 205.

VIPADYNAMIC
This statement is not always necessary. Use this statement to define dynamic VIPA or the
functions related to dynamic VIPA, such as sysplex distributor and dynamic VIPA takeover.

Chapter 3. Base functions 83

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

Refer to Communications Server for z/OS V1R12 TCP/IP Implementation Volume 3: High
Availability, SG24-7898 for details about high availability and load balancing functions using
dynamic VIPA.

AUTOLOG
The procedures specified in AUTOLOG statement are initialized at TCP/IP startup, so you do
not have to start the TCP/IP applications manually after the TCP/IP startup. AUTOLOG also
monitors procedures started under its auspices, and will restart a procedure that terminates
for any reason unless NOAUTOLOG is specified on the PORT statement.

For UNIX servers, some special rules apply;

 If the procedure name on the AUTOLOG statement is eight characters long, no jobname
needs be specified.
 If the procedure name on the AUTOLOG statement is less than eight characters long and
the job spawns listener threads with different names, you might have to specify the
JOBNAME parameter and ensure that the jobname matches that coded on the PORT
statement. In the following example, jobname FTPDE1 on the PORT statement matches
JOBNAME on the AUTOLOG statement:
PORT
20 TCP * NOAUTOLOG ;OMVS
21 TCP FTPDA1 ;Contol Port

AUTOLOG 1
FTPDA JOBNAME FTPDA1 ; FTP Server
ENDAUTOLOG

START
Specify a device name on a START statement to initialize the interface at the TCP/IP stack
startup. The following is an example of START statement for an OSA and a HiperSockets
device. VIPA does not need to be started because it is virtual and always active.

If you do not specify a device name on a START statement, you can initialize the device with
the TCPIP,procname,START,devicename command after the TCP/IP stack startup.
START OSA20A0
START IUTIQDF4

IPCONFIG
IPv4 features are defined within IPCONFIG. There is a separate configuration section for IPv6
parameters. Refer to “[Link] statements” on page 418 for commonly used
IPCONFIG statements.

TCPCONFIG
TCP features are defined within TCPCONFIG. Refer to “[Link] statements” on
page 418 for commonly used TCPCONFIG statements.

UDPCONFIG
UDP features are defined within UDPCONFIG. Refer to “[Link] statements” on
page 418 for commonly used UDPCONFIG statements.

GLOBALCONFIG
GLOBALCONFIG statement defines the parameters that are affective to the entire TCP/IP
stack. Refer to “[Link] statements” on page 418 for commonly used
GLOBALCONFIG statements.

84 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

IPCONFIG6
All IPv6 features are defined within IPCONFIG6.

Locating [Link]
The following search order is used to locate the [Link] configuration file:
1. //PROFILE DD
//PROFILE DD DSN=[Link](PROFA30)
2. [Link]
3. [Link]
4. [Link]
5. [Link]

The PROFILE must exist. Otherwise, the TCP/IP address space will terminate abnormally
with the following message:
EZZ0332I DD:PROFILE NOT FOUND. CONTINUING PROFILE SEARCH
EZZ0325I INITIAL PROFILE COULD NOT BE FOUND

We recommend using the //PROFILE DD statement in the TCP/IP address space JCL
procedure to explicitly allocate the PROFILE data set.

3.4.3 VTAM Resource

As mentioned in the introduction, VTAM provides the Data Link Control layer (Layer 2 of the
OSI model) for TCP/IP, including support of the Multi-Path Channel (MPC) interfaces. MPC
protocols are used to define the DLC layer for OSA-Express devices in QDIO.

OSA-Express QDIO connections are configured through a TRLE definition. All TRLEs are
defined as VTAM major nodes. For further information about MPC-related devices/interfaces
refer to Chapter 4, “Connectivity” on page 117.

A TRLE definition we used for our OSA-Express in QDIO mode is shown in Example 3-11.

Example 3-11 TRLE VTAM major node definition for device OSA2080
OSA2080 VBUILD TYPE=TRL
OSA2080T TRLE LNCTL=MPC, *
READ=2080, *
WRITE=2081, *
DATAPATH=(2082-2087), *
PORTNAME=OSA2080, 1 *
MPCLEVEL=QDIO

Because VTAM provides the DLC layer for TCP/IP, then VTAM must be started before TCP/IP.
The major node (in our case, OSA2080) should be activated when VTAM is initializing. This
will ensure the TRLE is active when the TCP/IP stack is started. This is accomplished by
placing an entry for OSA2080 in the VTAM startup list ATCCONxx. The portname 1
(Example 3-11) must also be the same as the device name defined in [Link] data
set on the DEVICE and LINK statements.

This definition can be used for OSA-Express, OSA-Express 2 and OSA-Express 3 using only
port 0.

Chapter 3. Base functions 85

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

With OSA-Express 3, you can use both ports on the same TRL statement as shown in
Example 3-12.

Example 3-12 TRL VTAM majnode definition for two ports for device OSA2080
OSA2080 VBUILD TYPE=TRL
OSA200T TRLE LNCTL=MPC, *
READ=2080, *
WRITE=2081, *
DATAPATH=(2082-2087), *
PORTNAME=OSA2080, *
PORTNUM=0, *
MPCLEVEL=QDIO
OSA201T TRLE LNCTL=MPC, *
READ=2088, *
WRITE=2089, *
DATAPATH=(208A-208D), *
PORTNAME=OSA2081, *
PORTNUM=1, *
MPCLEVEL=QDIO

3.4.4 [Link]
The resolver configuration file is often called [Link]. The [Link] configuration
data set is the anchor configuration data set for the TCP/IP stack and all TCP/IP servers and
clients running on that stack.

The [Link] configuration data set is read during initialization of all TCP/IP server and
client functions. [Link] contains the configuration for the resolver address space. We
define the way name-to-address or address-to-name resolution is performed by the resolver.

[Link] is also used by the TCP/IP applications to specify the TCP/IP stack it establishes
an affinity with. The associated TCP/IP stack name is specified with TCPIPJOBNAME
statement. Other stack-specific statements are HOSTNAME, which is the host name of the
TCP/IP stack, and DATASETPREFIX, which is the data set prefix (hlq) to be used for
searching a configuration data set.

The syntax for the parameters in the [Link] file can be found in z/OS Communications
Server: IP Configuration Guide, SC31-8775. A sample [Link] configuration file is
provided in [Link](TCPDATA). You can define the [Link] parameters in an
MVS data set or z/OS UNIX file system file.

For further information about [Link] file and the resolver address space, refer to
Chapter 2, “The resolver” on page 19.

3.4.5 Configuring the local hosts file

You can set up the local hosts file to support local host name resolution. If you use only the
local hosts file for this purpose, your sockets applications will only be able to resolve names
and IP addresses that appear in your local hosts file.

If you need to resolve host names outside your local area, you can configure the resolver to
use a domain name server (see the NSINTERADDR or NAMESERVER statement in the
[Link] configuration file). A domain name server can be used in conjunction with the
local hosts file. If you have configured your resolver to use a name server, it will always try to

86 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

do so, unless your applications were written with a RESOLVE_VIA_LOOKUP symbol in the
source code.

Refer to Chapter 2, “The resolver” on page 19 for further explanation and details.

Chapter 3. Base functions 87

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

3.5 Implementing the TCP/IP stack

In this scenario we create a TCP/IP stack by the name of TCPIPA on the SC30 system (LPAR
A11). We define four OSAs and three HiperSockets interfaces and static routing. Figure 3-1
illustrates the OSA2080 and OSA20A0 pair connecting to the same VLAN using two different
OSA-Express features. The same applies to the OSA20C0 and OSA20E0 pair. We also
defined a dynamic XCF connection, which in our environment can use either a Coupling
Facility link or HiperSockets (CHPID F7).

XCF
10.1.7.x1

A11 (SC30) A13 (SC31) A16 (SC32) A18 (SC33)

TCPIPA TCPIPB
TCPIPC TCPIPD
PROFA30 (dynamic routes) PROFB31 (dynamic routes)
PROFC32 (dynamic routes) PROFD33 (dynamic routes)
PROFAS30 (static routes) PROFBS31 (static routes)
PROFCS32 (static routes) PROFDS33 (static routes)
VIPA3L [Link]/24 VIPA3L [Link]/24
VIPA1L [Link]/24 VIPA1L [Link]/24
VIPA1L [Link]/24 VIPA1L [Link]/24
VIPA2L [Link]/24 VIPA2L [Link]/24
VIPA2L [Link]/24 VIPA2L [Link]/24
OSA2080I [Link]/24 OSA2080I [Link]/24
OSA2080I [Link]/24 OSA2080I [Link]/24
OSA20A0I [Link]/24 OSA20A0I [Link]/24
OSA20A0I [Link]/24 OSA20A0I [Link]/24
OSA20C0I [Link]/24 OSA20C0I [Link]/24
OSA20C0I [Link]/24 OSA20C0I [Link]/24
OSA20E0I [Link]/24 OSA20E0I [Link]/24
OSA20E0I [Link]/24 OSA20E0I [Link]/24
IUTIQDF4L [Link]/24 IUTIQDF4L [Link]/24
IUTIQDF4L [Link]/24 IUTIQDF4L [Link]/24
IUTIQDF5L [Link]/24 IUTIQDF5L [Link]/24
IUTIQDF5L [Link]/24 IUTIQDF5L [Link]/24
IUTIQDF6L [Link]/24 IUTIQDF6L [Link]/24
IUTIQDF6L [Link]/24 IUTIQDF6L [Link]/24
XCF [Link]/24 XCF [Link]/24
XCF [Link]/24 XCF [Link]/24
VIPADEFINE [Link]/24 VIPADEFINE [Link]/24
VIPADEFINE [Link]/24 VIPADEFINE [Link]/24
VIPARANGE [Link]/24 VIPARANGE [Link]/24
VIPARANGE [Link]/24 VIPARANGE [Link]/24
([Link]-29) ([Link]-49)
([Link]-19) ([Link]-29)
HiperSockets CHPID F4 Devices E800-E81F IPADDR 10.1.4.x1
HiperSockets CHPID F5 Devices E900-E91F IPADDR 10.1.5.x1
HiperSockets CHPID F6 Devices EA00-EA1F IPADDR 10.1.6.x1
HiperSockets CHPID F7 Devices EB00-EB1F (DYNAMICXCF) IPADDR 10.1.7.x1

CHPID 02 CHPID 03 CHPID 04 CHPID 05

OSA2080 OSA20A0 OSA-Express 1000BASE-T OSA20C0 OSA20E0
10.1.2.x1 10.1.2.x2 10.1.3.x1 10.1.3.x2
2080-208F 20A0-20AF 20C0-20CF 20E0-2E0F

TRUNK MODE TRUNK MODE TRUNK MODE TRUNK MODE

VLAN 10 VLAN 11
[Link] [Link]

SWITCH

Figure 3-1 Network diagram

To implement the TCP/IP stack to support base functions, perform the following steps:
1. Create a [Link] file.
2. Create a [Link] file.
3. Check BPXPRMxx.
4. Create a TCP/IP cataloged procedure.
5. Add RACF definitions.
6. Create a VTAM TRL major node for MP CIPA OSA.

Allocate the TCPPARMS library to be used for explicitly allocated configuration data sets for
the stack, or create a new member in your existing TCPPARMS library. For example, we
allocated [Link](DATAA30).

88 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

3.5.1 Create [Link] file

We defined a global [Link] and a local [Link] for TCPIPA, as shown in
Example 3-13 and Example 3-14.

Example 3-13 Global [Link] file

; *****************************************
; [Link](GLOBAL)
; *****************************************\
DOMAINORIGIN [Link]
SEARCH [Link] [Link]
DATASETPREFIX TCPIP
MESSAGECASE MIXED
NSINTERADDR [Link]
NSPORTADDR 53
RESOLVEVIA UDP
RESOLVERTIMEOUT 10
RESOLVERUDPRETRIES 1
LOOKUP LOCAL

We created a local [Link] file for the TCPIPA stack.

Example 3-14 Local [Link] file

; *****************************************
; [Link](DATAA30)
; *****************************************
TCPIPJOBNAME TCPIPA 1
HOSTNAME WTSC30A 2
DATASETPREFIX TCPIPA
MESSAGECASE MIXED

In this example, the numbers correspond to the following information:

1. Specifies the procedure name of TCPIPA stack.
2. Specifies the host name of the TCPIPA stack.

Update the domain name server

If you are using a domain name server, ensure that it is updated with your new host name and
address.

Update the local hosts file

If you are not using a domain name server, edit your global [Link] file or the local
[Link] file and add your new host name and address.

Chapter 3. Base functions 89

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

3.5.2 Create the [Link] file

We created a TCP/IP profile and included the statements described in this section.

INTERFACE statement
We configured two OSA-Express3 features, each having four ports. We configured only two
ports on each card with the INTERFACE statement. For redundancy we defined two VLANs,
with each pair using one port per feature and each pair attached to the same VLAN. This
facilitates ARP Takeover.

DEVICE and LINK statement

We defined HiperSockets and VIPA devices with the DEVICE and LINK statements, and the
others with the INTERFACE statement.

HOME statement
We assigned a IP address for each interface that was configured with a DEVICE/LINK
statement pair.

BEGINROUTES statement
We defined static routes with BEGINROUTES statement to route a traffic to other hosts on a
network using the OSA-Express or HiperSockets interfaces.

PORT statement
We reserved TCP ports for some applications with PORT statement

START statement
We defined a START statement to initialize the interfaces at the TCP/IP stack startup.

DYNAMICXCF statement
We defined a DYNAMICXCF statement to dynamically define the device to join the sysplex.

Example 3-15 shows a sample [Link] file.

Example 3-15 [Link] file

; *****************************************
; [Link](PROFA30S)
; *****************************************
ARPAGE 20
;
GLOBALCONFIG NOTCPIPSTATISTICS
;
IPCONFIG DATAGRAMFWD SYSPLEXROUTING
;
DYNAMICXCF [Link] [Link] 1
;
SOMAXCONN 10
;
TCPCONFIG TCPSENDBFRSIZE 64K TCPRCVBUFRSIZE 64K SENDGARBAGE FALSE
TCPCONFIG TCPMAXRCVBUFRSIZE 256K
TCPCONFIG UNRESTRICTLOWPORTS
;
UDPCONFIG UNRESTRICTLOWPORTS

90 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

;
;INTERFACE OSA20x0I DEFINE IPAQENET (OSA-E) PORTNAME OSA20x0
;TRL MAJ NODE: OSA2080,OSA20A0,OSA20C0,AND OSA20E0
;
INTERFACE OSA2080I
DEFINE IPAQENET
PORTNAME OSA2080
IPADDR [Link]/24
VLANID 10
VMAC
;
INTERFACE OSA20A0I
DEFINE IPAQENET
PORTNAME OSA20A0
IPADDR [Link]/24
VLANID 10
VMAC
;
INTERFACE OSA20C0I
DEFINE IPAQENET
PORTNAME OSA20C0
IPADDR [Link]/24
VLANID 11
VMAC
;
INTERFACE OSA20E0I
DEFINE IPAQENET
PORTNAME OSA20E0
IPADDR [Link]/24
VLANID 11
VMAC
;
;HIPERSOCKETS DEFINITIONS
DEVICE IUTIQDF4 MPCIPA
LINK IUTIQDF4L IPAQIDIO IUTIQDF4
DEVICE IUTIQDF5 MPCIPA
LINK IUTIQDF5L IPAQIDIO IUTIQDF5
DEVICE IUTIQDF6 MPCIPA
LINK IUTIQDF6L IPAQIDIO IUTIQDF6

;
;STATIC VIPA DEFINITIONS
DEVICE VIPA1 VIRTUAL 0
LINK VIPA1L VIRTUAL 0 VIPA1
DEVICE VIPA2 VIRTUAL 0
LINK VIPA2L VIRTUAL 0 VIPA2
;
HOME
[Link] VIPA1L
[Link] VIPA2L
[Link] IUTIQDF4L
[Link] IUTIQDF5L
[Link] IUTIQDF6L
BEGINRoutes
; Direct Routes - Routes that are directly connected to my interfaces

Chapter 3. Base functions 91

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

; Destination Subnet Mask First Hop Link Name Packet Size

ROUTE [Link]/24 = OSA2080I MTU 1492
ROUTE [Link]/24 = OSA20C0I MTU 1492
ROUTE [Link]/24 = OSA20E0I MTU 1492
ROUTE [Link]/24 = IUTIQDF4L MTU 8192
ROUTE [Link]/24 = IUTIQDF5L MTU 8192
ROUTE [Link]/24 = IUTIQDF6L MTU 8192
;
PORT
20 TCP OMVS NOAUTOLOG ; FTP Server
21 TCP FTPDA1 ; control port
23 TCP TN3270A BIND [Link] ; OE Telnet Server
500 UDP IKED ; @ADI
520 UDP OMPROUTE NOAUTOLOG ; OMPROUTE RIPV2 port
521 UDP OMPROUTE NOAUTOLOG ; OMPROUTE RIPV2 port
4500 UDP IKED ; @AD
514 UDP OMVS ; UNIX SyslogD Server 3
;
START OSA2080I
START OSA20C0I
START OSA20E0I
START OSA20A0I
START IUTIQDF4
START IUTIQDF5
START IUTIQDF6

3.5.3 Check BPXPRMxx

Refer to [Link](BPXPRMxx) and make sure you have your TCP/IP stack name
defined in it. If you do not have the stack name in BPXPRMxx, refer to 3.3.3, “Changes to
[Link] members” on page 68.

3.5.4 Create TCP/IP cataloged procedure

We created a cataloged procedure for TCPIPA stack, as shown in Example 3-16.

Example 3-16 Address space JCL procedure (SC30)

//TCPIPA PROC PARMS='CTRACE(CTIEZB00),IDS=00',
// PROFILE=PROFA&SYSCLONE,TCPDATA=DATAA&SYSCLONE 1
//TCPIPA EXEC PGM=EZBTCPIP,REGION=0M,TIME=1440,
// PARM=('&PARMS',
// 'ENVAR("RESOLVER_CONFIG=//''[Link](&TCPDATA)''")')
//SYSPRINT DD SYSOUT=*,DCB=(RECFM=VB,LRECL=132,BLKSIZE=136)
//ALGPRINT DD SYSOUT=*,DCB=(RECFM=VB,LRECL=132,BLKSIZE=136)
//CFGPRINT DD SYSOUT=*,DCB=(RECFM=VB,LRECL=132,BLKSIZE=136)
//SYSOUT DD SYSOUT=*,DCB=(RECFM=VB,LRECL=132,BLKSIZE=136)
//CEEDUMP DD SYSOUT=*,DCB=(RECFM=VB,LRECL=132,BLKSIZE=136)
//SYSERROR DD SYSOUT=*
//PROFILE DD DISP=SHR,DSN=[Link](&PROFILE.)
//SYSTCPD DD DSN=[Link](&TCPDATA.),DISP=SHR 2

92 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

In this example, the numbers correspond to the following information:

1. Illustrates the use of SYSTEM SYMBOLS.
2. SYSTCPD DD statement pointing to [Link](DATAA30).

3.5.5 Add RACF definitions

The RACF administrator needs to add RACF definitions to assign started task user IDs to
new address spaces, as shown in Example 3-17.

Example 3-17 Defining TCPIP*.* procedure to started task

ADDGROUP TCPGRP OMVS(UID(100))
ADDUSER TCPIP DFLTGRP(TCPGRP) OMVS(UID(0) HOME(/) PROGRAM(/bin/sh)) NOPASSWORD
SETROPTS GENERIC(STARTED)
SETROPTS CLASSACT(STARTED) RACLIST(STARTED)
DEFINE STARTED TCPIP*.* STDATA(USER(TCPIP) GROUP(TCPGRP))
SETROPTS RACLIST(STARTED) REFRESH

3.5.6 Create a VTAM TRL major node for MPCIPA OSA

We defined our TRLEs in VTAM. Remember to include it in the VTAM startup list in
ATCCONxx. Example 3-18 and Example 3-19 are sample TRL major nodes for one OSA
device. We then created TRLEs for all OSA devices.

Example 3-18 displays the TRLE VTAM major node definition for device OSA2080.

Example 3-18 TRLE VTAM major node definition for device OSA2080
OSA2080 VBUILD TYPE=TRL
OSA200T TRLE LNCTL=MPC, *
READ=2080, *
WRITE=2081, *
DATAPATH=(2082-2087), *
PORTNAME=OSA2080, *
PORTNUM=0, *
MPCLEVEL=QDIO
OSA201T TRLE LNCTL=MPC, *
READ=2088, *
WRITE=2089, *
DATAPATH=(208A-208D), *
PORTNAME=OSA2081, *
PORTNUM=1, *
MPCLEVEL=QDIO

Example 3-19 displays the TRLE’s VTAM major node definitions for devices OSA20A0 and
OSA20A1.

Example 3-19 TRLE VTAM major node definition for device OSA20A0
OSA20A0 VBUILD TYPE=TRL
OSA20A0T TRLE LNCTL=MPC, *
READ=20A0, *
WRITE=20A1, *
DATAPATH=(20A2-20A7), *
PORTNAME=OSA20A0, *

Chapter 3. Base functions 93

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

PORTNUM=0, *
MPCLEVEL=QDIO
*
OSA20A1 VBUILD TYPE=TRL
OSA20A1T TRLE LNCTL=MPC, *
READ=20A8, *
WRITE=20A9, *
DATAPATH=(20AA-20AE), *
PORTNAME=OSA20A1, *
PORTNUM=1, *
MPCLEVEL=QDIO

Note: If server-specific configuration data sets can be explicitly allocated using DD

statements, we recommend that you create the configuration data set as a member in the
stack-specific TCPPARMS library. If the data set has to be implicitly allocated, remember to
create it with the stack-specific data set prefix.

3.6 Activating the TCP/IP stack

If you IPL your z/OS system with PARMLIB definitions similar to our environment, you should
get messages similar to those shown in Example 3-20. These are some of the messages that
can be used to verify the accuracy of the current environment customization data sets used in
z/OS UNIX and TCP/IP initialization.

Note that messages issued by z/OS UNIX begin with the prefix BPX.

Example 3-20 IPL and start TCPIPA

/* IPL and start of TCPIPA
IEE252I MEMBER BPXPRM1A FOUND IN [Link] 1
CEE3739I LANGUAGE ENVIRONMENT INITIALIZATION COMPLETE 2
IEE252I MEMBER CTIEZB00 FOUND IN [Link]
IEE252I MEMBER CTIIDS00 FOUND IN [Link]
IEE252I MEMBER CTINTA00 FOUND IN [Link]
/*
EZZ4202I Z/OS UNIX - TCP/IP CONNECTION ESTABLISHED FOR TCPIPA
BPXF206I ROUTING INFORMATION FOR TRANSPORT DRIVER TCPIPA HAS BEEN 3
INITIALIZED OR UPDATED.
/*
EZZ4313I INITIALIZATION COMPLETE FOR DEVICE OSA2080
EZZ4313I INITIALIZATION COMPLETE FOR DEVICE OSA2081
EZZ4313I INITIALIZATION COMPLETE FOR DEVICE OSA20C0
EZZ4313I INITIALIZATION COMPLETE FOR DEVICE OSA20E0
EZZ4340I INITIALIZATION COMPLETE FOR INTERFACE OSA20A0I
EZZ4340I INITIALIZATION COMPLETE FOR INTERFACE OSA20A0X
EZB6473I TCP/IP STACK FUNCTIONS INITIALIZATION COMPLETE.
EZZ4313I INITIALIZATION COMPLETE FOR DEVICE IUTIQDF5
EZZ4313I INITIALIZATION COMPLETE FOR DEVICE IUTIQDF6
EZZ4313I INITIALIZATION COMPLETE FOR DEVICE IUTIQDF4
EZB6473I TCP/IP STACK FUNCTIONS INITIALIZATION COMPLETE. 5
EZAIN11I ALL TCPIP SERVICES FOR PROC TCPIPA ARE AVAILABLE.
/*
EZBH006E GLOBALCONFIG SYSPLEXMONITOR RECOVERY was not specified when
IPCONFIG DYNAMICXCF or IPCONFIG6 DYNAMICXCF was configured.
/*
EZD1176I TCPIPA HAS SUCCESSFULLY JOINED THE TCP/IP SYSPLEX GROUP 4
EZBTCPCS

94 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

EZD1214I INITIAL DYNAMIC VIPA PROCESSING HAS COMPLETED FOR TCPIPA

EZZ4313I INITIALIZATION COMPLETE FOR DEVICE IUTIQDIO
/*

In this example, the numbers correspond to the following information:

1. The first important message indicates whether the correct UNIX customization data set is
used. In our environment it is BPXPRM1A. This contains the root system upon which all
other file systems are mounted, and it is critical for the current establishment of the correct
UNIX System Services environment.

The next set of messages shows the initialization of SMS. SMS is a critical component,
because the zFSs are SMS-managed. Note that the file systems are mounted subsequently
starting with the root.
2. This message indicates that the Language Environment is available to be exploited by
TCP/IP Lotus®, WebSphere®, and parts of the z/OS base, as well as by languages such
as C/C++, COBOL, and others.

The next set of messages indicates the successful establishment of the physical file system
and availability for socket services for both IPv4 and IPv6. The resolver messages indicate
that the resolver process is available to support network resolution, which can be critical to
some applications. Note that the initialization of the resolver is completed before TCP/IP.
3. The following two messages indicate the successful initialization of the UNIX System
Services environment and TCP/IP services according to the BPXPRMxx definitions.
4. Our environment is defined within a sysplex; therefore, message EZD1176I indicates the
connectivity to other active TCP/IP stacks within the sysplex.

Initialization of devices must be completed before they achieve READY status (displayed
using the NETSTAT DEVLNKS) and connected to the network.
5. The EZB6473I and EZAIN11I messages are the final initialization messages to complete
the successful initialization of the TCP/IP stack.

3.6.1 UNIX System Services verification

A few commands can be used to perform a simple verification of the z/OS UNIX environment
after an maintenance IPL; for example, D SMS verifies that the system is running a functional
SMS environment, as shown in Example 3-21.

Example 3-21 Output of “D SMS” command

RESPONSE=SC30
IGD002I [Link] DISPLAY SMS 836
SCDS = [Link]
ACDS = [Link]
COMMDS = [Link]
DINTERVAL = 150
REVERIFY = NO
ACSDEFAULTS = NO
SYSTEM CONFIGURATION LEVEL INTERVAL SECONDS
SC30 2010/09/29 [Link] 10
SC31 2010/09/29 [Link] 10
SC32 2010/09/29 [Link] 10
SC33 2010/09/29 [Link] 10
WTSCPLX5 ---------- -------- N/A

Chapter 3. Base functions 95

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

Example 3-22 shows the output from the SMS display that supports four different LPARs. The
captured information pertains to SC30, SC31, SC32 and SC33.

Example 3-22 Displaying the OMVS system that is running

D OMVS,ASID=ALL
BPXO070I 15.48.10 DISPLAY OMVS 878
OMVS 000F ACTIVE OMVS=(2A) 1
USER JOBNAME ASID PID PPID STATE START CT_SECS
OMVSKERN BPXOINIT 0025 1 0 MRI----- 07.00.40 1.3 2
LATCHWAITPID= 0 CMD=BPXPINPR
SERVER=Init Process AF= 0 MF=00000 TYPE=FILE
OMVSKERN 0000 65539 1 1L------ 07.00.41 .0
IBMUSER CEA 0018 16842756 1 1F---P-- 07.00.47 .0
LATCHWAITPID= 0 CMD=CEAPSRVR
OMVSKERN SYSLOGDA 0043 50397189 1 HF------ 07.00.41 .6
LATCHWAITPID= 0 CMD=/usr/sbin/syslogd -c -i -u -f /etc/syslo
NET NET 001F 65542 1 1F---P-- 07.00.41 28.5
LATCHWAITPID= 0 CMD=ISTMGCEH
IBMUSER HZSPROC 002D 33619975 1 1R---B-- 07.00.47 5.3
LATCHWAITPID= 0 CMD=HZSTKSCH
RMF RMFGAT 0046 65545 1 1R---P-- 07.00.45 611.9
LATCHWAITPID= 0 CMD=ERB3GMFC
TCPIP TCPIP 0045 65546 1 MF---B-- 07.00.45 22.6
LATCHWAITPID= 0 CMD=EZBTCPIP
IBMUSER JES2S001 0021 16842765 1 1R------ 07.00.47 .7
LATCHWAITPID= 0 CMD=IAZNJTCP
TCPIP TCPIP 0045 16842766 1 1F---B-- 07.00.48 22.6
LATCHWAITPID= 0 CMD=EZASASUB
TCPIP TCPIP 0045 33619983 1 1F---B-- 07.00.48 22.6
LATCHWAITPID= 0 CMD=EZACFALG
TCPIP FTPMVS1 0042 33619986 1 1FI----- 07.00.59 .0
LATCHWAITPID= 0 CMD=FTPD
TCPIP NFSCLNT 002C 33619987 1 HR---B-- 07.00.56 .8
LATCHWAITPID= 0 CMD=GFSCINIT
SERVER=MVSNFSC AF= 0 MF=00000 TYPE=FILE
OMVSKERN INETD1 0040 33619988 1 1FI----- 07.00.55 .0
LATCHWAITPID= 0 CMD=/usr/sbin/inetd /etc/[Link]
TCPIP REXECD 004B 65557 1 1FI----- 07.00.58 .0
LATCHWAITPID= 0 CMD=RSHD
TCPIP TN3270 0049 65558 1 MR---B-- 07.00.58 11.2
LATCHWAITPID= 0 CMD=EZBTNINI
TCPIP PORTMAP 004A 33619991 1 1FI----- 07.00.58 .0
LATCHWAITPID= 0 CMD=PORTMAP
TCPIP FTPOE1 0041 65560 1 1FI----- 07.00.58 .0
LATCHWAITPID= 0 CMD=FTPD
IBMUSER JES2S001 0021 65561 1 1R------ 07.01.04 .7
LATCHWAITPID= 0 CMD=IAZNJSTK
CS02 CS02 0047 33620005 1 MRI----- 09.09.35 .7
LATCHWAITPID= 0 CMD=EXEC
CS02 0000 33620006 33620005 1Z------ 09.26.43 .0
TCPIP TCPIPA 005D 50397230 1 1F---B-- 13.59.13 26.3
LATCHWAITPID= 0 CMD=EZACFALG
TCPIP TCPIPD 005F 16842800 1 1R---B-- 14.01.08 1.2
LATCHWAITPID= 0 CMD=EZACFALG
TCPIP TCPIPD 005F 33620018 1 MR---B-- 14.01.06 1.2

96 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

LATCHWAITPID= 0 CMD=EZBTCPIP
TCPIP TCPIPA 005D 50397245 1 1F---B-- 13.59.13 26.3 3
LATCHWAITPID= 0 CMD=EZASASUB
TCPIP OMPC 0065 50397268 1 HS------ 13.58.29 1.3
LATCHWAITPID= 0 CMD=OMPROUTE
TCPIP TCPIPB 0056 16842838 1 1F---B-- 13.31.54 1.8
LATCHWAITPID= 0 CMD=EZACFALG
TCPIP TCPIPB 0056 65632 1 1F---B-- 13.31.54 1.8
LATCHWAITPID= 0 CMD=EZASASUB
TCPIP TCPIPB 0056 16842849 1 MF---B-- 13.31.51 1.8
LATCHWAITPID= 0 CMD=EZBTCPIP
TCPIP TRAPFWDB 0064 65637 1 1FI----- 13.32.04 .0
LATCHWAITPID= 0 CMD=EZASNTRA
TCPIP TNLUNS30 0067 33620070 1 MR---B-- 15.22.03 .2
LATCHWAITPID= 0 CMD=EZBTNINI
TCPIP SNMPQEB 0063 16842855 1 1FI----- 13.32.04 .0
LATCHWAITPID= 0 CMD=SQESERV
TCPIP TNLUNS30 0067 83951720 1 1F---B-- 15.22.03 .2
LATCHWAITPID= 0 CMD=EZBTSSUB
TCPIP TCPIPC 0060 65646 1 1F---B-- 13.58.19 1.6
LATCHWAITPID= 0 CMD=EZASASUB
TCPIP TCPIPC 0060 65651 1 1F---B-- 13.58.19 1.6
LATCHWAITPID= 0 CMD=EZACFALG
TCPIP TCPIPC 0060 65652 1 MR---B-- 13.58.16 1.6
LATCHWAITPID= 0 CMD=EZBTCPIP
TCPIP TCPIPA 005D 83951733 1 MF---B-- 13.59.10 26.3
LATCHWAITPID= 0 CMD=EZBTCPIP
TCPIP FTPDA1 0050 50397302 1 1FI----- 13.59.23 .0
LATCHWAITPID= 0 CMD=FTPD
TCPIP OMPA 0053 50397304 1 HS------ 14.17.39 427.9
LATCHWAITPID= 0 CMD=OMPROUTE
TCPIP TN3270A 005E 67174525 1 1F---B-- 15.12.20 .2
LATCHWAITPID= 0 CMD=EZBTSSUB
TCPIP TN3270A 005E 16842893 1 MR---B-- 15.12.20 .2
LATCHWAITPID= 0 CMD=EZBTNINI
IBMUSER IOASRV 004D 83951763 1 1FI----- 15.44.13 .0
LATCHWAITPID= 0 CMD=IOAXTSRV
TCPIP SNMPDB 0068 83951764 1 1FI----- 15.46.24 .0
LATCHWAITPID= 0 CMD=EZASNMPD
CS03 CS03 0052 83951774 1 1RI----- 17.36.25 3.3
LATCHWAITPID= 0 CMD=EXEC
CS03 CS03 005B 16842915 33620132 1CI----- 17.55.49 .1
LATCHWAITPID= 0 CMD=sh -L
CS03 CS03 005B 33620132 33619988 1FI----- 17.55.38 .1
LATCHWAITPID= 0 CMD=otelnetd -Y [Link] -p cs03 -A ansi -

In Example 3-22 on page 96:

 The OMVS member that is running is related to 1 BPXPRM97A.
 The initialization process is running as superuser 2 OMVSKERN, and the PID is 1 (the first
process to start).
 There is another TCP/IP started task running 3.

What is also significant here is that OMVS=DEFAULT is not displayed in the output. In our
previous review of the z/OS UNIX environment, we mentioned that the z/OS UNIX System

Chapter 3. Base functions 97

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

Services must be customized in full-function mode. The display tells you that, at the very
least, your system is not running in default mode (minimal mode).

Also notice the different TPC/IP stacks and tasks associated with them. There is TCPIPA and
TCPIP (the default stack), both executing EZBTCPIP. There are also multiple tasks
associated with the same RACF user ID, TCPIP. This offers the advantage of easier
maintenance and system definitions, However, this also presents the disadvantage of having
no distinguishing features among messages for individual tasks. Many users of TCP/IP and
UNIX System Services would assign individual RACF user IDs to each OMVS user for easier
problem determination.

For a thorough discussion about the use and implementation of RACF, refer to
Communications Server for z/OS TCP/IP Implementation Volume 4: Security and
Policy-Based Networking, SG24-7899.

Example 3-23 shows the display of available file systems after the initialization of the z/OS
UNIX System Services environment. The display should list all of the files defined in the
mount statement in the BPXPRMxx member, which in our scenario is BPXPRM9A.

Example 3-23 Output of D OMVS,F

D OMVS,F
OMVS 000F ACTIVE OMVS=(2A)
TYPENAME DEVICE ----------STATUS----------- MODE MOUNTED LATCHES
ZFS 784 ACTIVE RDWR 09/28/2010 L=63
NAME=[Link] 11.48.32 Q=0
PATH=/u/rc30
OWNER=SC30 AUTOMOVE=Y CLIENT=N
ZFS 115 ACTIVE RDWR 09/28/2010 L=57
NAME=[Link] 07.13.15 Q=0
PATH=/SC33/web/hod
OWNER=SC33 AUTOMOVE=U CLIENT=Y
ZFS 101 ACTIVE RDWR 09/28/2010 L=55
NAME=[Link] 07.13.08 Q=0
PATH=/SC33/var
OWNER=SC33 AUTOMOVE=U CLIENT=Y
ZFS 100 ACTIVE RDWR 09/28/2010 L=54
NAME=[Link] 07.13.08 Q=0
PATH=/SC33/etc
OWNER=SC33 AUTOMOVE=U CLIENT=Y
ZFS 99 ACTIVE RDWR 09/28/2010 L=53
NAME=[Link] 07.13.08 Q=0
PATH=/SC33
OWNER=SC33 AUTOMOVE=U CLIENT=Y
ZFS 72 ACTIVE RDWR 09/28/2010 L=47
NAME=[Link].BW311 07.04.24 Q=0
PATH=/SC31/web/bw311
OWNER=SC31 AUTOMOVE=U CLIENT=Y
ZFS 42 ACTIVE RDWR 09/28/2010 L=40
NAME=[Link].BW301 07.00.48 Q=0
PATH=/SC30/web/bw301
OWNER=SC30 AUTOMOVE=U CLIENT=N
ZFS 17 ACTIVE RDWR 09/28/2010 L=33
NAME=[Link] 07.00.41 Q=0
PATH=/pp/HOD/hostondemand/private
OWNER=SC30 AUTOMOVE=Y CLIENT=N

98 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

ZFS 16 ACTIVE RDWR 09/28/2010 L=32

NAME=[Link] 07.00.41 Q=0
PATH=/pp/HOD
OWNER=SC30 AUTOMOVE=Y CLIENT=N
ZFS 14 ACTIVE READ 09/28/2010 L=27
NAME=[Link] 07.00.39 Q=0
PATH=/Z1CRB1/usr/lpp/zosmf/V1R12
OWNER=SC30 AUTOMOVE=Y CLIENT=N
ZFS 13 ACTIVE READ 09/28/2010 L=26
NAME=OMVS.ZOSR1C.Z1CRB1.SBBN7HFS 07.00.39 Q=0
PATH=/Z1CRB1/usr/lpp/zWebSphereOEM/V7R0
OWNER=SC30 AUTOMOVE=Y CLIENT=N
ZFS 12 ACTIVE READ 09/28/2010 L=25
NAME=[Link] 07.00.39 Q=0
PATH=/Z1CRB1/usr/lpp/ixm
OWNER=SC30 AUTOMOVE=Y CLIENT=N
ZFS 11 ACTIVE READ 09/28/2010 L=24
NAME=OMVS.ZOSR1C.Z1CRB1.JAVA64V6 07.00.39 Q=0
PATH=/Z1CRB1/usr/lpp/java/J6.0_64
OWNER=SC30 AUTOMOVE=Y CLIENT=N
ZFS 10 ACTIVE READ 09/28/2010 L=23
NAME=OMVS.ZOSR1C.Z1CRB1.JAVA31V6 07.00.38 Q=0
PATH=/Z1CRB1/usr/lpp/java/J6.0
OWNER=SC30 AUTOMOVE=Y CLIENT=N
ZFS 9 ACTIVE READ 09/28/2010 L=22
NAME=OMVS.ZOSR1C.Z1CRB1.JAVA64V5 07.00.38 Q=0
PATH=/Z1CRB1/usr/lpp/java/J5.0_64
OWNER=SC30 AUTOMOVE=Y CLIENT=N
ZFS 8 ACTIVE READ 09/28/2010 L=21
NAME=OMVS.ZOSR1C.Z1CRB1.JAVA31V5 07.00.38 Q=0
PATH=/Z1CRB1/usr/lpp/java/J5.0
OWNER=SC30 AUTOMOVE=Y CLIENT=N
ZFS 3 ACTIVE RDWR 09/28/2010 L=16
NAME=[Link] 07.00.37 Q=16
PATH=/Z1CRB1
OWNER=SC30 AUTOMOVE=Y CLIENT=N
AUTOMNT 18 ACTIVE RDWR 09/28/2010 L=34
NAME=*AMD/u 07.00.41 Q=0
PATH=/u
OWNER=SC30 AUTOMOVE=Y CLIENT=N
TFS 102 ACTIVE RDWR 09/28/2010 L=56
NAME=/SC33/TMP 07.13.08 Q=0
PATH=/SC33/tmp
MOUNT PARM=-s 500
OWNER=SC33 AUTOMOVE=U CLIENT=Y
TFS 78 ACTIVE RDWR 09/28/2010 L=51
NAME=/SC32/TMP 07.05.17 Q=0
PATH=/SC32/tmp
MOUNT PARM=-s 500
OWNER=SC32 AUTOMOVE=U CLIENT=Y
TFS 48 ACTIVE RDWR 09/28/2010 L=44
NAME=/SC31/TMP 07.04.15 Q=0
PATH=/SC31/tmp
MOUNT PARM=-s 500
OWNER=SC31 AUTOMOVE=U CLIENT=Y

Chapter 3. Base functions 99

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

TFS 7 ACTIVE RDWR 09/28/2010 L=20

NAME=/DEV 07.00.38 Q=0
PATH=/SC30/dev
MOUNT PARM=-s 10
OWNER=SC30 AUTOMOVE=U CLIENT=N
TFS 6 ACTIVE RDWR 09/28/2010 L=19
NAME=/SC30/TMP 07.00.38 Q=0
PATH=/SC30/tmp
MOUNT PARM=-s 500
OWNER=SC30 AUTOMOVE=U CLIENT=N
HFS 789 ACTIVE RDWR 09/28/2010 L=62
NAME=[Link] 11.50.30 Q=0
PATH=/u/cs02
OWNER=SC30 AUTOMOVE=Y CLIENT=N
HFS 679 ACTIVE RDWR 09/28/2010 L=61
NAME=[Link] 11.05.22 Q=0
PATH=/u/cs03
OWNER=SC30 AUTOMOVE=Y CLIENT=N
HFS 615 ACTIVE RDWR 09/28/2010 L=60
NAME=[Link] 10.39.56 Q=0
PATH=/u/cs01
OWNER=SC33 AUTOMOVE=Y CLIENT=Y
HFS 604 ACTIVE RDWR 09/28/2010 L=59
NAME=[Link] 10.35.13 Q=0
PATH=/u/cs06
OWNER=SC32 AUTOMOVE=Y CLIENT=Y
HFS 602 ACTIVE RDWR 09/28/2010 L=58
NAME=[Link] 10.35.03 Q=0
PATH=/u/cs04
OWNER=SC31 AUTOMOVE=Y CLIENT=Y
HFS 411 ACTIVE RDWR 09/28/2010 L=52
NAME=[Link] 09.15.21 Q=0
PATH=/u/cs05
OWNER=SC30 AUTOMOVE=Y CLIENT=N
HFS 77 ACTIVE RDWR 09/28/2010 L=50
NAME=[Link] 07.05.17 Q=0
PATH=/SC32/var
OWNER=SC32 AUTOMOVE=U CLIENT=Y
HFS 76 ACTIVE RDWR 09/28/2010 L=49
NAME=[Link] 07.05.17 Q=0
PATH=/SC32/etc
OWNER=SC32 AUTOMOVE=U CLIENT=Y
HFS 75 ACTIVE RDWR 09/28/2010 L=48
NAME=[Link] 07.05.17 Q=0
PATH=/SC32
OWNER=SC32 AUTOMOVE=U CLIENT=Y
HFS 67 ACTIVE RDWR 09/28/2010 L=46
NAME=[Link] 07.04.23 Q=0
PATH=/SC31/wasbwconfig/bwcell/bwnodeb
OWNER=SC31 AUTOMOVE=U CLIENT=Y
HFS 60 ACTIVE RDWR 09/28/2010 L=45
NAME=[Link] 07.04.22 Q=0
PATH=/SC31/zWebSphereBW
OWNER=SC31 AUTOMOVE=U CLIENT=Y
HFS 47 ACTIVE RDWR 09/28/2010 L=43

100 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

NAME=[Link] 07.04.15 Q=0

PATH=/SC31/var
OWNER=SC31 AUTOMOVE=U CLIENT=Y
HFS 46 ACTIVE RDWR 09/28/2010 L=42
NAME=[Link] 07.04.15 Q=0
PATH=/SC31/etc
OWNER=SC31 AUTOMOVE=U CLIENT=Y
HFS 45 ACTIVE RDWR 09/28/2010 L=39
NAME=[Link] 07.04.15 Q=0
PATH=/SC31
OWNER=SC31 AUTOMOVE=U CLIENT=Y
HFS 37 ACTIVE RDWR 09/28/2010 L=38
NAME=[Link] 07.00.47 Q=0
PATH=/SC30/wasbwconfig/bwcell/bwnodea
OWNER=SC30 AUTOMOVE=U CLIENT=N
HFS 30 ACTIVE RDWR 09/28/2010 L=37
NAME=[Link] 07.00.46 Q=0
PATH=/SC30/wasbwconfig/bwcell/bwdmnode
OWNER=SC30 AUTOMOVE=U CLIENT=N
HFS 23 ACTIVE RDWR 09/28/2010 L=36
NAME=[Link] 07.00.45 Q=0
PATH=/SC30/zWebSphereBW
OWNER=SC30 AUTOMOVE=U CLIENT=N
HFS 15 ACTIVE RDWR 09/28/2010 L=31
NAME=[Link] 07.00.41 Q=0
PATH=/pp
OWNER=SC30 AUTOMOVE=Y CLIENT=N
HFS 5 ACTIVE RDWR 09/28/2010 L=18
NAME=[Link] 07.00.37 Q=0
PATH=/SC30/var
OWNER=SC30 AUTOMOVE=U CLIENT=N
HFS 4 ACTIVE RDWR 09/28/2010 L=17
NAME=[Link] 07.00.37 Q=0
PATH=/SC30/etc
OWNER=SC30 AUTOMOVE=U CLIENT=N
HFS 2 ACTIVE RDWR 09/28/2010 L=15
NAME=[Link] 07.00.37 Q=0
PATH=/SC30
OWNER=SC30 AUTOMOVE=U CLIENT=N
HFS 1 ACTIVE RDWR 09/28/2010 L=14
NAME=[Link] 07.00.37 Q=0
PATH=/
OWNER=SC30 AUTOMOVE=Y CLIENT=N

Example 3-24 shows some of the files defined in the active BPXPRM9A member for
comparative purposes only. We can compare the names defined in the active BPXPRM9A
member with the names that are actually active by using the D OMVS,F command.

Example 3-24 BPXPRM1A member

ROOT FILESYSTEM('[Link]')
TYPE(HFS)
AUTOMOVE
MODE(RDWR)

MOUNT FILESYSTEM('WTSCPLX5.&SYSNAME..[Link]')

Chapter 3. Base functions 101

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

MOUNTPOINT('/&SYSNAME.')
UNMOUNT
TYPE(HFS) MODE(RDWR)

MOUNT FILESYSTEM('OMVS.ZOSR1B.&SYSR1..ROOT')
MOUNTPOINT('/$VERSION')
AUTOMOVE
TYPE(HFS) MODE(RDWR)

MOUNT FILESYSTEM('OMVS.&SYSNAME..ETC')
MOUNTPOINT('/&SYSNAME./etc')
UNMOUNT
TYPE(HFS) MODE(RDWR)

MOUNT FILESYSTEM('OMVS.&SYSNAME..VAR')
MOUNTPOINT('/&SYSNAME./var')
UNMOUNT
TYPE(HFS) MODE(RDWR)

MOUNT FILESYSTEM('/&SYSNAME./TMP')
TYPE(TFS) MODE(RDWR)
MOUNTPOINT('/&SYSNAME./tmp')
PARM('-s 500')
UNMOUNT

MOUNT FILESYSTEM('/DEV')
MOUNTPOINT('/dev')
TYPE(TFS)
PARM('-s 10')
UNMOUNT

MOUNT FILESYSTEM('OMVS.ZOSR1B.&SYSR1..JAVA31V5')
MOUNTPOINT('/usr/lpp/java/J5.0')
TYPE(HFS) MODE(RDWR)
MOUNT FILESYSTEM('OMVS.ZOSR1B.&SYSR1..JAVA64V5')
MOUNTPOINT('/usr/lpp/java/J5.0_64')
TYPE(HFS) MODE(RDWR)
MOUNT FILESYSTEM('OMVS.ZOSR1B.&SYSR1..JAVA31V6')
MOUNTPOINT('/usr/lpp/java/J6.0')
TYPE(HFS) MODE(RDWR)
MOUNT FILESYSTEM('OMVS.ZOSR1B.&SYSR1..JAVA64V6')
MOUNTPOINT('/usr/lpp/java/J6.0_64')
TYPE(HFS) MODE(RDWR)
MOUNT FILESYSTEM('OMVS.ZOSR1B.&SYSR1..XML')
MOUNTPOINT('/usr/lpp/ixm')
TYPE(HFS) MODE(RDWR)

The OMVS processes can also be displayed within the z/OS UNIX environment, and similar
comparisons can be made. Use the shall environment to look at UNIX processes and to
execute the UNIX command ps -ef. This displays all processes and their environments in
forest or family tree format.

Refer to z/OS UNIX System Services Planning, GA22-7800 and z/OS UNIX System Services
User's Guide, SA22-7802 for detailed information about UNIX commands in the z/OS UNIX
environment.

102 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

Notice that in Example 3-25, the UNIX System Services after this initialization is running with
user ID BPXROOT. The reason for this is because RACF cannot map a UNIX System
Services UID to an MVS user ID correctly if there are multiple MVS user IDs defined with the
same UID. So RACF uses the last referenced MVS user ID.

Example 3-25 UNIX System Services processes display from the shell
1 @ SC30:/u/cs01>ps -ef
UID PID PPID C STIME TTY TIME CMD
BPXROOT 1 0 - Oct 25 ? 0:02 BPXPINPR
BPXROOT 33619971 1 - Oct 25 ? 0:00 CEAPSRVR
BPXROOT 16842756 1 - Oct 25 ? 0:08 HZSTKSCH
NET 50397189 1 - Oct 25 ? 1:29 ISTMGCEH
BPXROOT 50397190 1 - Oct 25 ? 0:02 BPXVCLNY
BPXROOT 65543 1 - Oct 25 ? 0:01 /usr/sbin/syslogd -c -i
BPXROOT 65543 1 - Oct 25 ? 0:01 /usr/sbin/syslogd -c -
-u -f /etc/[Link]
BPXROOT 65545 1 - Oct 25 ? 22:55 ERB3GMFC
BPXROOT 65546 1 - Oct 25 ? 1:14 EZBTCPIP
BPXROOT 16842763 1 - Oct 25 ? 1:14 EZACFALG
BPXROOT 65548 1 - Oct 25 ? 1:14 EZASASUB
BPXROOT 33619981 1 - Oct 25 ? 0:02 BPXVCMT
BPXROOT 16842766 1 - Oct 25 ? 0:01 IAZNJTCP
BPXROOT 16842768 1 - Oct 25 ? 0:00 /usr/sbin/inetd
/etc/[Link]
BPXROOT 65553 1 - Oct 25 ? 0:02 GFSCINIT
BPXROOT 33619986 1 - Oct 25 ? 0:00 PORTMAP
BPXROOT 33619987 1 - Oct 25 ? 0:01 IAZNJSTK
BPXROOT 16842772 1 - Oct 25 ? 0:31 EZBTZMST
BPXROOT 65557 1 - Oct 25 ? 0:00 RSHD
BPXROOT 16842774 1 - Oct 25 ? 0:00 IOAXTSRV
BPXROOT 65559 1 - Oct 25 ? 0:00 FTPD
BPXROOT 65560 1 - Oct 25 ? 0:00 FTPD
BPXROOT 33620011 1 - [Link] ? 0:05 EZBTCPIP
BPXROOT 33620012 1 - [Link] ? 0:05 EZACFALG
BPXROOT 65581 1 - [Link] ? 0:05 EZASASUB
BPXROOT 16842799 1 - [Link] ? 0:04 OMPROUTE
CS04 33620034 1 - [Link] ? 0:02 OMVS
CS04 65603 33620034 - [Link] ttyp0000 0:02 sh -L
BPXROOT 83951684 65603 - [Link] ttyp0000 0:00 sh
BPXROOT 16842821 83951684 - [Link] ttyp0000 0:00 ps -ef

Here are some typical UNIX commands:

 The mkdir/u/cso1 command creates the directory for the user mount point. The
permission bits would be set as specified in the etc/profile or $home/.profile.
 The ls -all command lists the files with their permission bits. From time to time you might
need to change the permission bits in the file.
 The chmod command is used to change the permission bits associated with files.
 The TSO/E interface can be used to work with zOS UNIX files. You can browse files using
the ISHELL PDSE interface or you can execute the obrowse command from the OMVS
shell environment. You can also edit files using the ISHELL tools, or you can use the oedit
command from the OMVS shell.

Chapter 3. Base functions 103

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

Note: Both obrowse and oedit are TSO commands. If you used telnet or rlogin to get to the
UNIX System Services shell, you have to use the cat command and the vi editor.

The ISHELL provides an ISPF look and feel. The OMVS shell provides a more UNIX or DOS
look and feel, and of course for real UNIX users there is the vi editor.

Starting z/OS Communications Server TCP/IP

Example 3-26 shows the startup of our TCP/IP stack.

Example 3-26 z/OS Communications Server TCP/IP startup

S TCPIPA
$HASP373 TCPIPA STARTED
IEE252I MEMBER CTIEZB00 FOUND IN [Link] 1
IEE252I MEMBER CTIIDS00 FOUND IN [Link]
IEE252I MEMBER CTINTA00 FOUND IN [Link]
EZZ0300I OPENED PROFILE FILE DD:PROFILE
EZZ0309I PROFILE PROCESSING BEGINNING FOR DD:PROFILE
EZZ0316I PROFILE PROCESSING COMPLETE FOR FILE DD:PROFILE 2
EZZ0641I IP FORWARDING NOFWDMULTIPATH SUPPORT IS ENABLED
EZZ0350I SYSPLEX ROUTING SUPPORT IS ENABLED 3
EZZ0624I DYNAMIC XCF DEFINITIONS ARE ENABLED 4
EZZ0338I TCP PORTS 1 THRU 1023 ARE NOT RESERVED
EZZ0338I UDP PORTS 1 THRU 1023 ARE NOT RESERVED
EZZ0613I TCPIPSTATISTICS IS DISABLED 5
EZZ4202I Z/OS UNIX - TCP/IP CONNECTION ESTABLISHED FOR TCPIPA 6
EZZ4340I INITIALIZATION COMPLETE FOR INTERFACE OSA2080I
EZZ4340I INITIALIZATION COMPLETE FOR INTERFACE OSA20A0I
EZZ4340I INITIALIZATION COMPLETE FOR INTERFACE OSA20C0I
EZZ4340I INITIALIZATION COMPLETE FOR INTERFACE OSA20D0I
EZB6473I TCP/IP STACK FUNCTIONS INITIALIZATION COMPLETE. 7
EZAIN11I ALL TCPIP SERVICES FOR PROC TCPIPA ARE AVAILABLE.
EZD1176I TCPIPA HAS SUCCESSFULLY JOINED THE TCP/IP SYSPLEX GROUP
EZD1214I INITIAL DYNAMIC VIPA PROCESSING HAS COMPLETED FOR TCPIP

In this example, the numbers correspond to the following information:

1. Shows how the member that defines CTRACE processing has been found (CTIEZB00).
This is discussed in 8.4.1, “Taking a component trace” on page 316.
2. Shows how the [Link] for the stack has been found and processed.
3. Sysplexrouting is enabled, so communication between z/OS TCP/IP is possible.
4. Dynamic XCFs are enabled (DYNAMICXCF parameter).
5. TCPIPSTATICTICS will not be generated.
6. Shows how the stack has been bound to UNIX System Services. It indicates that the
Common INET pre-router has successfully obtained a copy of the IP layer routing table
from the stack.
7. The stack (TCP/IP) is successfully initialized and READY FOR WORK.

Important: Because TCP/IP shares its Data Link Controls (DLCs) with VTAM, you must
restart TCP/IP if you restart VTAM.

104 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

Note: See 1.3.3, “Reusable address space ID” on page 6 if you want to run TCPIP in a
reusable address space ID.

3.6.2 Verifying TCP/IP configuration

After the configuration files are updated we verified the configuration and we restarted the
TCP/IP address space, ensuring that we saw the following message:
EZB6473I TCP/IP STACK FUNCTIONS INITIALIZATION COMPLETE

If the message is not displayed, the messages issued by the TCP/IP address space should
describe why TCP/IP did not start.

Displaying the TCP/IP configuration

To display the enabled features and operating characteristics of a TCP/IP stack, enter any of
the following commands:
 TSO/E command NETSTAT CONFIG
 MVS command D TCPIP,procname,NETSTAT,CONFIG
 UNIX shell command onetstat -f

Example 3-27 shows the output from the NETSTAT CONFIG display.

Example 3-27 NETSTAT CONFIG display

D TCPIP,TCPIPA,NETSTAT,CONFIG
EZD0101I NETSTAT CS V1R12 TCPIPA 442
TCP CONFIGURATION TABLE: 1
DEFAULTRCVBUFSIZE: 00131072 DEFAULTSNDBUFSIZE: 00131072
DEFLTMAXRCVBUFSIZE: 00262144 SOMAXCONN: 0000000010
MAXRETRANSMITTIME: 120.000 MINRETRANSMITTIME: 0.500
ROUNDTRIPGAIN: 0.125 VARIANCEGAIN: 0.250
VARIANCEMULTIPLIER: 2.000 MAXSEGLIFETIME: 30.000
DEFAULTKEEPALIVE: 00000120 DELAYACK: YES
RESTRICTLOWPORT: NO SENDGARBAGE: NO
TCPTIMESTAMP: YES FINWAIT2TIME: 600
TTLS: NO
UDP CONFIGURATION TABLE: 2
DEFAULTRCVBUFSIZE: 00065535 DEFAULTSNDBUFSIZE: 00065535
CHECKSUM: YES
RESTRICTLOWPORT: NO UDPQUEUELIMIT: NO
IP CONFIGURATION TABLE: 3
FORWARDING: YES TIMETOLIVE: 00064 RSMTIMEOUT: 00060
IPSECURITY: NO
ARPTIMEOUT: 01200 MAXRSMSIZE: 65535 FORMAT: LONG
IGREDIRECT: YES SYSPLXROUT: YES DOUBLENOP: NO
STOPCLAWER: NO SOURCEVIPA: YES
MULTIPATH: CONN PATHMTUDSC: YES DEVRTRYDUR: 0000000090
DYNAMICXCF: YES
IPADDR: [Link] SUBNET: [Link] METRIC: 08
SECCLASS: 255
QDIOACCEL: YES QDIOACCELPRIORITY: 1
IQDIOROUTE: N/A
TCPSTACKSRCVIPA: NO
IPV6 CONFIGURATION TABLE:

Chapter 3. Base functions 105

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

FORWARDING: YES HOPLIMIT: 00255 IGREDIRECT: NO

SOURCEVIPA: NO MULTIPATH: NO ICMPERRLIM: 00003
IGRTRHOPLIMIT: NO
IPSECURITY: NO
DYNAMICXCF: NO
TCPSTACKSRCVIPA: NO
TEMPADDRESSES: NO
SMF PARAMETERS: 4
TYPE 118:
TCPINIT: 00 TCPTERM: 00 FTPCLIENT: 00
TN3270CLIENT: 00 TCPIPSTATS: 00
TYPE 119:
TCPINIT: YES TCPTERM: YES FTPCLIENT: YES
TCPIPSTATS: YES IFSTATS: NO PORTSTATS: NO
STACK: NO UDPTERM: NO TN3270CLIENT: YES
IPSECURITY: NO PROFILE: NO DVIPA: NO
GLOBAL CONFIGURATION INFORMATION: 5
TCPIPSTATS: NO ECSALIMIT: 0000000K POOLLIMIT: 0000000K
MLSCHKTERM: NO XCFGRPID: 21 IQDVLANID: 21
SEGOFFLOAD: YES SYSPLEXWLMPOLL: 060 MAXRECS: 100
EXPLICITBINDPORTRANGE: 00000-00000 IQDMULTIWRITE: YES
WLMPRIORITYQ: NO
SYSPLEX MONITOR:
TIMERSECS: 0060 RECOVERY: YES DELAYJOIN: YES AUTOREJOIN: NO
MONINTF: NO DYNROUTE: NO JOIN: YES
ZIIP:
IPSECURITY: NO IQDIOMULTIWRITE: YES
NETWORK MONITOR CONFIGURATION INFORMATION: 6
PKTTRCSRV: NO TCPCNNSRV: NO NTASRV: NO
SMFSRV: YES
IPSECURITY: YES PROFILE: YES CSSMTP: YES CSMAIL: NO DVIPA: YES
AUTOLOG CONFIGURATION INFORMATION: WAIT TIME: 0300
PROCNAME: FTPDA JOBNAME: FTPDA1
PARMSTRING:
DELAYSTART: NO
PROCNAME: OMPA JOBNAME: OMPA
PARMSTRING:
DELAYSTART: NO
PROCNAME: IOASRV JOBNAME: IOASRV
PARMSTRING:
DELAYSTART: NO
END OF THE REPORT

Parameters such as SOURCEVIPA can be either ENABLED or DISABLED. A value of 01 in

the NETSTAT CONFIG display means it is ENABLED. In this example, the numbers
correspond to the following information:
1. The settings in effect in the TCPCONFIG parameters.
2. The settings for the UDPCONFIG parameters.
3. The settings in effect in the IPCONFIG parameters.
4. The settings in effect for SMFCONFIG.
5. The settings in effect for GLOBALCONFIG.
6. The setting in effect for Network Monitoring Information.

106 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

Displaying the status of devices

You can display the status of devices by using the MVS display command TSO NETSTAT, as
shown in Example 3-28 on page 107, or UNIX onetstat -d.

Example 3-28 Results of device display

D TCPIP,TCPIPA,N,DE
........................................................................ Lines
deleted
INTFNAME: OSA2080I INTFTYPE: IPAQENET INTFSTATUS: READY 1
PORTNAME: OSA2080 DATAPATH: 2082 DATAPATHSTATUS: READY
CHPIDTYPE: OSD
SPEED: 0000001000
IPBROADCASTCAPABILITY: NO
VMACADDR: 02000C776873 VMACORIGIN: OSA VMACROUTER: LOCAL
ARPOFFLOAD: YES 2 ARPOFFLOADINFO: YES
CFGMTU: 1492 ACTMTU: 1492
IPADDR: [Link]/24
VLANID: 10 3 VLANPRIORITY: DISABLED
DYNVLANREGCFG: NO DYNVLANREGCAP: YES
READSTORAGE: GLOBAL (4096K)
INBPERF: BALANCED
CHECKSUMOFFLOAD: YES SEGMENTATIONOFFLOAD: YES
SECCLASS: 255 MONSYSPLEX: NO
ISOLATE: NO OPTLATENCYMODE: NO
MULTICAST SPECIFIC:
MULTICAST CAPABILITY: YES
GROUP REFCNT SRCFLTMD
----- ------ --------
[Link] 0000000001 EXCLUDE
SRCADDR: NONE
........................................................................ Lines
deleted

In this example, the numbers correspond to the following information:

1. Indicates the overall status of the OSA interface OSA2080I: READY. If this status is not
READY, verify that the VTAM Major node is active. You can do this by using the VTAM
command D NET,TRL.
2. The OFFLOAD feature is enabled.
3. The VLAN ID defined on the LINK statement in the PROFILE data set.

Example 3-29 Results of the TRLE display

D NET,TRL,TRLE=OSA2080T
IST075I NAME = OSA2080T, TYPE = TRLE 337
IST1954I TRL MAJOR NODE = OSA2080
IST486I STATUS= ACTIV, DESIRED STATE= ACTIV 1
IST087I TYPE = LEASED , CONTROL = MPC , HPDT = YES
IST1715I MPCLEVEL = QDIO MPCUSAGE = SHARE
IST2263I PORTNAME = OSA2080 PORTNUM = 0 OSA CODE LEVEL = 000C
IST2337I CHPID TYPE = OSD CHPID = 02
IST1577I HEADER SIZE = 4096 DATA SIZE = 0 STORAGE = ***NA***
IST1221I WRITE DEV = 2081 STATUS = ACTIVE STATE = ONLINE 2
IST1577I HEADER SIZE = 4092 DATA SIZE = 0 STORAGE = ***NA***
IST1221I READ DEV = 2080 STATUS = ACTIVE STATE = ONLINE 2

Chapter 3. Base functions 107

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

IST924I -------------------------------------------------------------
IST1221I DATA DEV = 2082 STATUS = ACTIVE STATE = N/A 3
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST1717I ULPID = TCPIPA
IST2309I ACCELERATED ROUTING ENABLED
IST2331I QUEUE QUEUE READ
IST2332I ID TYPE STORAGE
IST2205I ------ -------- ---------------
IST2333I RD/1 PRIMARY 4.0M(64 SBALS)
IST2305I NUMBER OF DISCARDED INBOUND READ BUFFERS = 0
IST1757I PRIORITY1: UNCONGESTED PRIORITY2: UNCONGESTED
IST1757I PRIORITY3: UNCONGESTED PRIORITY4: UNCONGESTED
IST2190I DEVICEID PARAMETER FOR OSAENTA TRACE COMMAND = 01-01-00-02
IST1801I UNITS OF WORK FOR NCB AT ADDRESS X'0F30D010'
IST1802I P1 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P2 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P3 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P4 CURRENT = 0 AVERAGE = 1 MAXIMUM = 2
IST924I -----------------------------------------------------------
IST1221I DATA DEV = 2083 STATUS = ACTIVE STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST1717I ULPID = TCPIPC
IST2309I ACCELERATED ROUTING ENABLED
IST2331I QUEUE QUEUE READ
IST2332I ID TYPE STORAGE
IST2205I ------ -------- ---------------
IST2333I RD/1 PRIMARY 4.0M(64 SBALS)
IST2305I NUMBER OF DISCARDED INBOUND READ BUFFERS = 0
IST1757I PRIORITY1: UNCONGESTED PRIORITY2: UNCONGESTED
IST1757I PRIORITY3: UNCONGESTED PRIORITY4: UNCONGESTED
IST2190I DEVICEID PARAMETER FOR OSAENTA TRACE COMMAND = 01-01-00-03
IST1801I UNITS OF WORK FOR NCB AT ADDRESS X'0FCF0010'
IST1802I P1 CURRENT = 0 AVERAGE = 1 MAXIMUM = 2
IST1802I P2 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P3 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P4 CURRENT = 0 AVERAGE = 1 MAXIMUM = 1
IST924I -----------------------------------------------------------
IST1221I DATA DEV = 2084 STATUS = RESET STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I -----------------------------------------------------------
IST1221I DATA DEV = 2085 STATUS = RESET STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I -------------------------------------------------
IST1221I DATA DEV = 2086 STATUS = RESET STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I -------------------------------------------------
IST1221I DATA DEV = 2087 STATUS = RESET STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I -------------------------------------------------
IST314I END

In this example, the numbers correspond to the following information:

1. The Major node is ACTIVE and ONLINE.
2. The READ and WRITE channels are ACTIVE and ONLINE.

108 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

3. The data channel is also ACTIVE.

Displaying storage usage

The z/OS Communications Server uses the Common Storage Manager (CSM) to manage
storage pools. The recommendation is to increase storage allocations by a minimum of
20 MB for TCP/IP in the CSA definition in IEASYSxx and the FIXED and ECSA definitions in
IVTPRMxx.

Check your storage utilization to ensure that you made the correct allocations. Storage usage
can also be controlled using the GLOBALCONFIG ECSALIMIT and GLOBALCONFIG
POOLLIMIT parameters. ECSALIMIT allows you to specify the maximum amount of extended
common service area (ECSA) that TCP/IP can use. POOLLIMIT allows you to specify the
maximum amount of authorized private storage that TCP/IP can use within the TCP/IP
address space.
The “DISPLAY TCPIP,tcpproc,STOR” command display and the NMI storage statistics report
are enhanced to distinguish the common storage that is used by dynamic LPA for load
modules from the ECSA storage that is used for control blocks.

You can also use the MVS command D TCPIP,tcpproc,STOR to display TCP/IP storage
usage, as illustrated in Example 3-30.

Example 3-30 Results of storage display

D TCPIP,TCPIPA,STOR
EZZ8453I TCPIP STORAGE
EZZ8454I TCPIPA STORAGE CURRENT MAXIMUM LIMIT
EZZ8455I TCPIPA ECSA 2850K 3270K NOLIMIT
EZZ8455I TCPIPA POOL 9035K 9041K NOLIMIT
EZZ8455I TCPIPA 64-BIT COMMON 1M 1M NOLIMIT
EZZ8455I TCPIPA ECSA MODULES 7451K 7451K NOLIMIT
EZZ8459I DISPLAY TCPIP STOR COMPLETED SUCCESSFULLY

Verifying [Link] statement values in z/OS

To display which [Link] statement values are being used and where they are being
obtained from, use trace resolver output. You can obtain trace resolver output at your TSO
screen by issuing the following TSO commands:
alloc f(systcpt) dsn(*)
READY
netstat up
READY
free f(systcpt)
READY

Tip: When directing trace resolver output to a TSO terminal, define the screen size to be
only 80 columns wide. Otherwise, trace output is difficult to read.

Verifying [Link] statement values in z/OS UNIX

To display which [Link] statement values are being used and where they are being
obtained from, use trace resolver output. You can obtain trace resolver output by issuing the
following z/OS UNIX shell commands:
#
export RESOLVER_TRACE=stdout
#

Chapter 3. Base functions 109

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

onetstat –u
#
set -A RESOLVER_TRACE

Verifying [Link]
Many configuration values specified within the [Link] file can be verified with the
TSO NETSTAT or z/OS UNIX onetstat commands. To verify the physical network and
hardware definitions, use the D TCPIP,,N,DEV, NETSTAT DEVLINKS or onetstat -d
commands. To see operating characteristics, use z/OS displays, namely NETSTAT CONFIG
or onetstat -f.

Verifying interfaces with PING and TRACERTE

PING and TRACERTE can be used in the TSO environment to verify adapters or interfaces
attached to the z/OS host. In the z/OS UNIX environment, oping and otracert can be used
with identical results. For information about the syntax and output of the commands, refer to
z/OS Communications Server: IP System Administrator’s Commands, SC31-8781. Given that
your [Link] file contains the interfaces of your installation and that the [Link]
file contains the correct TCPIPJOBNAME, the TCP/IP address space is configured and you
can go on to configuring routes, servers, and so on.

3.7 Reconfiguring the system with z/OS commands

The z/OS Communications Server provides the VARY OBEYFILE command to change the
running TCP/IP configuration dynamically. This command replaces the OBEYFILE TSO
command.

The VARY command is an z/OS Console command. It allows you to add, delete, or
completely redefine all devices dynamically, as well as change TN3270 parameters, routing,
and almost any TCP/IP parameter in the profile. These changes are in effect until the TCP/IP
started task is started again, or another VARY OBEYFILE command overrides them.

Authorization is through the user’s RACF profile containing the [Link]

definition. There is no OBEY statement in the [Link], which in earlier MVS TCP/IP
implementations provided authorization.

110 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

For further details about the VARY OBEYFILE command, see z/OS CS: IP System
Administrator’s Commands, SC31-8781. For more information about RACF definitions, see
IBM z/OS Communications Server TCP/IP Implementation Volume 2: Standard Applications,
SG24-7897.

3.7.1 Deleting a device and adding or changing a device

You can use the OBEYFILE command to reconfigure the devices being used by the stack.
Reconfiguration could be the deletion of existing devices, the addition of new devices, or the
redefinition of an existing device. The syntax of the statements for OBEYFILE processing is
the same as that being used in [Link].

Device reconfiguration is a three-step process:

1. Stop the device with an z/OS console command (VARY STOP) or with a VARY OBEYFILE
that names a data set in which the STOP command is defined.
2. Activate an OBEYFILE that deletes the links and the devices.
3. Activate an OBEYFILE that adds the new or changed links and devices and then starts
them.

Deleting and adding back a device

If you want to delete a device, then the order of the steps that you perform is important. The
DELETE statement in [Link] allows you to remove LINK, DEVICE, and PORT or
PORTRANGE definitions. You must delete a resource that is defined using the INTERFACE
statement using the DELETE parameter.

The sequence for deleting and adding back a resources that was defined using the
INTERFACE statement is as follows:
1. Stop the device.
2. Delete the interface.
3. Add the new or changed interface.
4. Start the device.

Use the following steps to delete and add back a resource that was defined using the
DEVICE, LINK, or HOME statements:
1. Stop the device.
2. Remove the HOME address by excluding it from the full stack’s HOME list.
3. Delete the link.
4. Delete the device.
5. Add the new or changed device.
6. Add the new or changed link.
7. Add the HOME statements for the full stack.
8. Add the full gateway statements for the stack if you are using static routing.
9. Start the device.

3.7.2 Modifying a device

In this example, we want to change the IP address of our OSA-Express interface/device
OSA2080I from [Link] to [Link]. This process involves stopping and deleting the
current interface or device, and then redefining and restarting it.

Chapter 3. Base functions 111

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

Note: You can delete and redefine OSA-Express resources defined with either the
INTERFACE statement or the DEVICE, LINK, or HOME statements by following the same
procedure but by creating different OBEYFILE commands. Because the INTERFACE
statement is now the preferred way of defining OSA devices, we use that procedure first in
the following examples.

Example 3-31 and Example 3-32 show the interface OSA2080I, or link OSA2080L, active
with associated IP address [Link].

Example 3-31 Displays netstat device before deletion (for INTERFACE defined)
D TCPIP,TCPIPA,N,DE
........................................................................ Lines
deleted
INTFNAME: OSA2080I INTFTYPE: IPAQENET INTFSTATUS: READY
PORTNAME: OSA2080 DATAPATH: 2082 DATAPATHSTATUS: READY
CHPIDTYPE: OSD
SPEED: 0000001000
IPBROADCASTCAPABILITY: NO
VMACADDR: 02000C776873 VMACORIGIN: OSA VMACROUTER: LOCAL
ARPOFFLOAD: YES ARPOFFLOADINFO: YES
CFGMTU: 1492 ACTMTU: 1492
IPADDR: [Link]/24
VLANID: 10 VLANPRIORITY: DISABLED
DYNVLANREGCFG: NO DYNVLANREGCAP: YES
READSTORAGE: GLOBAL (4096K)
INBPERF: BALANCED
CHECKSUMOFFLOAD: YES SEGMENTATIONOFFLOAD: YES
SECCLASS: 255 MONSYSPLEX: NO
ISOLATE: NO OPTLATENCYMODE: NO
MULTICAST SPECIFIC:
MULTICAST CAPABILITY: YES
GROUP REFCNT SRCFLTMD
----- ------ --------
[Link] 0000000001 EXCLUDE
SRCADDR: NONE
........................................................................ Lines
deleted

Example 3-32 Display netstat home before deletion (for DEVICE/LINK/HOME defined)
D TCPIP,TCPIPA,N,HO
........................................................................ Lines
deleted
INTFNAME: OSA2080I
ADDRESS: [Link]
FLAGS:
........................................................................ Lines
deleted

Notice the address of OSA2080I ([Link]). We needed to change this in the running system
by stopping, deleting, redefining, and adding back the OSA-Express device and link and
home address.

112 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

Because the STOP command is executed as the last statement within an OBEYFILE
regardless of its position within the file, you cannot execute STOP and DELETE in one step.
Trying to do so will result in the error messages. You should stop the interface or device with
console command, as shown in Example on page [Link] to stop the interface or
device

V TCPIP,TCPIPA,STOP,OSA2080I
EZZ0060I PROCESSING COMMAND: VARY TCPIP,TCPIPA,STOP,OSA2080I
EZZ0053I COMMAND VARY STOP COMPLETED SUCCESSFULLY
EZZ4341I DEACTIVATION COMPLETE FOR INTERFACE OSA2080I
EZZ4315I DEACTIVATION COMPLETE FOR DEVICE OSA2080I

Then, delete it from the stack, as shown in Example 3-33.

Example 3-33 Command to delete the interface or device

V TCPIP,TCPIPA,O,[Link](OBDELINT)
EZZ0060I PROCESSING COMMAND: VARY TCPIP,TCPIPA,O,[Link](OBDELINT)
EZZ0300I OPENED OBEYFILE FILE '[Link](OBDELINT)'
EZZ0309I PROFILE PROCESSING BEGINNING FOR '[Link](OBDELINT)'
ZZ0316I PROFILE PROCESSING COMPLETE FOR FILE '[Link](OBDELINT)'
EZZ0053I COMMAND VARY OBEY COMPLETED SUCCESSFULLY

Enter either the NETSTAT DEV or NETSTAT HOME commands to check that the device you
wanted to delete is missing from the list.

Example 3-34 and Example 3-35 show the statements necessary to delete the device.

Example 3-34 OBEYFILE member to delete the device OSA2080I (INTERFACE defined)
INTERFACE OSA2080I
DELETE

Example 3-35 OBEYFILE member to delete the device OSA2080 (DEVICE/LINK/HOME defined)
HOME
[Link] VIPA1L
[Link] VIPA2L
;;;[Link] OSA2080I
[Link] OSA20C0I
[Link] OSA20E0I
[Link] OSA20A0I
[Link] IUTIQDF4L
[Link] IUTIQDF5L
[Link] IUTIQDF6L
;
DELETE LINK OSA2080I
DELETE DEVICE OSA2080

Note: With DEVICE/LINK/HOME defined devices, you have to provide the complete
HOME definition that excludes the device that you want to delete, because the new HOME
statement replaces the existing one. This step is not necessary with devices defined using
the INTERFACE statement.

Chapter 3. Base functions 113

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

Then, add either the interface or the device and link back with the changed address
definition 3, as shown in Example 3-36 and Example 3-37.

Example 3-36 OBEYFILE member to add the interface

INTERFACE OSA2080I
DEFINE IPAQENET
PORTNAME OSA2080
IPADDR [Link]/24 3;
START OSA2080I

Example 3-37 OBEYFILE member to add the device (ADDA30)

DEVICE OSA2080 MPCIPA
LINK OSA2080I IPAQENET OSA2080 VLANID 10
;
HOME
[Link] VIPA1L
[Link] VIPA2L
[Link] OSA2080I 3
[Link] OSA20C0I
[Link] OSA20E0I
[Link] OSA20A0I
[Link] IUTIQDF4L
[Link] IUTIQDF5L
[Link] IUTIQDF6L

Issue the command shown in Example 3-38 to add the device and link associated with its own
IP address.

Example 3-38 Adding the device and link

V TCPIP,TCPIPA,O,[Link](OBADDINT)
EZZ0060I PROCESSING COMMAND: VARY TCPIP,TCPIPA,O,[Link](OBADDINT)
EZZ0300I OPENED OBEYFILE FILE '[Link](OBADDINT)'
EZZ0309I PROFILE PROCESSING BEGINNING FOR '[Link](OBADDINT)'
ZZ0316I PROFILE PROCESSING COMPLETE FOR FILE '[Link](OBADDINT)'
EZZ0053I COMMAND VARY OBEY COMPLETED SUCCESSFULLY

Then, follow with a display to verify the addition to the stack, as shown in Example 3-39.

Example 3-39 Display with OSA2080 using a new address

D TCPIP,TCPIPE,N,HOME
........................................................................ Lines
deleted
LINKNAME: OSA2080I
ADDRESS: [Link]
FLAGS:
........................................................................ Lines
deleted

114 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:56 am 7896 Base [Link]

3.8 Job log versus syslog as diagnosis tool

In the past, the TCP/IP job log was used to detect problems. Most procedures now send
messages to the syslogd daemon or the MVS console log. Refer to IBM z/OS
Communications Server TCP/IP Implementation Volume 2: Standard Applications,
SG24-7897 for more information about the syslog daemon. Individual server documentation
also provides information about diagnosis.

3.9 Message types: Where to find them

For an explanation of z/OS UNIX and TCP/IP messages or SNA sense codes, refer to the
publications listed in Table 3-1.

Table 3-1 Messages and return code publications

Message type Publication

Messages with prefix BPX z/OS MVS System Messages, Vol 3 (ASB-BPX), SA22-7633

Messages with prefix EZA For Communications Server for z/OS IP, refer to z/OS Communications Server:
IP Messages Volume 1 (EZA), SC31-8783

Messages with prefix EZB For Communications Server for z/OS IP, refer to z/OS Communications Server:
IP Messages Volume 2 (EZB, EZD), SC31-8784

Messages with prefix EZY For Communications Server for z/OS IP, refer to z/OS Communications Server:
IP Messages Volume 3 (EZY), SC31-8785

Messages with prefix EZZ and SNM For Communications Server for z/OS IP, refer to z/OS Communications Server:
IP Messages Volume 4 (EZZ, SNM), SC31-8786

Messages with prefix FOMC, z/OS UNIX System Services Messages and Codes, SA22-7807
FOMM, FOMO, FSUC, and FSUM

Eight-digit SNA sense codes and z/OS Communications Server: IP and SNA Codes, SC31-8791
DLC codes

UNIX System Services return z/OS UNIX System Services Messages and Codes, SA22-7807
codes and reason codes

3.10 Additional information

When you install and customize the Communications Server for z/OS IP, it can be very
helpful to have the following documentation and product publications available:
 Implementation and migration plans, fallback plans, and test plans that you have created
and customized for your environment
 Printouts of procedures and data sets that you will be using for the implementation
 z/OS Program Directory, Program Number 5694-A01, GI10-0670
 z/OS XL C/C++ Run-Time Library Reference, SA22-7821
 z/OS Migration, GA22-7499
 z/OS Communications Server: IP Configuration Guide, SC31-8775
 z/OS Communications Server: IP Configuration Reference, SC31-8776

Chapter 3. Base functions 115

7896 Base [Link] Draft Document for Review March 21, 2011 7:56 am

 z/OS Communications Server: IP Messages Volume 1 (EZA), SC31-8783

 z/OS Communications Server: IP Messages Volume 2 (EZB, EZD), SC31-8784
 z/OS Communications Server: IP Messages Volume 3 (EZY), SC31-8785
 z/OS Communications Server: IP Messages Volume 4 (EZZ, SNM), SC31-8786
 z/OS Communications Server: IP and SNA Codes, SC31-8791
 OSA-Express Customer’s Guide and Reference, SA22-7935
 z/OS UNIX System Services Planning, GA22-7800
 z/OS UNIX System Services User’s Guide, SA22-7801
 z/OS UNIX System Services Messages and Codes, SA22-7807
 z/OS MVS System Messages, Vol 1 (ABA-AOM), SA22-7631
 z/OS MVS System Messages, Vol 2 (ARC-ASA), SA22-7632
 z/OS MVS System Messages, Vol 3 (ASB-BPX), SA22-7633
 z/OS MVS System Messages, Vol 4 (CBD-DMO), SA22-7634
 z/OS MVS System Messages, Vol 5 (EDG-GFS), SA22-7635
 z/OS MVS System Messages, Vol 6 (GOS-IEA), SA22-7636
 z/OS MVS System Messages, Vol 7 (IEB-IEE), SA22-7637
 z/OS MVS System Messages, Vol 8 (IEF-IGD), SA22-7638
 z/OS MVS System Messages, Vol 9 (IGF-IWM), SA22-7639
 z/OS MVS System Messages, Vol 10 (IXC-IZP), SA22-7640

116 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Chapter 4. Connectivity
In today’s networked world, the usability of a computer system is defined by its connectivity.
While there are many ways for TCP/IP traffic to reach IBM mainframes, this chapter discusses
the most commonly used and the most dynamic types of mainframe connectivity.

Detailed topics regarding these interfaces are provided, including useful implementation
information, design scenarios, and setup examples.

This chapter discusses the following topics.

Section Topic
4.1, “What is connectivity” on Network connectivity options supported by z/OS and
page 118 Communications Server TCP/IP, IBM System z servers, and
key characteristics of VLAN implementation
4.2, “Recommended interfaces” on Recommended interfaces supported by System z hardware
page 120 and z/OS Communications Server
4.3, “Connectivity for the z/OS Basic implementation information for z/OS and
environment” on page 136 Communications Server when connecting to the immediate
LAN environment
4.4, “OSA-Express QDIO Configuration examples, with dependencies, considerations,
connectivity” on page 139 and our recommendations for an OSA-Express interface
4.5, “OSA-Express QDIO Configuration examples, with dependencies, considerations,
connectivity with Connection and our recommendations for isolating traffic across a shared
Isolation” on page 156 OSA port
4.6, “HiperSockets connectivity” on Configuration examples, with dependencies, considerations,
page 180 and our recommendations for a HiperSockets interface
4.7, “Dynamic XCF connectivity” on Configuration examples, with dependencies, considerations,
page 188 and our recommendations for a dynamic XCF interface
4.8, “Controlling and activating Commands to start and stop devices, as well as activate
devices” on page 195 modified device definitions
4.9, “Problem determination” on How to determine why certain connectivity options are not
page 197 working

© Copyright IBM Corp. 2011. All rights reserved. 117

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

4.1 What is connectivity

Connectivity is the pipeline through which data is exchanged between clients and servers
through physical and logical communication interfaces and the network. IBM System z
servers provide a wide range of interface options for connecting your z/OS system to an IP
network or to another IP host. Some interfaces offer point-to-point or point-to-multipoint
connectivity. Others support Local Area Network (LAN) connectivity.

Figure 4-1 depicts the physical interfaces (and device types) provided by System z servers.
The physical network interface is enabled through z/OS Communications Server (TCP/IP)
definitions.

HYPERchannel
Sysplex
MPCPTP (XCF) CF
System p Environment
CLAW
or OEM
Servers MPCPTP CDLC (NCP)

CTC (FICON 3745/3746

or ESCON) System z
Servers
Token Ring

LCS/MPCIPA
(1000BASE-T)
LCS/MPCIPA (FENET)
ATM (Native) (GbE)
MPCIPA
(10 GbE LR)

ATM (LANE)
LCS/MPCIPA
ATM Ethernet
Network

Figure 4-1 System z: physical interfaces

4.1.1 System z network connectivity

System z network connectivity is handled by the physical and logical interfaces to enable the
transport of IP datagrams. Using the OSI model as an example, it spans Layer 1 (physical
layer) and Layer 2 (data link control layer). The z/OS Communications Server supports
several types of interfaces connecting to different networking environments. These
environments vary from point-to-point connections (such as MPCPTP, CTC, and CLAW), to
LAN connections (such as LCS and MPCIPA).

118 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

The supported IPv4 interfaces are listed in Table 4-1.

Table 4-1 System z network interfaces

Interface type Attachment type Protocol type Description

Asynchronous ATM Native mode through ATM network Enables TCP/IP to send data to an
transfer mode OSA-Express ATM network using an OSA-Express
(ATM) ATM adapter.

Channel data link Network Control Program Point-to-point ESCON® attachments can be used
control (CDLC) through 3745/3746 to provide native IP transport
between the 3746 IP and host
systems running the z/OS
Communications Server.

Common link IBM System p® Point-to-point Provides access from IBM System p
access to Channel attached routers Point-to-Multipoint server directly to a TCP/IP stack
workstation over a channel.
(CLAW) Can also be used to provide
connectivity to other vendor
platforms.

Channel-to- FICON/ESCON channel Point-to-point Provides access to TCP/IP hosts by

channel (CTC) way of a CTC connection
established over a FICON or
ESCON channel.

HYPERchannel Series A devices Point-to-Multipoint Provides access to TCP/IP hosts by

way of a series A devices and series
DX devices that function as series A
devices.

LAN Channel OSA-Express: LAN: A variety of channel adapters

Station (LCS)  1000BASE-T  IEEE802.3 support a protocol called the LCS.
 Fast Ethernet  IEEE802.3 The most common are
 Token Ring  IEEE802.5 OSA-Express features configured
 ATM Native and LAN  ATM network as CHPID type OSE. LCS supports
Emulation native IP flows on z/OS. CHPID type
OSE also supports the Link Station
Architecture (LSA) protocol, which
supports native SNA flows for VTAM
on z/OS.

MultiPath Channel HiperSocketsa Internal LAN Provides access to TCP/IP hosts,

IP Assist OSA-Express: LAN: using OSA-Express in Queued
(MPCIPA)  10 Gigabit Ethernet  IEEE802.3 Direct I/O (QDIO) mode and
 Gigabit Ethernet  IEEE802.3 HiperSockets using the internal
 1000BASE-T  IEEE802.3 Queued Direct I/O (iQDIO).
 Fast Ethernet  IEEE802.3
 Token Ring  IEEE802.5
 ATM LAN Emulation  ATM network

MultiPath Channel IUTSAMEH (XCF link) Point-to-point Provides access to directly connect
Point-to-Point z/OS hosts or z/OS LPARs, or by
(MPCPTP) configuring it to utilize Coupling
Facility links (if it is part of a sysplex).

SAMEHOST (Data SNALINK LU0 Point-to-point Enables communication between

Link Control) SNALINK LU6.2 Point-to-point z/OS Communications Server IP
X25NPSI X.25 network and other servers running on the
same MVS image.

Chapter 4. Connectivity 119

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

a. Can also be used in conjunction with DYNAMICXCF

For further information about these protocols, refer to z/OS Communications Server: IP
Configuration Reference, SC31-8776.

4.2 Recommended interfaces

This section discusses the recommended interfaces supported by System z hardware and
z/OS Communications Server. We highly recommend their use because they deliver the best
throughput and performance, as well as offer the most flexibility and highest levels of
availability. These interfaces include:
 OSA-Express
 HiperSockets
 Dynamic Cross-system Coupling Facility (dynamic XCF)

4.2.1 High-bandwidth and high-speed networking technologies

z/OS Communications Server supports high-bandwidth and high-speed networking
technologies provided by OSA-Express and HiperSockets.
 The OSA-Express features comply with the most commonly used IEEE standards, used in
LAN environments.
 HiperSockets is used for transporting IP traffic between TCP/IP stacks running in logical
partitions (LPARs) within a System z server at memory speed.

Both interfaces use the System z I/O architecture called queued direct input/output (QDIO).

QDIO is a highly efficient data transfer mechanism that satisfies the increasing volume of
applications and bandwidth demands. It dramatically reduces system overhead, and
improves throughput by using system memory queues and a signaling protocol to directly
exchange data between the OSA-Express microprocessor and network software, using data
queues in main memory and utilizing Direct Memory Access (DMA).

The components that make up QDIO are Direct Memory Access (DMA), Priority Queueing,
dynamic OSA Address Table building, LPAR-to-LPAR communication, and Internet Protocol
(IP) Assist functions.

HiperSockets implementation is based on the OSA-Express QDIO protocol, hence the name
internal QDIO (iQDIO). The System z microcode for HiperSockets emulates the link control
layer of an OSA-Express QDIO interface. The communication is through system memory of
the server using I/O queues. IP traffic is transferred at memory speeds between LPARs,
eliminating the I/O subsystem overhead and external network delays.

Recommendation: Some OSA-Express features also support LCS (known as non-QDIO

mode). However, we recommend the use of QDIO mode in conjunction with the
OSA-Express Ethernet features wherever possible.

With QDIO, I/O interrupts and I/O path-lengths are minimized, resulting in significantly
improved performance versus non-QDIO mode, reduction of System Assist Processor
(SAP) utilization, improved response time, and server cycle reduction.

120 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

z/OS Communications Server can only transport IP traffic over OSA-Express in QDIO mode
and HiperSockets. However, SNA can be transported over IP connections using
encapsulation technologies such as Enterprise Extender (EE) and TN3270.

For more information about EE, refer to Enterprise Extender Implementation Guide,
SG24-7359. For TN3270 details, refer to IBM z/OS V1R12 Communications Server TCP/IP
Implementation Volume 2: Standard Applications, SG24-7897.

4.2.2 OSA-Express (MPCIPA)

As mentioned, the OSA-Express can use the I/O architecture called QDIO when defined as
channel type (CHPID) OSD. QDIO provides a highly optimized data transfer interface that
eliminates the need for channel command words (CCWs) and interrupts during data
transmission, resulting in accelerated TCP/IP packet transmission. This is done by providing
a data queue between TCP/IP and the OSA-Express. OSA-Express uses a direct memory
access (DMA) protocol to transfer the data to and from the TCP/IP stack.

The OSA-Express also provides the offloading of IP processing from the host, which is called
IP assist (IPA). With IP assist, the OSA-Express offloads the following processing from the
host:
 All MAC handling is done in the card. The TCP/IP stack no longer has to fully format the
datagrams for LAN-specific media.
 ARP processing for identifying the physical address.
 Packet filtering, screening, and discarding of LAN packets.

Table 4-2 lists the OSA-Express3, OSA-Express2, and OSA-Express Ethernet features that
are available on the System z servers. The mode of operation in which they can run and the
necessary TCP/IP and VTAM definition types are included.

Table 4-2 OSA-Express features to support native TCP/IP data flows

OSA-Express feature Operation TCP/IP TCP/IP link type VTAM
mode device type definitions

10 GbE LR QDIO MPCIPA IPAQENET TRLE

GbE QDIO MPCIPA IPAQENET TRLE

1000BASE-T QDIO MPCIPA IPAQENET TRLE

Non-QDIO LCS ETHERNet, 802.3, or N/A

ETHEROR802.3

Note: The 1000Base-T feature can also support native SNA data flows to VTAM when
configured in Non-QDIO mode. The VTAM device type protocol is called Link Station
Architecture (LSA).

OSA-Express QDIO IPv4 address registration

The Dynamic OSA Address Table (OAT) contains certain active IP addresses displayed in the
HOME list of the TCP/IP stack; the addresses are downloaded into the OSA-Express when
the interface is started.

Chapter 4. Connectivity 121

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

z/OS Communications Server registers IPv4 addresses in the OSA OAT for two distinct
purposes:
 Inbound routing
 ARP offload

Several factors contribute to the types of IPv4 addresses in a TCP/IP stack that are registered
in the OAT. These factors are summarized as follows:
 Which type of definition statement defines the characteristics of the adapter interface in
the TCP/IP stack: DEVICE/LINK or INTERFACE?
 Does the adapter interface definition include a virtual MAC (VMAC) keyword?
 Has VMAC ROUTEALL been coded or defaulted for the adapter interface?
 Has VMAC ROUTLCL been coded for the adapter interface?

Depending on these four factors, different addresses are registered in the OSA as described
here for the purposes of inbound routing and ARP offload:
 Inbound routing
– For INTERFACE statement with VMAC ROUTEALL, we do not register any IP
addresses for the purpose of inbound routing. We only register an IP address for the
purpose of supporting ARP offload.
– For INTERFACE without VMAC ROUTEALL or for DEVICE/LINK, we register the entire
home list for the purpose of inbound routing. (For DEVICE/LINK with VMAC
ROUTEALL, this registration is extraneous and is not used at all.)
 ARP offload
– We always register the home IP address for the purpose of ARP offload.
– If you have multiple OSAs on the same (V)LAN or Physical Network (PNET), and ARP
takeover is in effect, then we register the IP address of the interface for which we are
taking over connection responsibility.
– We also register VIPAs for ARP offload purposes as follows:
• For the INTERFACE statement with subnet mask configured on the statement, we
register only the VIPAs that are in the same subnet as the OSA.
• For the INTERFACE statement without a subnet mask coded on it. For DEV/LINK,
we register all the active VIPAs in the Home list.
For both of the above bullets, if there are multiple OSAs on the same (V)LAN or
Physical Network (PNET), we register these VIPAs on only one of the OSAs.

Note about displaying registered addresses: OSA/SF has a Get OAT function that
retrieves the registered IP addresses in the OAT. However, the displayed table is
incomplete, containing only a limited number of the addresses that the stack has registered
with the OSA device. When performing problem determination for the OSA, do not assume
that OSA/SF is showing you everything you need to know. You may have to solicit the help
of Level 2 defect support to see everything that has really been registered in the OSA.

122 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

OSA-Express VLAN support

The OSA-Express Ethernet features also support IEEE standards 802.1p/q (priority tagging
and VLAN identifier tagging). Deploying VLAN IDs enables a physical LAN to be partitioned
or subdivided into discrete virtual LANs. This support is provided by the z/OS TCP/IP stack
and OSA-Express in QDIO mode. It allows a TCP/IP stack to register specific single or
multiple VLAN IDs for both IPv4 and IPv6 for the same OSA-Express port. Note that the
VLAN IDs for IPv4 can be different than the VLAN ID for IPv6.

Note: The INTERFACE statement is required if one stack is going to attach to multiple
VLANs though a single OSA port.

When a VLAN ID is configured for an OSA-Express interface in the TCP/IP stack, the
following occurs:
 The TCP/IP stack becomes VLAN-enabled, and the OSA-Express port is considered to be
part of a VLAN.
 During activation, the TCP/IP stack registers the VLAN ID value to the OSA-Express port.
 A VLAN tag is added to all outbound packets.
 The OSA-Express port filters all inbound packets based on the configured VLAN ID.

If the TCP/IP stack is also configured with PRIRouter or SECRouter for an OSA-Express port
that has a VLAN ID defined, then the stack serves as an IP router for the configured VLAN ID.
If OSA-Express ports are shared across multiple TCP/IP routing stacks, consider using virtual
MAC support for your environment instead of the PRIRouter and SECRouter options. See
Chapter 6, “VLAN and Virtual MAC support” on page 265 for details.

VLAN support of Generic Attribute Registration Protocol (GVRP)

GVRP is defined in the IEEE 802.1p standard for the control of IEEE 802.1q VLANs. It can be
used to help simplify networking administration and management of VLANs. With GVRP
support, an OSA-Express3 or OSA-Express2 port can register or de-register its VLAN IDs
with a GVRP-capable switch and dynamically update its table as the VLANs change. Support
of GVRP is exclusive to System z9® or above and is applicable to all of the OSA-Express3
and OSA-Express2 features when in QDIO mode. Defining DYNVLANREG in the LINK
statement of the OSA-Express3 and OSA-Express2 port will enable GVRP.

OSA-Express router support

OSA-Express also provides primary (PRIRouter) and secondary (SECRouter) router support.
This function allows a single TCP/IP stack, on a per-protocol (IPv4 and IPv6) basis, to register
and act as a router stack based on a given OSA-Express port. Secondary routers can also be
configured to provide for conditions in which the primary router becomes unavailable and the
secondary router takes over for the primary router.

Chapter 4. Connectivity 123

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

Figure 4-2 shows how the PRIRouter function works in a shared OSA environment.

HiperSockets Network [Link]/24

LPAR A LPAR B LPAR C LPAR D

HS4 [Link] HS4 [Link] HS4 [Link] HS4 [Link]

VIPA [Link] VIPA [Link] VIPA [Link] VIPA [Link]
OSA [Link] OSA [Link] OSA [Link] OSA [Link]

PRIRouter SECRouter
VIPA [Link] VIPA [Link] VIPA [Link] VIPA [Link]
OSA [Link] OSA [Link] OSA [Link] OSA [Link]
HS4 [Link] HS4 [Link] HS4 [Link] HS4 [Link]

OSA 1 OSA 2

Connect to [Link] Connect to [Link]

Figure 4-2 How PRIRouter works with a shared OSA

In Figure 4-2, the terminal user connects to [Link]. Note that each stack sharing OSA1
has registered the IP addresses for VIPAs, OSAs, and the HiperSockets in the OSA Address
Table (OAT). However, the address [Link] is not represented in OSA1’s OAT. Therefore,
the packet from the terminal that arrives at OSA1 is sent to the primary routing TCP/IP stack
in LPAR A. The TCP/IP stack in LPAR A uses its routing table to forward the packet to LPAR
D, where IP address [Link] resides.

The connection to IP address [Link] is simpler. Because the address is represented in the
OAT of OSA1, the OSA can immediately forward the request to the correct TCP/IP stack in
LPAR C.

If LPAR A should become unavailable, then the TCP/IP stack in LPAR B or C will take over the
routing responsibility for OSA1.

VLAN and primary/secondary router support: VMAC support

The OSA-Express primary router support takes into consideration VLAN ID support (VLAN ID
registration and tagging) and interacts with it. OSA-Express supports a primary and
secondary router on a per-VLAN basis (per registered VLAN ID).

Therefore, if an OSA interface is configured with a specific VLAN ID and also configured as a
primary or secondary router, that stack serves as a router for just that specific VLAN. This

124 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

allows each OSA-Express (CHPID) to have a primary router per VLAN. Configuring primary
routers (one per VLAN) has many advantages and preserves traffic isolation for each VLAN.

If OSA-Express ports are shared across multiple TCP/IP routing stacks, consider using virtual
MAC support for your environment instead of the PRIRouter and SECRouter options. See
Chapter 6, “VLAN and Virtual MAC support” on page 265 for details.

High latency network

Streaming a workload over large bandwidth and high latency networks (such as satellite links)
is, in general, constrained by the TCP window size. The problem is that it takes time to send
data over such a network. At any given point in time, data filling the full window size is in
transit and cannot be acknowledged until it starts arriving at the receiver side. The sender can
send up to the window size and then must wait for an ACK to advance the window size before
the next chunk can be sent.

The left hand side of Figure 4-3 depicts a high-latency network where the TCP window size is
too small. The round trip time (RTT) is relatively long and the window size is relatively small.
Therefore, the sender fills the window before it receives an ACK for the data at the start of the
window. This forces the sender to delay sending additional data until it receives an ACK or a
window update. Over a long distance connection, this can cause transmission stalls and
suboptimal performance.

The right hand side demonstrates a situation where the window size is large enough for the
high-latency network. The sender has not yet sent the last bit of the window size before it
receives an ACK for the first bit of the current window. The z/OS TCP maximum windows size
is 512K (defined in the TCPMAXRCVBUFRSIZE in the TCPCONFIG section). However, a
window size of 512K may not always be enough to achieve this behavior.

Inefficient window size Efficient window size

Sender Receiver Sender Receiver

Data Data
Round Round
Data
trip Window trip
time size time K
AC Window
(RTT) (RTT)
K size
AC
Data
Data

K
AC
C K
A

Time Time

Figure 4-3 High latency network and window size

The solution for high latency networks

z/OS Communications Server implements Dynamic Right Sizing (DRS) to address the
problem related to high latency networks. DRS is described in a paper published by the Los
Alamos National Laboratory (LANL), which can be found at the following address:
[Link]

Chapter 4. Connectivity 125

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

The goal of the DRS function is to keep the pipe full for inbound streaming TCP connections
over networks with large capacity and high latency and prevent the sender from being
constrained by the receiver’s advertised window size.

If a TCP connection uses a receive buffer size larger then 64K, the stack detects a high
latency inbound streaming TCP connection and dynamically increases the receive buffer size
for the connection (in an attempt to not constrain the sender). This in turn adjusts the
advertised receive window and allows window size to grow as high as 2M. In other words, The
TCP receive buffer size can grow as high as 2M for certain TCP connections irrespective of
the TCPMAXRCVBUFRSIZE value. The stack disables the function for a connection if the
application is not keeping up with the pace.

DRS does not take effect for applications that set a value less then 64K on the SO_RCVBUF
socket option on SETSOCKOPT().

If TCPRCVBUFRSIZE is less then 64K, then DRS does not take effect for applications that do
not use the SO_RCVBUF socket option.

Implementation
To configure an OSA-Express3 feature to operate in optimized latency mode, use the
INTERFACE statement with the OLM parameter. Because optimized latency mode affects
both inbound and outbound interrupts, it supersedes other inbound performance settings set
by the INBPERF parameter.

Optimized latency mode is limited to the OSA-Express3 Ethernet feature in QDIO mode
running with an IBM System z10®.

Restrictions
You must observe the following restrictions:
 Traffic that is either inbound over or being forwarded to an OSA-Express3 feature
configured to operate in optimized latency mode is not eligible for the accelerated routing
provided by HiperSockets Accelerator and QDIO Accelerator.
 For an OSA-Express3 configured to operate in optimized latency mode, the stack ignores
the value coded on the INBPERF parameter. The value assigned to the INBPERF is
DYNAMIC.

Guidelines
Because of the operating characteristics of optimized latency mode, other configuration
changes might be required:
 For outbound traffic to gain the benefit of optimized latency mode, direct traffic to priority
queues 1, 2, or 3 using the WLMPRIORITYQ parameter in the GLOBALCONFIG
statement or using Policy Agent and configuring a policy with the SetSubnetPrioTosMask
statement.
 Although an OSA-Express feature supports multiple outbound write priority queues,
outbound optimized latency mode is performed only for traffic on priority queue 1 (priority
level 1). The TCP/IP stack combines all the traffic directed to priority queues 1, 2, and 3
into priority queue 1 for any OSA-Express3 feature operating in optimized latency mode.
 Configure the WLMPRIORITYQ parameter with no subparameters, which assigns a
default mapping of service class importance levels to OSA-Express outbound priority
queues. This default mapping directs traffic assigned to the higher priority service class
importance levels 1–4 to queues that operate in optimized latency mode, and enables the
appropriate types of traffic to benefit from optimized latency mode.

126 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

 Ensure that there are no more than four concurrent users of an OSA-Express3 feature that
is configured with optimized latency mode.
 When enabling multipath routing using the PERPACKET option, do not configure a
multipath group that contains an OSA-Express3 feature configured with optimized latency
mode and any other type of device.

For more information regarding OSA-Express features and capabilities, refer to OSA-Express
Implementation Guide, SG24-5948.

OSA multiple inbound queue support

Outbound traffic separation (assignment to specific priority queue) on the multiple write
queues can be accomplished by using Policy Agent and configuring a policy with the
SetSubnetPrioTosMask statement, and by using the WLMPRIORITYQ parameter on the
GLOBALCONFIG statement. Each priority queue is processed independently of the others.

The left hand side of Figure 4-4 depicts OSA single inbound queue support. All inbound
QDIO traffic is received on a single read queue regardless of the data type. This includes both
batch and interactive traffic and both traffic destined for this TCP/IP stack and traffic to be
forwarded by this TCP/IP stack. The maximum amount of storage available for inbound traffic
is limited to the read buffer size (64K read SBALs) times the maximum number of read buffers
(126). Multiple processes only run for inbound traffic when data is accumulating on the read
queue – typically during burst periods when z/OS Communications Server is not keeping up
with the OSA. This can cause bulk data packets for a single TCP connection to arrive at the
TCP layer out of order. Each time the TCP layer on the receiving side sees out of order data,
it transmits a duplicate ACK. A single process is used to package the data, queue it, and
schedule the TCP/IP stack to process it. This same process also performs acceleration
functions, such as Sysplex Distributor connection routing accelerator. The TCP/IP stack
separates the traffic types to be forwarded to the appropriate stack component that will
process them.

For these reasons, z/OS Communications Server is becoming the bottleneck as

OSA-Express3 10 GbE nears line speed. z/OS Communications Server is injecting latency
and increasing processor utilization.

Chapter 4. Connectivity 127

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

A single inbound queue Multiple inbound queues

z/OS z/OS
Application Application
Application Application
Application Application

TCP/IP TCP/IP

z z
CP CP CP CP CP CP CP CP CP CP

OSA OSA

other

bulk
SD
1 2 3 4 1 2 3 4

Figure 4-4 QDIO inbound queueing

The solution for the bottleneck on a single read queue

z/OS Communications Server supports inbound traffic separation using multiple read queues.
The right hand side of Figure 4-4 on page 128 depicts OSA multiple inbound queue support.
TCP/IP will register with OSA which traffic to be received on each read queue. The
OSA-Express Data Router function routes traffic to the correct queue. Each read queue can
be serviced by a separate process. The primary input queue is used for general traffic. One or
more ancillary input queues (AIQs) are used for specific traffic types.

The supported traffic types are streaming bulk data and sysplex distributor. Examples of bulk
data traffic are FTP, TSM, NFS, and TDMF®. Both IP versions are supported for all types of
traffic.

With bulk data traffic separated onto its own read queue, TCP/IP will service the bulk data
queue from a single processor. This solves the out of order delivery issue. With sysplex
distributor traffic separated onto its own read queue, it can be efficiently accelerated or
presented to the target application. All other traffic is processed simultaneous with the bulk
data and sysplex distributor traffic.

The dynamic LAN idle timer is updated independently for each read queue. This ensures the
most efficient processing of inbound traffic based on the traffic type.

Implementation
The QDIO inbound workload queueing function is enabled with the INBPERF DYNAMIC
WORKLOADQ setting on IPAQENET and IPAQENET6 INTERFACE statements.
WORKLOADQ is not supported for INBPERF DYNAMIC on IPAQENET LINK statements. The
VMAC parameter can be specified with or without macaddr.

For more information, refer to the IPAQENET INTERFACE and IPAQENET6 INTERFACE
statements in z/OS Communications Server: IP Configuration Reference, SC31-8776.

128 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Verification
Refer to a WorkloadQueueing field in the Netstat DEvlinks/-d report to see whether the QDIO
inbound workload queueing function is enabled. This information can also be returned by the
GetIfs callable NMI.

Moreover, you can use other commands to obtain more information about the QDIO inbound
workload queueing function for the QDIO interface. The output for the Display ID=trlename
and Display TRL,TRLE=trlename commands shows whether this function is in use for the
QDIO interface as follows:
 For each input queue, it includes the queue ID and queue type in addition to the read
storage. The queue type is PRIMARY for the primary input queue, BULKDATA for the bulk
data AIQ, and SYSDIST for the sysplex distributor connection routing AIQ.
 The queue type value N/A indicates that the queue is initialized but is not currently in use
by the TCP/IP stack.

In addition, the queue ID and queue type can be used to correlate with VTAM tuning statistics,
packet trace, and OSA-Express Network Traffic Analyzer (OSAENTA) trace output for the
QDIO interface. The Netstat ALL/-A report includes the interface name for bulk data TCP
connections that are using this function. This information can also be returned by the
GetConnectionDetail callable NMI. The Netstat STATS/-S report includes the total number of
segments received for all connections from the bulk data AIQ of this function. This information
can also be returned by the GetGlobalStats callable NMI

For more information, refer to z/OS Communications Server: IP System Administrator’s

Commands, SC31-8781, and the DISPLAY ID and DISPLAY TRL commands in z/OS
Communications Server: SNA Operation, SC31-8779.

4.2.3 OSA-Express for zEnterprise (z196)

The IBM zEnterprise 196 (z196) offers communications access to two new internal networks
through OSA-Express3 adapters that are configured with an appropriate channel path ID
(CHPID) type. The following list describes the two new internal networks:
 The intranode management network - It provides connectivity between network
management applications within the z196 node and it can be accessed through
1000BASE-T Ethernet OSA-Express3 adapters that are configured with a CHPID type of
OSM.
 The intraensemble data network - It provides access to other images that are connected to
the intraensemble data network and to applications and appliances that are running in an
IBM zEnterprise BladeCenter® Extension (zBX). This internal network can be accessed
through 10 GbE OSA-Express3 adapters that are configured with a CHPID type of OSX

Restrictions:
 Access to the intranode management network is restricted to authorized management
applications, and is only available through Port 0 of any OSA-Express3 CHPID
configured with type OSM. Port 1 is not available for these communications.
 Connectivity to the intranode management network is restricted to stacks that are
enabled for IPv6.
 Connectivity to the intranode management network and to the intraensemble data
network is allowed only when the central processor complex (CPC) is a member of an
ensemble.

Chapter 4. Connectivity 129

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

See Figure 4-5 on page 130 for zEnterprise design.

Customer
Managed
Management
Network

zEnterprise Node
Intranode TOR TOR
HMC TOR TOR
Management
Network TOR TOR
(INMN) TOR TOR
OSM

z196
zBX zBX
OSX Intraensemble
Data Network
(IEDN)
OSD

Customer
Managed BladeCenter
Data Chassis
Networks

Figure 4-5 zEnterprise Node

Tip: OSA/SF cannot manage these CHPIDs. You can see the related informations about
these CHPIDs by using the “DISPLAY TCPIP,,OSAINFO” command.

4.2.4 HiperSockets (MPCIPA)

HiperSockets, also known as internal Queued Direct I/O (iQDIO), is a hardware feature that
provides high-speed LPAR-to-LPAR communications within the same sever (through
memory). It also provides secure data flows between LPARs and high availability, if there is no
network attachment dependency or exposure to adapter failures.

HiperSockets can be used to communicate among consolidated servers within a single

System z server. All the hardware boxes running these separate servers can be eliminated,
along with the cost, complexity, and maintenance of the networking components that
interconnect them.

Consolidated servers that have to access corporate data residing on the System z server can
do so at memory speeds, bypassing all the network overhead and delays.

HiperSockets can be customized to accommodate varying traffic sizes. With HiperSockets, a

maximum frame size can be defined according to the traffic characteristics transported for
each HiperSockets.

130 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Because there is no server-to-server traffic outside the System z server, a much higher level
of network availability, security, simplicity, performance, and cost effectiveness is achieved as
compared with servers communicating across a LAN, such as:
 HiperSockets has no external components. It provides a very secure connection. For
security purposes, servers can be connected to different HiperSockets or VLANs within
the same HiperSockets. All security features, like IPSec or IP filtering, are available for
HiperSockets interfaces as they are with other TCP/IP network interfaces.
 HiperSockets looks like any other TCP/IP interface; therefore, it is transparent to
applications and supported operating systems.
 HiperSockets can also improve TCP/IP communications within a sysplex environment
when the DYNAMICXCF is used (for example, in cases where Sysplex Distributor uses
HiperSockets within the same System z server to transfer IP packets to the target
systems).

The HiperSockets device is represented by the IQD channel ID (CHPID) and its associated
subchannel devices. All LPARs that are configured in HCD/IOCP to use the same IQD CHPID
have internal connectivity and, therefore, have the capability to communicate using
HiperSockets.

VTAM will build a single HiperSockets MPC group using the subchannel devices associated
with a single IQD CHPID. VTAM will use two subchannel devices for the read and write control
devices, and 1 to 8 devices for data devices. Each TCP/IP stack will be assigned a single data
device.

Therefore, in order to build the MPC group, there must be a minimum of three subchannel
devices defined (within HCD) and associated with the same IQD CHPID. The maximum
number of subchannel devices that VTAM will use is 10 (supporting 8 data devices or 8
TCP/IP stacks) per LPAR or MVS image.

When the server that supports HiperSockets and the CHPIDs has been configured in HCD
(IOCP), TCP/IP connectivity is provided if:
 DYNAMICXCF is configured on the IPCONFIG (IPv4) or the IPCONFIG6 (IPv6)
statements.
 A user-defined HiperSockets (MPCIPA) DEVICE and LINK for IPv4 or (IPAQIDIO)
INTERFACE for IPv6 is configured and started.

IQD CHPID can be viewed as a logical LAN within the server. System z servers allow up to
16 separate IQD CHPIDs, creating the capability of having up to 16 separate logical LANs
within the same server.

Each IQD CHPID can be assigned to a set of LPARs (configured in HCD), making it possible
to isolate these LPARs in separate logical LANs, as shown in of Figure 4-6.

Chapter 4. Connectivity 131

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

System z Server

Production Test

LPAR 1 LPAR 2 LPAR 3 LPAR 4

TCP/IP TCP/IP TCP/IP TCP/IP

Stack A Stack B Stack C Stack D

HiperSockets HiperSockets HiperSockets

(CHPID FD) (CHPID FE) (CHPID FF)

Figure 4-6 HiperSockets: multiple logical LANs

HiperSockets multiple write

The HiperSockets multiple write facility moves multiple buffers of data with a single write
operation. This facility was added to reduce CPU utilization and to improve performance for
large outbound messages over HiperSockets.

Restriction: HiperSockets multiple write is effective only on an IBM System z10 and when
z/OS is not running as a guest in a z/VM® environment.

To enable the HiperSockets multiple write facility on all HiperSockets interfaces, including
interfaces created for dynamic XCF, add the IQDMULTIWRITE parameter to the
GLOBALCONFIG statement.

For more information, see Appendix B, “Additional parameters and functions” on page 411.

For a review of the scenarios we used to test HiperSockets multiple write, see Appendix A,
“HiperSockets Multiple Write”, in IBM z/OS V1R11 Communications Server TCP/IP
Implementation Volume 3: High Availability, Scalability, and Performance, SG24-7800.

HiperSockets multiple write assist with IBM zIIP

On an IBM System z10, an additional assist for HiperSockets data that is using the multiple
write facility is available through the IBM System z10 Integrated Information Processor (zIIP).

To enable HiperSockets traffic that is using the multiple write facility to be processed on
available zIIPs, specify the ZIIP IQDIOMULTIWRITE parameter on the GLOBALCONFIG
statement.

HiperSockets VLAN support

HiperSockets connections defined through DYNAMICXCF coding or through individual
DEVICE and LINK statement coding also support VLAN tagging. This allows you to split the
internal LAN represented by a single HiperSockets CHPID into multiple virtual LANs,
providing isolation for security or administrative purposes. Only stacks attached to the same
HiperSockets VLAN can communicate with each other. Stacks attached to a different

132 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

HiperSockets VLAN on the same CHPID cannot use the HiperSockets path to communicate
with the stacks on a different VLAN.

Note: The VLAN ID assigned to a HiperSockets device applies to both IPv4 and IPv6
connections over that CHPID.

HiperSockets accelerator
The Communications Server takes advantage of the technological advances and
high-performing nature of the I/O processing offered by HiperSockets with the IBM System z
servers and OSA-Express, using the QDIO architecture. This is achieved by optimizing IP
packet forwarding processing that occurs across these two types of technologies. This
function is referred to as HiperSockets Accelerator. It is a configurable option, and is activated
by defining the IQDIORouting option on the IPCONFIG statement.

When the TCP/IP stack is configured with HiperSockets Accelerator, it allows IP packets
received from HiperSockets to be forwarded to an OSA-Express port (or vice versa) without
the need for those IP packets to be processed by the TCP/IP stack.

When using this function, one or more LPARs contain the routing stack, which manages
connectivity through OSA-Express ports to the LAN, while the other LPARs connect to the
routing stack using the HiperSockets, as shown in Figure 4-7.

System z Server

LPAR A LPAR B LPAR C LPAR D

TCP/IP TCP/IP TCP/IP TCP/IP

Stack A Stack B Stack C Stack D

LPAR E

HiperSockets HiperSockets
(CHPID FE) (CHPID FD)

OSA OSA
Gigabit Ethernet Network

Figure 4-7 HiperSockets Accelerator

Note: This example is intended purely to demonstrate IP traffic flow. We do not

recommend implementing HiperSockets Accelerator using a single LPAR.

Detailed information about the subject of HiperSockets is available in HiperSockets

Implementation Guide, SG24-6816.

Chapter 4. Connectivity 133

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

4.2.5 Dynamic XCF

You have a choice of defining the XCF connectivity to other TCP/IP stacks individually or
using the dynamic XCF definition facility. Dynamic XCF significantly reduces the number of
definitions that you need to create whenever a new system joins the sysplex or when you
need to start up a new TCP/IP stack. These changes become more numerous as the number
of stacks and systems in the sysplex grows. This could lead to configuration errors. With
dynamic XCF, you do not need to change the definitions of the existing stacks in order to
accommodate the new stack.

From an IP topology perspective, DYNAMICXCF establishes fully meshed IP connectivity to

all other z/OS TCP/IP stacks in the sysplex. You only need one end-point specification in each
stack for fully meshed connectivity to all other stacks in the sysplex. When a new stack gets
started, Dynamic XCF connectivity is automatically established.

Note: Only one dynamic XCF network is supported per sysplex.

Dynamic XCF is required to support Sysplex Distributor and nondisruptive dynamic VIPA
movement (discussed in detail in IBM z/OS V1R11 Communications Server TCP/IP
Implementation Volume 3: High Availability, Scalability, and Performance, SG24-7800).

Dynamic XCF uses Sysplex Sockets support, allowing the stacks to communicate with each
other and exchange information such as VTAM CPNAMEs, MVS SYSCLONE value, and IP
addresses. The dynamic XCF definition is activated by coding the IPCONFIG DYNAMICXCF
parameter in the TCP/IP profile.

Dynamic XCF creates definitions for DEVICE, LINK, HOME, and BSDROUTINGPARMS
statements and the START statement dynamically. When activated, the dynamic XCF devices
and links appear to the stack as though they had been defined in the TCP/IP profile. They can
be displayed using standard commands, and they can be stopped and started.

During TCP/IP initialization the stack joins the XCF group, ISTXCF, through VTAM. When
other stacks in the group discover the new stack, the definitions are created automatically, the
links are activated, and the remote IP address for each link is added to the routing table. After
the remote IP address has been added, IP traffic can flow across one of the following
interfaces:
 IUTSAMEH (within the same LPAR)
 HiperSockets (within the same server)
 XCF signaling (different server, either using the Coupling Facility link or a CTC connection)

Dynamic XCF support is illustrated in Figure 4-8.

134 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

TCP/IP IUTSAMEH TCP/IP HiperSockets TCP/IP

Stack A Stack B Stack C

LPAR 1 LPAR 2

System z Server 1

)
Channel-to-Channel

ng
Coupling Facility Link

ali
(CTC)

gn
CF

Si
CF
(X

TCP/IP
Stack D

LPAR 3

System z Server 2

Figure 4-8 Dynamic XCF support

HiperSockets DYNAMICXCF connectivity

z/OS images within the same server with DYNAMICXCF coded will use HiperSockets
DYNAMICXCF connectivity instead of standard XCF connectivity, under these conditions:
 The TCP/IP stacks must be on the same server.
 For the DYNAMICXCF HiperSockets device (IUTIQDIO), the stacks must be using the
same IQD CHPID, even with different channel subsystems (spanning).
 The stacks must be configured through HCD or the IOCDS to use HiperSockets.
 For IPv6 HiperSockets connectivity, both stacks must be at z/OS V1R7 or higher.
 The initial HiperSockets activation must complete successfully.

When an IPv4 DYNAMICXCF HiperSockets device and link are created and successfully
activated, a subnetwork route is created across the HiperSockets link. The subnetwork is
created by using the DYNAMICXCF IP address and mask. This allows any LPAR within the
same server to be reached, even ones that are not within the sysplex. To do that, the LPAR
that is outside of the sysplex environment must define at least one IP address for the
HiperSockets endpoint that is within the subnetwork defined by the DYNAMICXCF IP address
and mask.

When multiple stacks reside within the same LPAR that supports HiperSockets, both
IUTSAMEH and HiperSockets links or interfaces will coexist. In this case, it is possible to
transfer data across either link. Because IUTSAMEH links have better performance, it is
always better to use them for intra-stack communication. A host route will be created by
DYNAMICXCF processing across the IUTSAMEH link, but not across the HiperSockets link.

For additional information about dynamic XCF, Sysplex Distributor, and nondisruptive
dynamic VIPA movement refer to IBM z/OS V1R12 Communications Server TCP/IP
Implementation Volume 3: High Availability, SG24-7898.

Chapter 4. Connectivity 135

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

4.3 Connectivity for the z/OS environment

The subsequent sections focus on the interface implementation only, which means
establishing Layer 2 and a subset of Layer 3 (IP addressing) connectivity. For details beyond
the basic implementation of the immediate LAN environment, also refer to:
 Chapter 5, “Routing” on page 205 for IP routing details
 Chapter 6, “VLAN and Virtual MAC support” on page 265 for use of virtual MAC
addresses
 Chapter 7, “Sysplex subplexing” on page 283 for isolating TCP/IP stack in a sysplex

To design connectivity in a z/OS environment, you must take the following considerations into
account:
 As a server environment, network connectivity to the external corporate network should be
carefully designed to provide a high-availability environment, avoiding single points of
failures.
 If a z/OS LPAR is seen as a stand-alone server environment on the corporate network, it
should be designed as an endpoint.
 If a z/OS LPAR will be used as a front-end concentrator (for example, making use of
HiperSockets Accelerator), it should be designed as an intermediate network or node.

Recommendation: Although there are specialized cases where multiple stacks per LPAR
can provide value, in general we recommend implementing only one TCP/IP stack per
LPAR. The reasons for this recommendation are as follows:
 A TCP/IP stack is capable of exploiting all available resources defined to the LPAR in
which it is running. Therefore, starting multiple stacks will not yield any increase in
throughput.
 When running multiple TCP/IP stacks, additional system resources, such as memory,
CPU cycles, and storage, are required.
 Multiple TCP/IP stacks add a significant level of complexity to TCP/IP system
administration tasks.
 It is not necessary to start multiple stacks to support multiple instances of an application
on a given port number, such as a test HTTP server on port 80 and a production HTTP
server also on port 80. This type of support can instead be implemented using
BIND-specific support where the two HTTP server instances are each associated to
port 80 with their own IP address, using the BIND option on the PORT reservation
statement.

One example where multiple stacks can have value is when an LPAR needs to be
connected to multiple isolated security zones in such a way that there is no network level
connectivity between the security zones. In this case, a TCP/IP stack per security zone can
be used to provide that level of isolation, without any network connectivity between the
stacks.

Based on these considerations, in the following sections we present best practice scenarios
for building a z/OS Communications Server TCP/IP configuration, using OSA-Express
(QDIO), HiperSockets (iQDIO), and dynamic XCF.

We built our connectivity scenarios with two OSA-Express3 1000BASE-T features (four ports
each) that are connected to the LAN environment (one layer3 switch). We also implemented a

136 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

HiperSockets internal LAN to interconnect all LPARs within the same System z10. Finally, we
used dynamic XCF connectivity for the Sysplex environment.

Note: In our environment we connected all the OSA ports to one switch, but in a
production implementation it is best to connect your OSAs to at least two switches

The scenarios we discuss are as follows:

 4.4.3, “Configuring OSA-Express with VLAN ID” on page 148
 4.6.3, “Configuring HiperSockets” on page 182
 4.7.3, “Configuring DYNAMICXCF” on page 189

Note: In this chapter, we define only our LPARs as end points.

For a complete picture of our implementation environment, refer to Appendix D, “Our

implementation environment” on page 457.

4.3.1 IOCP definitions

Example 4-1 on page 137 is an excerpt of the IOCP statements we used in our System z
environment (only showing OSA-Express CHPID 02 and HiperSockets CHPID F4). These
statements are required by the input/output subsystem and the operating system. Because all
of our OSA-Express and HiperSockets connectivity will be used across all four LPARs, we
defined the CHPIDs as shared.

Example 4-1 IOCP statements

ID MSG2='SYS6.IODF64 - 2010-09-23 11:18',SYSTEM=(2817,1), *
LSYSTEM=SCZP301, *
TOK=('SCZP301',00800006991E2094111808480110266F00000000,*
00000000,'10-09-23','[Link]','SYS6','IODF64')
RESOURCE PARTITION=((CSS(0),(A0A,A),(A0B,B),(A0C,C),(A0D,D),(A*
0E,E),(A0F,F),(A01,1),(A02,2),(A03,3),(A04,4),(A05,5),(A*
06,6),(A07,7),(A08,8),(A09,9)),(CSS(1),(A1B,B),(A1E,E),(*
A1F,F),(A11,1),(A12,2),(A13,3),(A14,4),(A15,5),(A16,6),(*
A17,7),(A18,8),(*,9),(*,A),(*,C),(*,D)),(CSS(2),(A2F,F),*
(A21,1),(A22,2),(*,3),(*,4),(*,5),(*,6),(*,7),(*,8),(*,9*
),(*,A),(*,B),(*,C),(*,D),(*,E)),(CSS(3),(A31,1),(*,2),(*
*,3),(*,4),(*,5),(*,6),(*,7),(*,8),(*,9),(*,A),(*,B),(*,*
C),(*,D),(*,E),(*,F)))

CHPID PATH=(CSS(1),0A),SHARED,
PARTITION=((A11,A13,A16,A18),(=)),CHPARM=02,PCHID=531,
TYPE=OSM 1
CHPID PATH=(CSS(1),0B),SHARED,
PARTITION=((A11,A13,A16,A18),(=)),CHPARM=02,PCHID=101,
TYPE=OSM

CNTLUNIT CUNUMBR=2340,PATH=((CSS(1),0A)),UNIT=OSM
IODEVICE ADDRESS=(2340,015),MODEL=M,UNITADD=00,CUNUMBR=(2340),
UNIT=OSA,MODEL=M,DYNAMIC=YES,LOCANY=YES
CNTLUNIT CUNUMBR=2360,PATH=((CSS(1),0B)),UNIT=OSM
IODEVICE ADDRESS=(2360,015),MODEL=M,UNITADD=00,CUNUMBR=(2360),
UNIT=OSA,MODEL=M,DYNAMIC=YES,LOCANY=YES

Chapter 4. Connectivity 137

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

CHPID PATH=(CSS(1),18),SHARED,
PARTITION=((A11,A13,A16,A18),(=)),PCHID=590,TYPE=OSX 1
CHPID PATH=(CSS(1),19),SHARED,
PARTITION=((A11,A13,A16,A18),(=)),CHPARM=02,PCHID=510,
TYPE=OSX

CNTLUNIT CUNUMBR=2300,PATH=((CSS(1),18)),UNIT=OSX
IODEVICE ADDRESS=(2300,015),MODEL=X,CUNUMBR=(2300),UNIT=OSA,
MODEL=X,DYNAMIC=YES,LOCANY=YES
CNTLUNIT CUNUMBR=2320,PATH=((CSS(1),19)),UNIT=OSX
IODEVICE ADDRESS=(2320,015),MODEL=X,UNITADD=00,CUNUMBR=(2320),
UNIT=OSA,MODEL=X,DYNAMIC=YES,LOCANY=YES

CHPID PATH=(CSS(1),02),SHARED,
PARTITION=((A11,A13,A16,A18),(=)),PCHID=530,TYPE=OSD
CHPID PATH=(CSS(1),03),SHARED,
PARTITION=((A11,A13,A16,A18),(=)),PCHID=100,TYPE=OSD
CHPID PATH=(CSS(1),04),SHARED,
PARTITION=((A11,A13,A16,A18),(=)),PCHID=181,TYPE=OSD
CHPID PATH=(CSS(1),05),SHARED,
PARTITION=((A11,A13,A16,A18),(=)),PCHID=291,TYPE=OSD

CNTLUNIT CUNUMBR=2080,PATH=((CSS(1),02)),UNIT=OSA
IODEVICE ADDRESS=(2080,015),UNITADD=00,CUNUMBR=(2080),UNIT=OSA
IODEVICE ADDRESS=208F,UNITADD=FE,CUNUMBR=(2080),UNIT=OSAD
CNTLUNIT CUNUMBR=20A0,PATH=((CSS(1),03)),UNIT=OSA
IODEVICE ADDRESS=(20A0,015),UNITADD=00,CUNUMBR=(20A0),UNIT=OSA
IODEVICE ADDRESS=20AF,UNITADD=FE,CUNUMBR=(20A0),UNIT=OSAD
CNTLUNIT CUNUMBR=20C0,PATH=((CSS(1),04)),UNIT=OSA
IODEVICE ADDRESS=(20C0,015),UNITADD=00,CUNUMBR=(20C0),UNIT=OSA
IODEVICE ADDRESS=20CF,UNITADD=FE,CUNUMBR=(20C0),UNIT=OSAD
CNTLUNIT CUNUMBR=20E0,PATH=((CSS(1),05)),UNIT=OSA
IODEVICE ADDRESS=(20E0,015),UNITADD=00,CUNUMBR=(20E0),UNIT=OSA
IODEVICE ADDRESS=20EF,UNITADD=FE,CUNUMBR=(20E0),UNIT=OSAD

CHPID PATH=(CSS(1),F4),SHARED,
PARTITION=((A11,A13,A16,A18),(=)),TYPE=IQD
CHPID PATH=(CSS(1),F5),SHARED,
PARTITION=((A11,A13,A16,A18),(=)),TYPE=IQD
CHPID PATH=(CSS(1),F6),SHARED,
PARTITION=((A11,A13,A16,A18),(=)),TYPE=IQD
CHPID PATH=(CSS(1),F7),SHARED,
PARTITION=((A11,A13,A16,A18),(=)),TYPE=IQD

CNTLUNIT CUNUMBR=E800,PATH=((CSS(1),F4)),UNIT=IQD
IODEVICE ADDRESS=(E800,032),CUNUMBR=(E800),UNIT=IQD
CNTLUNIT CUNUMBR=E900,PATH=((CSS(1),F5)),UNIT=IQD
IODEVICE ADDRESS=(E900,032),CUNUMBR=(E900),UNIT=IQD
CNTLUNIT CUNUMBR=EA00,PATH=((CSS(1),F6)),UNIT=IQD
IODEVICE ADDRESS=(EA00,032),CUNUMBR=(EA00),UNIT=IQD
CNTLUNIT CUNUMBR=EB00,PATH=((CSS(1),F7)),UNIT=IQD
IODEVICE ADDRESS=(EB00,032),CUNUMBR=(EB00),UNIT=IQD

138 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Attention: 1 The CHPIDs type OSM and OSX are only used if you are connected to a
zBX (zEnterprise Blade Center.

There are other ways to build the IOCDS for an OSA-Express adapter than the one depicted
in Example 4-1 on page 137. This applies particularly to an OSA-Express3, which can
contain more than a single port on the same CHPID. However, in our labs, we used the
method shown in Example 4-1 on page 137. Consult 4.4.1, “Dependencies: CHPID, IOCDS,
port numbers, portnames, and port sharing” on page 140 to see other alternatives to define
the IOCDS and to review our recommendations.

4.3.2 VTAM definitions

Before getting started with configuring the scenarios in the following sections, it is important
to understand the role of VTAM in the TCP/IP configuration.

z/OS Communications Server provides a set of High Performance Data Transfer (HPDT)
services that includes MultiPath Channel (MPC), a high-speed channel interface designed for
network protocol use (for example, APPN or TCP/IP).

Multiple protocols can either share or have exclusive use of a set of channel paths to an
attached platform. MPC provides the ability to have multiple device paths, defined as a single
logical connection.

The term MPC group is used to define a single MPC connection that can contain multiple
read and write paths. The number of read and write paths does not have to be equal, but
there must be at least one read and write path defined within each MPC group.

MPC groups are defined using the Transport Resource List (TRL), where each defined MPC
group becomes an entry (that is, a TRLE) in the TRL table. The configuration and control of
the MultiPath Channel (MPC) interfaces are provided by VTAM. They are enabled in VTAM as
TRLE minor nodes.

You must define the channel paths that are a part of the group in the TRLE. Each TRLE is
identified by a resource_name. For OSA-Express, the TRLE also has a port_name to identify
the association between VTAM and TCP/IP, allowing connectivity to the OSA-Express port.
OSA-Express3 Gigabit Ethernet and 1000Base-T also defines port_num to identify which
port the TRLE definition applies to.

For HiperSockets, the TRLE is generated dynamically by VTAM.

For details about defining a TRLE, refer to z/OS Communications Server: SNA Resource
Definition, SC31-8778.

4.4 OSA-Express QDIO connectivity

Configuring an OSA-Express (QDIO mode) in a single stack scenario is the simplest way to
integrate your z/OS TCP/IP stack into a LAN environment. This scenario, however, still needs
to be planned to avoid any single points of failure. Therefore, we must have at least two
OSA-Express features connecting to two different switches in the network.

Because we are dealing with multiple LPARs in our server, for redundancy purposes we have
shared the OSA-Express ports (CHPID type OSD) across all LPARs.

Chapter 4. Connectivity 139

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

In this scenario, we have two OSA-Express3 1000BASE-T features, each with four ports, two
ports per channel. One port of each channel was used unless the second port was needed
for testing of new functions. This allowed us to have four CHPIDs (02, 03, 04, and 05), shared
by our four LPARs (SC30, SC31, SC32 and SC33), as shown in Figure 4-9 on page 140.

SC30 SC31 SC32 SC33

TCPIPA TCPIPB TCPIPC TCPIPD

VIPA1L [Link]/24 VIPA1L [Link]/24 VIPA1L [Link]/24 VIPA1L [Link]/24

OSA2080I [Link]/24 OSA2080I [Link]/24 OSA2080I [Link]/24 OSA2080I [Link]/24

OSA20A0I [Link]/24 OSA20A0I [Link]/24 OSA20A0I [Link]/24 OSA20A0I [Link]/24
OSA20C0I [Link]/24 OSA20C0I [Link]/24 OSA20C0I [Link]/24 OSA20C0I [Link]/24
OSA20E0I [Link]/24 OSA20E0I [Link]/24 OSA20E0I [Link]/24 OSA20E0I [Link]/24

CHPID 02 CHPID 03 CHPID 04 CHPID 05

OSA2080 OSA20A0 OSA-Express 1000BASE-T OSA20C0 OSA20E0
10.1.2.x1 10.1.2.x2 10.1.3.x1 10.1.3.x2
2080-208F 20A0-20AF 20C0-20CF 20E0-2E0F

TRUNK MODE TRUNK MODE TRUNK MODE TRUNK MODE

VLAN 10 VLAN 11
[Link] [Link]

SWITCH

Figure 4-9 OSA-Express (QDIO) implementation

To make better use of our OSA-Express ports and to control data traffic patterns, we defined
one port on each OSA-Express feature with a separate VLAN ID, creating two subnetworks to
be used by all LPARs. In a high availability configuration, these OSA-Express ports will be the
path to all of our IP addresses for the LAN environment.

4.4.1 Dependencies: CHPID, IOCDS, port numbers, portnames, and port

sharing
To implement this scenario, we have the following dependencies:
 The OSA-Express port must be defined as CHPID type OSD to the server using HCD or
IOCP to enable QDIO. This CHPID must be defined as shared to all LPARs that will use
the OSA-Express port (see Example 4-1 on page 137).
 To define an OSA-Express port in QDIO mode, we use the MPCIPA DEVICE statement,
specifying the PORTNAME value from the TRLE definition as the device_name. The
TRLE must be defined as MPCLEVEL=QDIO.
 The Virtual LAN identifiers (VLAN IDs) defined to each OSA-Express port must be
recognized by the switch.
 The switch ports where the OSA-Express ports are connected must be configured in trunk
mode.

140 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

OSA-Express2 and OSA-Express3 Adapter and port layouts

While an OSA-Express2 adapter (with the exception of the 10 Gbe adapter) contains two
ports, the OSA-Express3 (with the exception of the 10 Gbe adapter) houses four ports.
Compare and contrast the layout of an OSA-Express2 and OSA-Express3 in Figure 4-10.

OSA-E2: 2-Port Adapter OSA-E3: 4-Port Adapter

Port 0
CHPID x Port 0 CHPID x
Port 1

Port 1
CHPID y Port 0
CHPID y
Port 0

Figure 4-10 Comparison: OSA-E2 2-port adapter and OSA-E3 4-port adapter

Each port of the OSA-Express2 adapter depicted in Figure 4-10 is on a separate CHPID:
CHPID x and CHPID y. Each port on each CHPID is defined with a separate port name and
resides at port number 0.

The OSA-Express3 is engineered with two ports on each CHPID: CHPID x and CHPID y. The
two ports on each CHPID are numbered port 0 and port 1. Note how the top half of the
OSA-E3 is the mirror image of the bottom half with regard to the port number assignments;
reading from top to bottom, you see Port 0, Port 1, Port 1, Port 0. As with any OSA port, the
portnames on the multi-port OSA-E3 must be unique to a CHPID. An explanation of this
portname assignment is provided in “Considerations for assigning the OSA portname” on
page 147.

Considerations for the IOCP or IOCDS Definitions for an OSA-Express3

We have shown you in Example 4-1 on page 137 an IOCDS that was originally built for an
OSA-Express2 configuration. When migrating to an OSA-Express3, you can choose to use a
similar IOCDS and spread the assigned addresses from a single address range across two
ports of the same CHPID that originally connected to only one port. You can also choose to
change your IOCDS to reflect separate address ranges or even separate logical control units,
despite the presence of only a single physical control unit on the CHPID. We now want to
show you a couple of different ways to implement an IOCDS for an OSA-Express3
implementation.

Chapter 4. Connectivity 141

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

Alternative 1: Single IODEVICE range for two E3 ports on single CHPID

In our scenarios where we used an OSA-Express3, we used exactly the same IOCDS
definitions as those we deployed for an OSA-Express2. You have seen this IOCDS in
Example 4-1 on page 137. We use as an example the IOCDS definitions for the devices on
OSA port OSA2080. In Example 4-2, you see at 1 that we have allocated fifteen addresses
(2080-208E) to QDIO connections starting with device address 2080.

Example 4-2 Sample CNTLUNIT and IODEVICE for an OSA on CHPID Type OSD (QDIO)
CNTLUNIT CUNUMBR=2080,PATH=((CSS(2),02)),UNIT=OSA
IODEVICE ADDRESS=(2080,015),CUNUMBR=(2080),UNIT=OSA 1

Example 4-2 corresponds to what you must code in a VTAM TRLE definition in order to
support a QDIO connection of a TCP/IP stack. Look at Example 4-3, where you see that the
VTAM TRLE that defines port number 0 (A) (the only port number on an OSA-Express2)
utilizes only the first nine addresses (2080-2088) of the allocated fifteen addresses
(2080-208E) on this CNTLUNIT.

Example 4-3 TRLE definition for PORTNUM=0 (Portname of OSA2080)

OSA2080 VBUILD TYPE=TRL
OSA2080P TRLE LNCTL=MPC, *
READ=2080, *
WRITE=2081, *
DATAPATH=(2082-2088), *
PORTNAME=OSA2080, *
PORTNUM=0, A *
MPCLEVEL=QDIO

To add the OSA-Express3 port that resides at port number 1 of the same CHPID, we use the
same IOCDS as before, but now we add a new TRLE definition for PORTNUM=1 (B). See the
TRLE example in Example 4-4.

Example 4-4 TRLE definition for PORTNUM=1 (Portname of OSA2081)

OSA2081 VBUILD TYPE=TRL
OSA201P TRLE LNCTL=MPC,
READ=2089, C
WRITE=208A,
DATAPATH=(208B-208D),
PORTNAME=OSA2081,
PORTNUM=1, B
MPCLEVEL=QDIO

In Example 4-4, we have simply started the addresses for PORTNUM=1 at 2089 of the
IOCDS C.

142 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Figure 4-11 shows the allocation of all the device addresses across the two ports of an
OSA-Express 3 card.

OSA-E2: 2-Port Adapter OSA-E3: 4-Port Adapter

Port 0
CHPID x Port 0 CHPID x IODEVICE 2080 - 2088
IODEVICE 2080 - 2088 Port 1
IODEVICE 2089 - 208D

Port 1
CHPID y Port 0
CHPID y
Port 0

Figure 4-11 Allocation of device addresses across two ports of an OSA-Express3

As you saw in Example 4-2 on page 142, the IOCP definitions have no awareness of the OSA
adapter’s two ports and simply assign device addresses; the VTAM definition for z/OS does
care about the port numbers and maps the number to the addresses (Example 4-3 on
page 142 and Example 4-4 on page 142). This address allocation scheme worked well for us
because we did not have to reconfigure the IOCP for our test. Other schemes may work
better for you, particularly if you are consolidating OSA ports from separate CHPIDs onto the
same CHPID of a new OSA-Express3.

Note: Our examples show you how to point to the two different ports with the PORTNUM
parameter in a z/OS example. Other System z operating systems, such as z/VM, Linux®
on z, z/VSE™, or TPF, have similar coding parameters to allocate addresses to port
number 0 versus port number 1. See the appropriate operating system documentation for
those definitions.

Bear in mind that a migration to OSA-Express 3 can affect more than just the IOCDS. You
also have other types of definitions in the operating system and potentially in access methods
(like VTAM) to migrate. The more you can keep the definitions the same across migrations,
the easier and more efficient the migration to a new platform or release becomes. This is
where the next two alternatives can make a difference for you.

Alternative 2: Two IODEVICE ranges for two E3 ports on a single CHPID

An alternative to the coding scheme we just showed you in Example 4-2 on page 142,
Example 4-3 on page 142, and Example 4-4 on page 142 is to use a different address for
each of the two ports. Such a scheme might make problem determination and operator
procedures easier for you, as message displays very clearly show the distinction between the
two ports, although they reside on the same CHPID.

Chapter 4. Connectivity 143

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

In Example 4-5, you see a range of addresses starting with 1000 (A) and another range
starting with 2000 (B), and the VTAM definitions in Example 4-6 show that these addresses
are used for OSA-E3 port numbers 0 and 1. (Compare with 1 and 2 in Example 4-6.)

Example 4-5 Separate device ranges for separate OSA-Express3 ports

CNTLUNIT CUNUMBR=1000,PATH=((CSS(0),10)),UNIT=OSA
IODEVICE ADDRESS=(1000,032),CUNUMBR=(1000),UNIT=OSA (A)
IODEVICE ADDRESS=(10FE,001),CUNUMBR=(1000),UNIT=OSAD
IODEVICE ADDRESS=(2000,032),UNITADD=20,CUNUMBR=(1000),UNIT=OSA (B)

Example 4-6 VTAM definitions for OSA-E3 port numbers 0 and 1 (two device ranges)
OSA1000 VBUILD TYPE=TRL
OSA1000P TRLE LNCTL=MPC, *
READ=1000, *
WRITE=1001, *
DATAPATH=(1002), *
PORTNAME=OSA1000, *
PORTNUM=0, 1 *
MPCLEVEL=QDIO

OSA2000 VBUILD TYPE=TRL

OSA2000P TRLE LNCTL=MPC, *
READ=2000, *
WRITE=2001, *
DATAPATH=(2002), *
PORTNAME=OSA2000, *
PORTNUM=1, 2 *
MPCLEVEL=QDIO

The diagram in Figure 4-12 shows you how the device addresses are allocated for this
example.

OSA-E2: 2-Port Adapter OSA-E3: 4-Port Adapter

Port 0
IODEVICE 1000 - 101F
CHPID x Port 0 CHPID x
IODEVICE 1000 - 101F Port 1
IODEVICE 2000 - 201F

Port 1
CHPID y Port 0
CHPID y
IODEVICE 2000 - 201F
Port 0

Figure 4-12 Consolidating two OSA ports from OSA-E2 onto a single CHPID of OSA-E3

144 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

With this alternative, you can preserve the device addresses in your VTAM definitions and
simply deal with a few changes in the IOCDS. This might represent a simple migration
scenario for you if you have many VTAM definitions to change.

Alternative 3: Two Logical Control Units on Physical CU for two E3 ports

The following examples show you how to make a logical distinction within the IOCP between
ports 0 and 1 of an OSA-E3 CHPID. You can specify separate logical control units with the
CUADD parameter. While in the past defining multiple Control Units had value only if you
were defining many devices, it appears customers migrating from OSA-Express2 channels to
multi-port OSA-Express3s are finding it easier to combine two OSA-Express CHPIDs into the
two ports of an OSA-Express3 CHPID.

Refer to Example 4-7, where you find the device range for port number 0 under CUADD=0 (A)
and the device range for port number 1 under CUADD=1 (B).

Example 4-7 Separate logical control unit for each OSA-E3 port
CNTLUNIT CUNUMBR=3000,CUADD=0 A,PATH=((CSS(0),02),(CSS(1),02)),UNIT=OSA
IODEVICE ADDRESS=(3000,032),UNITADD=00,CUNUMBR=(3000),UNIT=OSA
IODEVICE ADDRESS=3020,UNITADD=FE,CUNUMBR=(3000),UNIT=OSAD
CNTLUNIT CUNUMBR=3500,CUADD=1 B,PATH=((CSS(0),02),(CSS(1),02)),UNIT=OSA
IODEVICE ADDRESS=(3500,032),UNITADD=00,CUNUMBR=(3500),UNIT=OSA

The VTAM definitions look similar to what you have seen before. Examine the coding in
Example 4-8.

Example 4-8 VTAM TRLEs for two logical control units and port numbers of an OSA-E3
OSA3000 VBUILD TYPE=TRL
OSA3000P TRLE LNCTL=MPC, *
READ=3000, *
WRITE=3001, *
DATAPATH=(3002), *
PORTNAME=OSA3000, *
PORTNUM=0, 1 *
MPCLEVEL=QDIO

OSA3500 VBUILD TYPE=TRL

OSA3500P TRLE LNCTL=MPC, *
READ=3500, *
WRITE=3501, *
DATAPATH=(3502), *
PORTNAME=OSA3500, *
PORTNUM=1, 2 *
MPCLEVEL=QDIO

Chapter 4. Connectivity 145

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

The device range beginning with 3000 has been assigned to port number 0 (1); the device
range starting with 3500 has been assigned to port number 1 (2). Refer to Figure 4-13.

OSA-E2: 2-Port Adapter OSA-E3: 4-Port Adapter

Port 0
IODEVICE 3000 - 301F
CHPID x Port 0 CHPID x CUADD=0 *
IODEVICE 3000 - 301F
Port 1
CUADD=0 (default) IODEVICE 3500 - 351F
CUADD=1

Port 1
CHPID y Port 0
CHPID y
IODEVICE 3500 - 351F
CUADD=0 (default) Port 0

* If you define multiple Control Units on a CHPID, you must

specify a CUADD keyword on all CNTLUNIT statements for
the CHPID. There is no default in this case.

Figure 4-13 Distinguishing OSA-E3 port numbers in the IOCDS with a CUADD parameter

Just as with the second alternative, you might find it easier to merge what were OSA
connections on two separate CHPIDs into a single CHPID and distinguish them not only with
separate address ranges, but also with separate logical control unit numbers.

Notes:
1. In all the IOCDS definitions we have shown you, we have coded the OSA/SF device on
CUADD=0, either by default or through explicit coding. The OSA/SF device must reside
on CUADD=0.
2. OSA supports Outbound Priority Queueing (multiple Outbound Queues) as long as no
more than 480 valid subchannels are defined for all LPARs sharing a CHPID. Each
logical partition sharing a CHPID gets a subchannel for every device defined on that
CHPID. Therefore, if you define a CHPID shared by 15 logical partitions and define 32
devices (either on one port or across two ports), you have used 480 valid subchannels
(15 * 32 = 480). If your definition requires more than 480 valid subchannels (with a
maximum of 1920), then the user must explicitly turn off Outbound Priority Queueing on
the CHPID definition by specifying CHPARM=02 in the IOCP or by specifying it in HCD.
HCD will prevent a device definition that will cause the 480 subchannel limit to be
broken. IOCP will issue an error message and not create an IOCDS if the limit is
broken.
3. If you need to define more than 254 devices for an unshared OSD channel path,
multiple control units must be defined. Specify a unique logical address for each control
unit using the CUADD keyword.

146 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Considerations for assigning the OSA portname

OSA Portname assignment for a QDIO implementation (CHPID Type of OSD) is important in
the z/OS operating system. The rule for assigning a portname is the same regardless of the
type of OSA adapter being implemented:

RULE: The portname of an OSA port must be unique on a CHPID.

This rule seems obvious, but you may find yourself confused when you contemplate a
migration from certain configurations of the OSA-Express2 to an implementation of a new
OSA-Express3. Consider Figure 4-14.

OSA-E2: 2-Port Adapter OSA-E3: 4-Port Adapter

Port 0
Portname GIG0x
CHPID x Port 0 CHPID x
Portname GIGx Port 1
Portname GIG1x
[Portname GIG0]

Port 1
CHPID y Port 0
CHPID y [Portname GIG0]
Portname GIGy Port 0
[Portname GIG0] [Portname GIG0]

On an OSA CHPID, the Portname value must On an OSA CHPID, the Portname value must
be unique to the CHPID. be unique to the CHPID.
This example depicts a single port per CHPID, This example depicts multiple ports per CHPID,
as in the design of an OSA-E2. as in the design of an OSA-E3.
The Portnames are not only unique to the The Portnames in the top of the visual are not
CHPID but also different from each other. only unique to the CHPID but also different
However, certain configurations would permit from each other: "GIG0x" and "GIG1x."
the Portnames to be the same as in "GIG0." No configuration can allow two OSA ports on
Example: If different VTAMs control the OSA the same CHPID to be assigned the same
TRLE definitions, the Portnames could be the Portname.
same (e.g., GIG0) across the two CHPIDs. Example: The Portnames in the bottom half
of the visual must bear unique portnames.
Otherwise, one port will fail to activate.

Figure 4-14 Providing unique portnames for OSA-Express ports

The figure shows that if you had attempted to move both ports named GIG0 to CHPIDy of the
OSA-E3, one port would not activate because the names are no longer unique to the CHPID.
The presence of duplicate names on the same CHPID would generate an SNA sense code of
8010311B.

4.4.2 Considerations for isolating traffic across a shared OSA port

VLANs, when properly implemented, can isolate traffic over a shared network and shared
OSA port. The isolation is complete if all TCP/IP stacks that share an OSA port implement
VLAN ID tagging and assign separate VLANIDs. For more information about this subject,
consult Chapter 6, “VLAN and Virtual MAC support” on page 265.

Chapter 4. Connectivity 147

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

Another method that is available to isolate traffic across a shared OSA port is OSA
Connection Isolation. This method can be deployed with or without out assigning a VLAN ID
or a VMAC to the OSA port. You can read more about this method in 4.5, “OSA-Express
QDIO connectivity with Connection Isolation” on page 156.

When planning connectivity for a LAN environment, there might not be a requirement to
isolate data traffic or services for certain servers or clients as we have shown in this scenario.
In such cases, VLAN IDs can be omitted.

If there is a requirement for VLANs, however, we recommend adding the VLAN IDs to your IP
addressing scheme to aid in the mapping of IP addresses to VLANs based on data traffic
patterns or access to resources.

Also, to simplify administration and management of VLANs, consider using Generic Attribute
VLAN Registration Protocol (GVRP) wherever possible. For details, refer to “VLAN support of
Generic Attribute Registration Protocol (GVRP)” on page 123.

4.4.3 Configuring OSA-Express with VLAN ID

To implement OSA-Express (QDIO) in our environment, we performed these tasks:
1. Verify the switch port configuration.
2. Define a TRLE in VTAM to represent each OSA-Express port.

In the TCP/IP profile:

1. Create DEVICE and LINK or INTERFACE statements for each OSA-Express port.
2. Create a HOME address to each defined LINK.
3. Define the characteristics of each LINK statement using BSDROUTINGPARMS. You can
code the BSDROUTINGPARMS statement even if you define the LINK characteristics in
OMPROUTE.

We explain these tasks in more detail in the following sections.

Verify the switch port configuration

It is important to be aware of the switch configuration and definitions to which the
OSA-Express ports will be connected. You will need confirm the following information:
 The switch ports to which the OSA-Express ports are connected.
Table 4-3 shows the OSA-Express and switch port assignment with VLAN IDs and mode
type in our configuration.

Table 4-3 OSA-Express and switch port assignment with VLAN IDs
OSA-Express port Connects to switch Switch port VLAN ID (mode)

CHPID 02 (2080) Switch 1 Interface GIGA 1/8 10 (Trunk mode)

CHPID 03 (20A0) Switch 1 interface GIGA 1/41 10 (Trunk mode)

CHPID 04 (20C0) Switch 1 Interface GIGA 1/43 11 (Trunk mode)

CHPID 05 (20E0) Switch 1 Interface GIGA 1/19 11 (Trunk mode)

148 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

 The IP subnetwork and mask.

We used the following:
– Subnetwork [Link], mask [Link] for VLAN 10
– Subnetwork [Link], mask [Link] for VLAN 11
 The appropriate switch ports should be defined in trunk mode, as shown in Example 4-9.

Example 4-9 Switch port definition from Switch 1 port 1/41

interface GigabitEthernet1/41
switchport
switchport trunk encapsulation dot1q
switchport mode trunk
no ip address

Define a TRLE in VTAM to represent each OSA-Express port

Each OSA-Express port must have a TRLE definition defined; see Example 4-10. The
PORTNAME 1 must match the device name of the DEVICE definition or the portname in
INTERFACE definition in the TCP/IP profile. The PORTNUM 2 operand is optional (default 0),
but required when defining the second port of an OSA-Express3. The statement MPCLEVEL
3 must be specified as QDIO.

Example 4-10 TRLE definition

OSA2080 VBUILD TYPE=TRL
OSA2080P TRLE LNCTL=MPC, *
READ=2080, *
WRITE=2081, *
DATAPATH=(2082-2088), *
1 PORTNAME=OSA2080, *
2 PORTNUM=0, *
3 MPCLEVEL=QDIO

For all OSA-Express ports in our scenarios, we used the following PORTNAMES:
 OSA2080
 OSA20A0
 OSA20C0
 OSA20E0

Create DEVICE and LINK or INTERFACE statements for each

OSA-Express port
The next step is to create the device and link or interface statements for each OSA-Express
port, as shown in Example 4-11 on page 150.

Note: We encourage and recommend the use of the INTERFACE statement, as it groups
all the definitions required in one spot, while DEVICE and LINK require that the IP address
be assigned in the HOME list.

The device definition of an OSA-Express port must be set as an MPCIPA device type 1. The
link definition describes the type of transport used (in our case, QDIO Ethernet, defined as
IPAQENET 2). VLAN ID 3 defines the VLAN number the packets will be tagged with as they
are being sent out to the switch.

Chapter 4. Connectivity 149

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

Note: You can only define a single VLAN per each OSA port with device and link
statement. If you want to define multiple VLANs on a single OSA port, you need to define it
with the interface statement.

Example 4-11 OSA-Express device and link definitions

;OSA DEFINITIONS
;TRL MAJ NODE: OSA2080,OSA20A0,OSA20C0,AND OSA20E0
DEVICE OSA2080 MPCIPA 1
LINK OSA2080L IPAQENET 2 OSA2080 VLANID 10 3
DEVICE OSA20C0 MPCIPA
LINK OSA20C0L IPAQENET OSA20C0 VLANID 11
DEVICE OSA20E0 MPCIPA
LINK OSA20E0L IPAQENET OSA20E0
DEVICE OSA20A0 MPCIPA
LINK OSA20A0L IPAQENET OSA20A0

The alternative interface statement of OSA-Express ports combines the definitions otherwise
coded in the device, link, home, beginroutes and bsdroutingparms statements, and as such
requires a label 1, the type of transport used (QDIO Ethernet, defined as IPAQENET 2 is the
only type allowed for IPv4 devices), a portname 3 matching the TRLE portname, an IP
address and optional subnetmask 4, optional MTU size 5, VLANID 6, VMAC 7 (required when
setting multiple VLANs on the same physical OSA port) and SOURCEVIPAINT 8 which
associates a specific VIPA with this interface.

Note: If SOURCEVIPAINT is coded, the whole INTERFACE definition block must be

defined in PROFILE after the VIPA DEVICE and LINK statements are defined.

Example 4-12 OSA-Express interface definition

INTERFACE OSA20A0I 1
DEFINE IPAQENET 2
PORTNAME OSA20A0 3
IPADDR [Link]/24 4
MTU 1492 5
VLANID 20 6
VMAC 7
SOURCEVIPAINT VIPA2L 8

150 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Create a HOME address to each defined LINK

If you are not implementing the connection with the INTERFACE statement, you must assign
an IP address to the LINK of each DEVICE/LINK pair. Each link configured must have its own
IP address configured on the HOME statement of the TCP/IP profile. Our OSA-Express ports
are defined with the IP addresses shown in Example 4-13.

Note: This step is not required when defining OSA ports through the INTERFACE
statement.

Example 4-13 OSA-Express HOME addresses

HOME
[Link] OSA2080L
[Link] OSA20C0L
[Link] OSA20E0L
[Link] OSA20A0L

Define the characteristics of each LINK statement using

BSDROUTINGPARMS
To define the link characteristics, such as MTU size 1 and subnet mask 2, we used the
BSDROUTINGPARMS statements (see Example 4-14).

Note: This step is not required when defining OSA ports through the INTERFACE
statement.

If not supplied, defaults are used from static routing definitions in BEGINROUTES or the
OMPROUTE configuration (dynamic routing definitions), if implemented.

If the link characteristics, BEGINROUTES statements, or the OMPROUTE configuration are

not defined, then the stack’s interface layer (based on hardware capabilities) and the
characteristics of devices and links are used. This, however, might not provide the
performance or function desired.

Example 4-14 BSDRoutingparms statements

BSDROUTINGPARMS TRUE
; Link name MTU Cost metric Subnet Mask Dest address
VIPA1L 1492 1 0 [Link] 0
OSA2080L 1492 0 [Link] 2 0
OSA20A0L 1492 0 [Link] 0
OSA20C0L 1492 0 [Link] 0
OSA20E0L 1492 0 [Link] 0
ENDBSDROUTINGPARMS

Note: Static and dynamic routing definitions will override or replace the link characteristics
defined through the BSDROUTINGPARMS statements. Refer to Chapter 5, “Routing” on
page 205 for more information about static and dynamic routing.

Chapter 4. Connectivity 151

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

4.4.4 Verifying the connectivity status

In this section, we verify the status of the OSA devices defined to the TCP/IP stack and
VTAM.

Verifying the device status in TCP/IP stack

To verify the status of all devices being activated in the TCP/IP stack we use the NETSTAT
command with the DEVLIST option, as shown in Example 4-15.

Example 4-15 Using command D TCPIP,TCPIPA,N,DEV to verify the Device status

D TCPIP,TCPIPA,N,DEV
........................................................................ Lines
deleted
INTFNAME: OSA2080I INTFTYPE: IPAQENET INTFSTATUS: READY
PORTNAME: OSA2080 DATAPATH: 2082 DATAPATHSTATUS: READY
CHPIDTYPE: OSD
SPEED: 0000001000
IPBROADCASTCAPABILITY: NO
VMACADDR: 02000C776873 VMACORIGIN: OSA VMACROUTER: LOCAL
ARPOFFLOAD: YES ARPOFFLOADINFO: YES
CFGMTU: 1492 ACTMTU: 1492
IPADDR: [Link]/24
VLANID: 10 VLANPRIORITY: DISABLED
DYNVLANREGCFG: NO DYNVLANREGCAP: YES
READSTORAGE: GLOBAL (4096K)
INBPERF: BALANCED
CHECKSUMOFFLOAD: YES SEGMENTATIONOFFLOAD: YES
SECCLASS: 255 MONSYSPLEX: NO
ISOLATE: NO OPTLATENCYMODE: NO
MULTICAST SPECIFIC:
MULTICAST CAPABILITY: YES
GROUP REFCNT SRCFLTMD
----- ------ --------
[Link] 0000000001 EXCLUDE
INTERFACE STATISTICS:
BYTESIN = 0
INBOUND PACKETS = 0
INBOUND PACKETS IN ERROR = 0
INBOUND PACKETS DISCARDED = 0
INBOUND PACKETS WITH NO PROTOCOL = 0
BYTESOUT = 168
OUTBOUND PACKETS = 2
OUTBOUND PACKETS IN ERROR = 0
OUTBOUND PACKETS DISCARDED = 0
........................................................................ Lines
deleted

Displaying TCP/IP device resources in VTAM

The device drivers for TCP/IP are provided by VTAM. When CS for z/OS IP devices are
activated, there must be an equivalent Transport Resource List Element (TRLE) defined to
VTAM. The devices that are exclusively used by z/OS Communications Server IP have
TRLEs that are automatically generated for them.

152 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Because the device driver resources are provided by VTAM, you have the ability to display the
resources using VTAM display [Link] display a list of all TRLEs active in VTAM, use
the command D NET,TRL, as shown in Example 4-16.

Example 4-16 D NET,TRL command output

D NET,TRL
IST350I DISPLAY TYPE = TRL 135
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = ISTTRL
IST1314I TRLE = IUTIQDF6 STATUS = ACTIV CONTROL = MPC
IST1314I TRLE = IUTIQDF5 STATUS = ACTIV CONTROL = MPC
IST1314I TRLE = IUTIQDF4 STATUS = ACTIV CONTROL = MPC
IST1314I TRLE = ISTT3033 STATUS = ACTIV CONTROL = XCF
IST1314I TRLE = ISTT3032 STATUS = ACTIV CONTROL = XCF
IST1314I TRLE = ISTT3031 STATUS = ACTIV CONTROL = XCF
IST1314I TRLE = IUTIQDIO STATUS = ACTIV CONTROL = MPC
IST1314I TRLE = IUTSAMEH STATUS = ACTIV CONTROL = MPC
IST1454I 8 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = TRLTONET
IST1314I TRLE = MPCNET STATUS = ACTIV CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA2000
IST1314I TRLE = OSA2000P STATUS = NEVAC CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA2020
IST1314I TRLE = OSA2020P STATUS = NEVAC CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA2080
IST1314I TRLE = OSA2080T STATUS = ACTIV CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA20A0
IST1314I TRLE = OSA20A0P STATUS = ACTIV CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA20A1
IST1314I TRLE = OSA20A1P STATUS = NEVAC CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA20C0
IST1314I TRLE = OSA20C0P STATUS = ACTIV CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA20E0
IST1314I TRLE = OSA20E0P STATUS = ACTIV CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA2100
IST1314I TRLE = OSA2100T STATUS = ACTIV CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------

Chapter 4. Connectivity 153

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

IST1954I TRL MAJOR NODE = OSA2120

IST1314I TRLE = OSA2120T STATUS = ACTIV CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST314I END

You can also display information of TRLEs grouped by control type, such as MPC or
XCF devices, as shown in Example 4-17.

Example 4-17 D NET,TRL,CONTROL=MPC

D NET,TRL,CONTROL=MPC
IST350I DISPLAY TYPE = TRL 276
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = ISTTRL
IST1314I TRLE = IUTIQDF6 STATUS = ACTIV CONTROL = MPC
IST1314I TRLE = IUTIQDF5 STATUS = ACTIV CONTROL = MPC
IST1314I TRLE = IUTIQDF4 STATUS = ACTIV CONTROL = MPC
IST1314I TRLE = IUTIQDIO STATUS = ACTIV CONTROL = MPC
IST1314I TRLE = IUTSAMEH STATUS = ACTIV CONTROL = MPC
IST1454I 5 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = TRLTONET
IST1314I TRLE = MPCNET STATUS = ACTIV CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA2000
IST1314I TRLE = OSA2000P STATUS = NEVAC CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA2020
IST1314I TRLE = OSA2020P STATUS = NEVAC CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA2080
IST1314I TRLE = OSA2080T STATUS = ACTIV CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA20A0
IST1314I TRLE = OSA20A0P STATUS = ACTIV CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA20A1
IST1314I TRLE = OSA20A1P STATUS = NEVAC CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA20C0
IST1314I TRLE = OSA20C0P STATUS = ACTIV CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA20E0
IST1314I TRLE = OSA20E0P STATUS = ACTIV CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA2100
IST1314I TRLE = OSA2100T STATUS = ACTIV CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED

154 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = OSA2120
IST1314I TRLE = OSA2120T STATUS = ACTIV CONTROL = MPC
IST1454I 1 TRLE(S) DISPLAYED
IST314I END

We can also get specific information about a single TRLE, using the TRLE name as
shown in Example 4-18, for an OSA-Express device.

Example 4-18 D NET,TRL,TRLE=OSA2080T

D NET,TRL,TRLE=OSA2080T
IST075I NAME = OSA2080T, TYPE = TRLE 336
IST1954I TRL MAJOR NODE = OSA2080
IST486I STATUS= ACTIV, DESIRED STATE= ACTIV
IST087I TYPE = LEASED , CONTROL = MPC , HPDT = YES
IST1715I MPCLEVEL = QDIO MPCUSAGE = SHARE
IST2263I PORTNAME = OSA2080 PORTNUM = 0 OSA CODE LEVEL = 000C
IST2337I CHPID TYPE = OSD CHPID = 02
IST1577I HEADER SIZE = 4096 DATA SIZE = 0 STORAGE = ***NA***
IST1221I WRITE DEV = 2081 STATUS = ACTIVE STATE = ONLINE
IST1577I HEADER SIZE = 4092 DATA SIZE = 0 STORAGE = ***NA***
IST1221I READ DEV = 2080 STATUS = ACTIVE STATE = ONLINE
IST924I ------------------------------------------------------------
IST1221I DATA DEV = 2082 STATUS = ACTIVE STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST1717I ULPID = TCPIPA
IST2309I ACCELERATED ROUTING ENABLED
IST2331I QUEUE QUEUE READ
IST2332I ID TYPE STORAGE
IST2205I ------ -------- ---------------
IST2333I RD/1 PRIMARY 4.0M(64 SBALS)
IST2305I NUMBER OF DISCARDED INBOUND READ BUFFERS = 0
IST1757I PRIORITY1: UNCONGESTED PRIORITY2: UNCONGESTED
IST1757I PRIORITY3: UNCONGESTED PRIORITY4: UNCONGESTED
IST2190I DEVICEID PARAMETER FOR OSAENTA TRACE COMMAND = 01-01-00-02
IST1801I UNITS OF WORK FOR NCB AT ADDRESS X'0F512010'
IST1802I P1 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P2 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P3 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P4 CURRENT = 0 AVERAGE = 2 MAXIMUM = 2
IST924I -----------------------------------------------------------
IST1221I DATA DEV = 2083 STATUS = ACTIVE STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST1717I ULPID = TCPIPC
IST2309I ACCELERATED ROUTING ENABLED
IST2331I QUEUE QUEUE READ
IST2332I ID TYPE STORAGE
IST2205I ------ -------- ---------------
IST2333I RD/1 PRIMARY 4.0M(64 SBALS)
IST2305I NUMBER OF DISCARDED INBOUND READ BUFFERS = 0
IST1757I PRIORITY1: UNCONGESTED PRIORITY2: UNCONGESTED
IST1757I PRIORITY3: UNCONGESTED PRIORITY4: UNCONGESTED
IST2190I DEVICEID PARAMETER FOR OSAENTA TRACE COMMAND = 01-01-00-03
IST1801I UNITS OF WORK FOR NCB AT ADDRESS X'0FCF0010'
IST1802I P1 CURRENT = 0 AVERAGE = 1 MAXIMUM = 2

Chapter 4. Connectivity 155

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

IST1802I P2 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0

IST1802I P3 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P4 CURRENT = 0 AVERAGE = 1 MAXIMUM = 1
IST924I -----------------------------------------------------------
IST1221I DATA DEV = 2084 STATUS = ACTIVE STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST1717I ULPID = TCPIPB
IST2310I ACCELERATED ROUTING DISABLED
IST2331I QUEUE QUEUE READ
IST2332I ID TYPE STORAGE
IST2205I ------ -------- ---------------
IST2333I RD/1 PRIMARY 4.0M(64 SBALS)
IST2305I NUMBER OF DISCARDED INBOUND READ BUFFERS = 0
IST1757I PRIORITY1: UNCONGESTED PRIORITY2: UNCONGESTED
IST1757I PRIORITY3: UNCONGESTED PRIORITY4: UNCONGESTED
IST2190I DEVICEID PARAMETER FOR OSAENTA TRACE COMMAND = 01-01-00-04
IST1801I UNITS OF WORK FOR NCB AT ADDRESS X'0F03F010'
IST1802I P1 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P2 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P3 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P4 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST924I -----------------------------------------------------------
IST1221I DATA DEV = 2085 STATUS = RESET STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I -----------------------------------------------------------
IST1221I DATA DEV = 2086 STATUS = RESET STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I -----------------------------------------------------------
IST1221I DATA DEV = 2087 STATUS = RESET STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I -----------------------------------------------------------
IST314I END

4.5 OSA-Express QDIO connectivity with Connection Isolation

Many customers share OSA-Express ports across logical partitions, especially if capacity is
not an issue. Each stack sharing the OSA port registers certain IP addresses and multicast
groups with the OSA.

Note: You may wish to revisit the discussion of IPv4 address registration in “OSA-Express
QDIO IPv4 address registration” on page 121.

For performance reasons, the OSA-Express bypasses the LAN and routes packets directly
between the stacks when possible. Examine Figure 4-15, where you see two TCP/IP stacks,
TCPIPA and TCPIPB, which share the same OSA port connected to subnet [Link]/24.

156 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

SC30 SC31

TCPIPA TCPIPB
PROFA30X PROFB31X

VIPA1L [Link]/24 VIPA1L [Link]/24

OSA2080X [Link]/24 OSA2080X [Link]/24

QDIO OSA
[Link]/24
[Link] [Link]
A
[Link] [Link]
[Link] [Link]
B

C C

Switch
Router

IP Network

Figure 4-15 Routing paths over an OSA port

For performance reasons, the OSA-Express bypasses the LAN and routes packets directly
between the stacks when possible. For unicast packets, OSA internally routes the packet
when the next-hop IP address is registered on the same LAN or VLAN by another stack
sharing the OSA port. Figure 4-15 illustrates examples of this action, where:
 A: You see how TCPIPA routes a packet to [Link] in TCPIPB over the OSA port without
exiting out onto the LAN because the next hop to reach the destination is registered in the
OSA Address Table (OAT); the TCPIPA routing table indicates that the destination can be
reached by hopping through the direct connection to the [Link]/24 network.
 B For multicast (for example, OSPF protocol packets), OSA internally routes the packet to
all sharing stacks on the same LAN or VLAN that registered the multicast group. Note how
TCPIPA and TCPIPB have each registered multicast addresses for OSP
(224.000.000.00n) in the OSA port.
 C OSA also sends the multicast/broadcast packet to the LAN. For broadcast (not
depicted), OSA internally routes the packet to all sharing stacks on the same LAN or
VLAN.

Thus, you see that stacks sharing an OSA-Express port can communicate over the OSA.
Some customers may express concerns about this efficient communication path and wish to
disable it because traffic flowing internally through the OSA adapter bypasses any security
features implemented on the external LAN. For example, the customer may have exploited
the virtualization features of the System z and of 10 Gigabit OSA adapters to build a
demilitarized zone (DMZ) on several LPARs of a System z as well as several production

Chapter 4. Connectivity 157

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

LPARs on the same System z footprint. Although they can implement firewall and Intrusion
Detection technologies within the LPARs to isolate the two zones (DMZ and Production) from
each other, they may have already invested in external security mechanisms on the LAN. If
traffic through a shared OSA bypasses the security on the LAN, they need to find a way to
prevent the internal routing across the shared OSA path.

Several network designs are available to provide isolation and force the traffic to bypass the
shared OSA path or to be prevented from using it:
 Implement IP filtering on the stacks in the adjacent zones by exploiting z/OS Policy Agent
with IP filtering and Intrusion Detection Services (IDS).
 Implement routing filters that block the advertisement of certain routing zones to parts of
the network from which they should remain concealed. Examples of such features are
OSPF range checking, RIP, or EIGRP routing filters.
 Implement Policy Based Routing (PBR) to eliminate the internal OSA path where it is not
desired.
 Define static routes so that paths to a stack sharing the OSA are forced to hop through a
router on the LAN.
 Configure the TCP/IP stacks in separate zones (IP subnets) with separate VLANs that
extend into the stacks themselves.
 Implement OSA Connection Isolation.

4.5.1 Description of Connection Isolation

Some environments require strict controls for routing data traffic between servers or nodes. In
certain cases, the LPAR-to-LPAR capability of a shared OSA port can prevent such controls
from being enforced. For example, you may need to ensure that traffic flowing through the
OSA adapter does not bypass firewalls or intrusion detection systems implemented on the
external LAN. We have described several ways to isolate traffic from different LPARs on a
shared OSA port, with one of these methods being OSA Connection Isolation.

The feature is called OSA Connection Isolation in z/OS, but it is also available in z/VM, where
it is called QDIO data connection isolation or VSWITCH port isolation. It allows you to
disable the internal routing on a QDIO connection basis, providing a means for creating
security zones and preventing network traffic between the zones. It also provides extra
insurance against a misconfiguration that might otherwise allow such traffic to flow as in the
case of an incorrectly defined IP filter. With interface isolation, internal routing can be
controlled on an LPAR basis. When interface isolation is enabled, the OSA will discard any
packets destined for a z/OS LPAR that is registered in the OAT as isolated.

4.5.2 Dependencies for Connection Isolation

QDIO interface isolation is supported by Communications Server for z/OS V1R12 and all
OSA-Express3 and OSA-Express2 features on System z10, and by all OSA-Express2
features on System z9, with an MCL update. Refer to the appropriate Preventive Service
Planning bucket for details regarding your System z server.

Coding ISOLATE on your INTERFACE statement enables the function. It tells the
OSA-Express not to allow communications to this stack other than over the LAN.
OSA-Express requires that both stacks sharing the port be non-isolated for direct routing to
occur.

158 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Because this function is specific to security, an OSA-Express interface that does not support
the Connection Isolation function cannot be activated. Examine the messages at 1 and 2 in
Example 4-19 which show an unsuccessful activation attempt for a QDIO interface whose
OSA does not support the ISOLATE function that was coded on it.

Example 4-19 Failure to activate an OSA interface that does not support the ISOLATE feature
V TCPIP,TCPIPF,START,OSA2080X
EZZ0060I PROCESSING COMMAND: VARY TCPIP,,START,OSA2080X
EZZ0053I COMMAND VARY START COMPLETED SUCCESSFULLY
EZD0022I INTERFACE OSA2080X DOES NOT SUPPORT THE ISOLATE FUNCTION 1
EZZ4341I DEACTIVATION COMPLETE FOR INTERFACE OSA2080X 2

To eliminate the ISOLATE specification on the device so that you can successfully activate it,
you must first STOP the interface before using the V TCPIP,,OBEYFILE command to modify
the ISOLATE parameter.

4.5.3 Considerations for Connection Isolation

When Connection Isolation is in effect on either or both endpoints of a connection on a shared
OSA port, OSA-Express will discard any packets when the next-hop address is registered in
the OSA by a sharing stack, that is, OSA discards unicast packets which previously qualified
for internal routing. It also ceases to route internal multicast or broadcast packets. It does,
however, continue to send the multicast or broadcast packets to the LAN. OSA-Express
requires that both stacks sharing the port be non-isolated for direct routing to occur.

If you have implemented static routing where Connection Isolation is in effect, it is simple to
code the appropriate routing statements to bypass the direct path through the OSA. If you are
running a dynamic routing protocol, you may see routing errors when the routing protocol
attempts to send packets over the ISOLATED OSA port. Such errors are “working as
designed” when ISOLATION has been introduced into the configuration.

Dynamic routing protocol implementations with RIP or OSPF require careful planning on
LANs where OSA-Express connection isolation is in effect; the dynamic routing protocol
learns of the existence of the direct path but is unaware of the isolated configuration, which
renders the direct path across the OSA port to the registered target unusable. If the direct
path that is operating as ISOLATEd is selected, you will experience routing failures.

If the visibility of such errors is undesirable, you can take other measures to avoid the failure
messages. If you are simply attempting to bypass the direct route in favor of another, indirect
route, you can accomplish this as well with some thoughtful design.

For example, you might purposely bypass the direct path by using Policy Based Routing
(PBR) or by coding static routes that supersede the routes learned by the dynamic routing
protocol. You might adjust the weights of connections to favor alternate interfaces over the
interfaces that have been coded with ISOLATE.

Chapter 4. Connectivity 159

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

Static routes to override the direct OSA path

Examine the sample network diagram in Figure 4-16.

SC30 SC31

TCPIPA TCPIPB
PROFA30X PROFB31X

VIPA1L [Link]/24 VIPA1L [Link]/24

OSA2080X [Link]/24 OSA2080X [Link]/24

ISOLATE [ISOLATE]
QDIO OSA
[Link]/24
[Link] [Link]
1
[Link]
[Link] X [Link]
[Link]

2
Switch
Router

1 = Direct route between TCPIPA and TCPIPB

2 = Indirect route between TCPIPA and TCPIPB
Figure 4-16 Routing TCPIPA and TCPIPB: block direct OSA path and multi-hop static route

In Figure 4-16 you see two TCP/IP stacks: TCPIPA and TCPIPB. They share an OSA port on
network [Link]/24. Both stacks are running a dynamic routing protocol that informs them
that there is a direct path (1) through the OSA port between each other. The routing protocol
knows nothing of the ISOLATE function that was introduced to prevent packets from using the
direct route. (ISOLATE must be coded on only one of the two TCP/IP stacks, although you
can code it on both in this diagram.)

Another path between the two TCP/IP stacks is available through an external, next-hop router
(2). However, the dynamic routing protocol does not apprise the TCP/IP stack of this route’s
existence because it is not the shortest path. Therefore, when a packet is sent from TCPIPA
to TCPIPB, the stack’s routing table will always try to send that packet through the shortest
path; the send will not be successful because the stacks have been ISOLATEd from each
other over the OSA port.

If TCPIPA and TCPIPB are not to communicate with each other at all, then there is no need to
alter the appearance of the existing routing table. A route failure in this instance could be
desirable. In order to produce a message that explains that the two endpoints are ineligible for
routing to each other at all, you would probably introduce an IP filter. (Note that the routing
failure itself has no failure message that indicates that ISOLATE is at fault.)

If, however, TCPIPA and TCPIPB do need to exchange information, you will need to deploy an
effective route that bypasses the direct route between them. Therefore, at TCPIPA, you might
add a non-replaceable static route to an IP address in TCPIPB; the static route in the

160 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

BEGINROUTES block points to the next-hop router on the path indicated with (2) in
Figure 4-16 on page 160.

The effect of ICMP redirect packets

To avoid the override of the ICMP redirect packets that would most likely occur from the router
to the originating host, you need to disable the receipt of ICMP redirects in the IP stacks or
disable ICMP redirects at the router. If you are using OMPROUTE, ICMP redirects are
automatically disabled, as evidenced by the message that appears during OMPROUTE
initialization:
EZZ7475I ICMP WILL IGNORE REDIRECTS DUE TO ROUTING APPLICATION BEING ACTIVE

An alternate path that is more desirable than the direct path

If you do choose to have two TCP/IP stacks that are sharing an OSA port communicate with
each other, and if you do not wish to introduce the static routes just described, another
alternative is to provide another path that has, for example, a lower weighted path. See the
diagram in Figure 4-17.

Cost = 90 2 Cost = 90

SC30 SC31

TCPIPA TCPIPB
PROFA30X PROFB31X

VIPA1L [Link]/24 VIPA1L [Link]/24

OSA2080X [Link]/24 OSA2080X [Link]/24

ISOLATE QDIO OSA [ISOLATE]

[Link]/24
[Link] [Link]
1
[Link]
[Link] X [Link]
[Link]
Cost = 100 Cost = 100
1 = Direct route between TCPIPA and TCPIPB
2 = Lower cost route between TCPIPA, TCPIPB
Figure 4-17 Routing TCPIPA and TCPIPB: block direct path provides a lower-cost alternate path

Chapter 4. Connectivity 161

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

In Figure 4-17 on page 161 you see a lower-cost route at (2). The dynamic routing protocol
continues to run, but now the favored route is the one over HiperSockets, XCF, CTC, or over
an alternate LAN connection. Although the dynamic routing protocol continues its awareness
of the direct OSA path, it prefers the path at (2).

Altering the routing table with Policy-Based Routing (PBR)

You may also wish to deploy Policy-Based Routing in order to bypass direct routes. Refer to
Chapter 4, “Policy Agent”, in IBM z/OS V1R12 Communications Server TCP/IP
Implementation Volume 4: Security, SG24-7899 for more information about how to
accomplish this task.

4.5.4 Configuring OSA-Express with Connection Isolation

In Figure 4-18, you see the test network that will be the basis of our testing for OSA Express
Connection Isolation. This diagram shows you how we have depicted the shared OSA
environment in other manuals in this series.

SC30 SC31 SC32 SC33

TCPIPA TCPIPB TCPIPC TCPIPD
PROFA30X PROFB31X PROFC32X PROFD33X

VIPA1L [Link]/24 VIPA1L [Link]/24 VIPA1L [Link]/24 VIPA1L [Link]/24

OSA2080X [Link]/24 OSA2080X [Link]/24 OSA2080X [Link]/24 OSA2080X [Link]/24

OSA20A0X [Link]/24 OSA20A0X [Link]/24 OSA20A0X [Link]/24 OSA20A0X [Link]/24
OSA20C0X [Link]/24 OSA20C0X [Link]/24 OSA20C0X [Link]/24 OSA20C0X [Link]/24
OSA20E0X [Link]/24 OSA20E0X [Link]/24 OSA20E0X [Link]/24 OSA20E0X [Link]/24

Static Routes DYNAMICXCF 10.1.7.x1: HiperSockets, XCF, EZASAMEMVS

CHPID 02 CHPID 03 OSA-Express 1000BASE-T CHPID 04 CHPID 05

OSA2080 OSA20A0 OSA20C0 OSA20E0
10.1.2.x1 10.1.2.x2 10.1.3.x1 10.1.3.x2
2080-208F 20A0-20AF OSPF Routes 20C0-20CF 20E0-2E0F
Totally Stubby Area
TRUNK MODE TRUNK MODE TRUNK MODE TRUNK MODE

VLAN 10 VLAN 11
[Link] [Link]

SWITCH

Figure 4-18 Stacks started with test profiles PROFA30X, B31X, C32X, and D33X

All of the System z TCP/IP stacks are members of an OSPF Totally Stubby Network. Note
that the TCP/IP profiles at each stack are named PROFA30X, PROFB31X, PROFC32X, and
PROFD33X. Each stack shares each of the four OSA ports depicted. In VLAN 10 and on
Subnet [Link]/24, you see two OSA ports on each stack: OSA2080 and OSA20A0. In
VLAN 11 and on Subnet [Link]/24, you see two OSA ports on each stack: OSA20C0 and
OSA20E0. Each stack also has a static VIPA in subnet [Link]/24. The OSA and VIPA
interfaces are all advertised with OSPF protocols. However, the connections implemented
with the DYNAMICXCF keyword use only static routing.

162 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Examine the revised visual in Figure 4-19. It attempts to depict in a clearer fashion than
Figure 4-18 on page 162 how the OSA ports are shared across the four LPARs. Each TCP/IP
stack has two connections into subnet [Link]/24 and two into subnet [Link]/24.

SC30 SC31 SC32 TCPIPD

SC33
PROFD33X
TCPIPA TCPIPB TCPIPC
PROFA30X PROFB31X PROFC32X VIPA1L
[Link]/24
VIPA1L [Link]/24 VIPA1L [Link]/24 VIPA1L [Link]/24 OSA2080X
[Link]/24
OSA2080X [Link]/24 OSA2080X [Link]/24 OSA2080X [Link]/24 OSA20A0X
OSA20A0X [Link]/24 OSA20A0X [Link]/24 OSA20A0X [Link]/24 [Link]/24
OSA20C0X [Link]/24 OSA20C0X [Link]/24 OSA20C0X [Link]/24 OSA20C0X
OSA20E0X [Link]/24 OSA20E0X [Link]/24 OSA20E0X [Link]/24 [Link]/24
OSA20E0X
[Link]/24

[Link] [Link] CHPID 02 [Link] [Link]

OSA2080
Registered: 10.1.2.x1
2080-208F
communication path
Next-hop through OSA port
VMACs/VLANID 10

CHPID 03
[Link] [Link] OSA20A0 [Link] [Link]
10.1.2.x2
Registered: 20A0-20AF communication path
Next-hop VMACs/VLANID 10 through OSA port

CHPID 04
[Link] [Link] [Link] [Link]
OSA20C0
10.1.3.x1
Registered: 20C0-20CF communication path
Next-hop VMACs/VLANID 11 through OSA port

[Link] [Link] CHPID 05 [Link] [Link]

OSA20E0
10.1.3.x2
Registered: 20E0-2E0F
communication path
Next-hop VMACs/VLANID 11 through OSA port

Figure 4-19 OAT entries for the stacks sharing the four OSA ports

The revised diagram shows you how stacks communicate with each other over the shared
OSA ports when the next-hop router IP address is registered in the OSA. For performance
reasons, the OSA-Express bypasses the LAN and routes packets directly between the stacks
when possible.

4.5.5 Verifying Connection Isolation on OSA2080X

In this section, we discuss verifying Connection Isolation on OSA2080X.

Scenario for testing

To simplify the testing of the connection isolation scenario, we started only the connections to
the OSA on CHPID 2. We then modified the existing configuration to implement OSA
Connection Isolation only on TCPIPA and TCPIPB on CHPID 2. Connection Isolation was not
exploited on CHPID 2 for TCPIPC and TCPIPD.

Chapter 4. Connectivity 163

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

The new configuration is depicted in Figure 4-20.

SC30 SC31 SC32 SC33

TCPIPA TCPIPB TCPIPC TCPIPD
PROFA30A PROFB31A PROFC32A PROFD33A

VIPA1L [Link]/24 VIPA1L [Link]/24 VIPA1L [Link]/24 VIPA1L [Link]/24

OSA2080 [Link]/24 OSA2080 [Link]/24 OSA2080 [Link]/24 OSA2080 [Link]/24

IUTIQDF4L [Link]/24 IUTIQDF4L [Link]/24 IUTIQDF4L [Link]/24 IUTIQDF4L [Link]/24

CHPID 02
OSA2080
X 10.1.2.x1

X 2080-208F

X
TRUNK MODE

VLAN 10
[Link]

SWITCH

Figure 4-20 OSA2080 shared across four TCP/IP Images

Notice in Figure 4-20 the communication paths that we have indicated with an X. In our
testing, we will not permit TCPIPA or TCPIPB to be reached directly over the shared OSA
port. Using the ISOLATE function, we prevented direct communication between TCPIPA and
TCPIPB by way of this port; we also prevented direct communication between either TCPIPA
or TCPIPB and either of the two remaining stacks in our configuration: TCPIPC and TCPIPD.

We continued to permit TCPIPC and TCPIPD to share the OSA path between each other.

Note: You might choose to design your OSA ISOLATE function so that no sharing TCP/IP
stack may use the direct path through the OSA. On the other hand, if you have abundant
bandwidth on the OSA port, you might choose to implement ISOLATE on only selected
sharing TCP/IP stacks, as we have done in our test.

164 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Coding ISOLATE on the INTERFACE statements

The ISOLATE keyword can be coded only on an INTERFACE statement. The coding for
TCPIPA and for TCPIPB is displayed at 1 and 2 in Example 4-20.

Example 4-20 ISOLATE coding on CHPID2 (OSA2080X) for PROFA30X and PROFB31X
INTERFACE OSA2080X
DEFINE IPAQENET
PORTNAME OSA2080
IPADDR [Link]/24
MTU 1492
VLANID 10
VMAC ROUTEALL
ISOLATE 1

INTERFACE OSA2080X
DEFINE IPAQENET
PORTNAME OSA2080
IPADDR [Link]/24
MTU 1492
VLANID 10
VMAC ROUTEALL
ISOLATE 2

The definitions for the interface in stacks TCPIPC and TCPIPD contain NOISOLATE, which is
also the default. See 3 and 4 in Example 4-21.

Example 4-21 NOISOLATE coding on CHPID2 (OSA2080X) for PROFC32X and PROFD33X
INTERFACE OSA2080X
DEFINE IPAQENET
PORTNAME OSA2080
IPADDR [Link]/24
MTU 1492
VLANID 10
VMAC ROUTEALL
NOISOLATE 3

INTERFACE OSA2080X
DEFINE IPAQENET
PORTNAME OSA2080
IPADDR [Link]/24
MTU 1492
VLANID 10
VMAC ROUTEALL
NOISOLATE 4

Displaying the DEVICE to verify that ISOLATE is enabled

A display of all the INTERFACEs on which we coded ISOLATE shows that ISOLATE is in
force (A), as shown in Example 4-22 on page 166.

Chapter 4. Connectivity 165

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

Example 4-22 ISOLATE coding on the Interface definition

INTFNAME: OSA2080X INTFTYPE: IPAQENET INTFSTATUS: READY
PORTNAME: OSA2080 DATAPATH: 2082 DATAPATHSTATUS: READY
SPEED: 0000001000
IPBROADCASTCAPABILITY: NO
VMACADDR: 020004749925 VMACORIGIN: OSA VMACROUTER: ALL
ARPOFFLOAD: YES ARPOFFLOADINFO: YES
CFGMTU: 1492 ACTMTU: 1492
IPADDR: [Link]/24
VLANID: 10 VLANPRIORITY: DISABLED
DYNVLANREGCFG: NO DYNVLANREGCAP: YES
READSTORAGE: GLOBAL (4096K) INBPERF: BALANCED
CHECKSUMOFFLOAD: YES
SECCLASS: 255 MONSYSPLEX: NO
ISOLATE: YES A OPTLATENCYMODE: NO

Viewing OSA/SF for ISOLATE enablement

The OSA/SF display shows where ISOLATE is enabled, as you can see in Example 4-23.
Look for the entries marked with X.

Example 4-23 OSA/SF and ISOLATE

************************************************************************
*** OSA/SF Get OAT output created [Link] on 11/02/2009 ***
*** IOACMD APAR level - OA26486 ***
*** Host APAR level - OA27643 ***
************************************************************************
*** Start of OSA address table for CHPID 02 ***
************************************************************************
* UA(Dev) Mode Port Entry specific information Entry Valid
************************************************************************
Image 2.3 (A11 ) CULA 0
80(2080)* MPC N/A OSA2080 (QDIO control) SIU ALL
82(2082) MPC 00 No4 No6 OSA2080 (QDIO data) Isolated X SIU ALL
VLAN 10 (IPv4)

Group Address Multicast Address

01005E000001 [Link]
01005E000005 [Link]

VMAC IP address
HOME 020005749925 [Link]
...
************************************************************************
Image 2.4 (A24 ) CULA 0
80(2080)* MPC N/A OSA2080 (QDIO control) SIU ALL
82(2082) MPC 00 No4 No6 OSA2080 (QDIO data) Isolated X SIU ALL
VLAN 10 (IPv4)

Group Address Multicast Address

01005E000001 [Link]
01005E000005 [Link]

VMAC IP address
HOME 020004749925 [Link]
...
************************************************************************

166 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Viewing routes with ISOLATE in effect

We recycled all four stacks that were sharing the OSA port named OSA2080. We are going to
test connectivity between [Link] at TCPIPA and [Link], 31, and 33 at the other three
stacks. We also want to test connectivity between [Link] and the VIPAs in [Link] at the
other three stacks.

First, we examine the routing table at TCPIPA to determine if we have routes that will take us
to those destinations, as shown in Example 4-24.

Example 4-24 View of routing at TCPIPA prior to adding static routes

D TCPIP,TCPIPA,N,ROUTE
IPV4 DESTINATIONS
DESTINATION GATEWAY FLAGS REFCNT INTERFACE
DEFAULT [Link] UGO 0000000000 OSA2080I
DEFAULT [Link] UGO 0000000000 OSA20A0I
[Link]/32 [Link] UH 0000000000 VIPA1L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000001 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 OSA2080I
[Link]/32 [Link] UGHO 0000000000 OSA2080I
[Link]/32 [Link] UGHO 0000000000 OSA20A0I
[Link]/32 [Link] UGHO 0000000000 OSA20A0I
[Link]/24 [Link] UO 0000000000 OSA20A0I
[Link]/24 [Link] UO 0000000000 OSA2080I
[Link]/32 [Link] UH 0000000000 VIPA2L
[Link]/32 [Link] UH 0000000000 OSA2080I
[Link]/32 [Link] UH 0000000000 OSA20A0I
[Link]/32 [Link] H 0000000000 OSA2081I
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/24 [Link] UO 0000000000 OSA20E0I
[Link]/32 [Link] UH 0000000000 OSA20C0I
[Link]/32 [Link] UH 0000000000 OSA20E0I
[Link]/24 [Link] UO 0000000000 IUTIQDF4L
[Link]/32 [Link] UH 0000000000 IUTIQDF4L
[Link]/24 [Link] UO 0000000000 IUTIQDF5L
[Link]/32 [Link] UH 0000000000 IUTIQDF5L
[Link]/24 [Link] UGO 0000000000 IUTIQDF4L
[Link]/24 [Link] UGO 0000000000 IUTIQDF5L
[Link]/32 [Link] UH 0000000000 IUTIQDF6L
[Link]/24 [Link] US 0000000000 IQDIOLNK0A01070

Chapter 4. Connectivity 167

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

B
[Link]/32 [Link] UH 0000000000 EZASAMEMVS
[Link]/32 [Link] UH 0000000000 IQDIOLNK0A01070
B
[Link]/32 [Link] UHS 0000000000 IQDIOLNK0A01070
B
[Link]/32 [Link] UHS 0000000000 EZASAMEMVS
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UH 0000000000 VIPL0A010817
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF4L
[Link]/32 [Link] UGHO 0000000000 IUTIQDF5L
[Link]/32 [Link] UGHO 0000000000 OSA2080I
[Link]/32 [Link] UGHO 0000000000 OSA2080I
[Link]/32 [Link] UGHO 0000000000 OSA20A0I
[Link]/32 [Link] UGHO 0000000000 OSA20A0I
[Link]/32 [Link] UH 0000000000 VIPA3L
[Link]/24 [Link] UGO 0000000002 OSA2080I
[Link]/24 [Link] UGO 0000000001 OSA20A0I
[Link]/32 [Link] UH 0000000005 LOOPBACK
[Link]/24 [Link] UGO 0000000000 OSA2080I
[Link]/24 [Link] UGO 0000000000 OSA20A0I
[Link]/32 [Link] UGHO 0000000000 OSA2080I
[Link]/32 [Link] UGHO 0000000000 OSA20A0I
[Link]/24 [Link] UGO 0000000000 OSA2080I
[Link]/24 [Link] UGO 0000000000 OSA20A0I
[Link]/24 [Link] UGO 0000000000 OSA2080I
[Link]/24 [Link] UGO 0000000000 OSA20A0I
IPV6 DESTINATIONS
DESTIP: ::1/128
GW: ::
INTF: LOOPBACK6 REFCNT: 0000000000
FLGS: UH MTU: 65535
86 OF 86 RECORDS DISPLAYED
END OF THE REPORT

168 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Test 1: effect of ISOLATE with basic dynamic routing table

Our first test uses a limited routing table, where there is only one path available among the
four TCP/IP stacks, and that path is blocked with the ISOLATE keyword at TCPIPA and
TCPIPB. We execute all the following commands at TCPIPA, where ISOLATE is coded.

First, we use traceroute against the three target addresses in network [Link]/24, as shown
in Example 4-25.

Example 4-25 traceroute from TCPIPA to native OSA Home address of TCPIPB
===> tracerte [Link] (tcp tcpipa V srcip [Link] Intf OSA2080X

CS V1R12: Traceroute to [Link] ([Link]):

1 * * *

The results are the same when trying to reach TCPIPC and TCPIPD from either TCPIPA or
TCPIPB: Because the route table indicates a direct path through the OSA, the stack attempts
to send the packet over the direct route and experiences a failure. This is what we expected
because we coded ISOLATE on OSA2080X in TCPIPA (and TCPIPB). Can we reach the
VIPAs over the OSA port that is indicated as a route in Example 4-24 on page 167? We issue
a traceroute to the VIPAs and discover that the available routes will not allow us to reach
them. See Example 4-26.

Example 4-26 traceroute from TCPIPA to VIPA address of TCPIPB

===> tracerte [Link] (tcp tcpipa V srcip [Link]

CS V1R12: Traceroute to [Link] ([Link]):

1 * * *

The results are the same when trying to reach the VIPAs at TCPIPC and TCPIPD from either
TCPIPA or TCPIPB: Because the route table indicates a direct path through the OSA, the
stack attempts to send the packet over the direct route and experiences a failure. This is what
we expected, because we coded ISOLATE on OSA2080X in TCPIPA (and TCPIPB).

ARP takeover with ISOLATE

As part of this test, we also activated a second interface on the same subnet in order to test
the ARP takeover function. The second interface was also coded with ISOLATE on TCPIPA
and TCPIPB. TCPIPC and TCPIPD were not defined with ISOLATE. When we deactivated
the second interface, the address from the adapter port we were taking over moved to the
remaining interface on that subnet, as you see at Y in Example 4-27.

Example 4-27 Effect of ARP takeover with ISOLATE

/**********************************************************************/
/* OSA/SF Query created [Link] on 10/12/2010 */
/* IOACMD APAR level - OA26486 */
/* Host APAR level - OA31645 */
/**********************************************************************/
************************************************************************
*** Start of OSA address table for CHPID 02 ***
************************************************************************
* UA(Dev) Mode Port Entry specific information Entry Valid
************************************************************************
Image 1.1 (A11 ) CULA 0
00(2080)* MPC N/A OSA2080 (QDIO control) SIU ALL

Chapter 4. Connectivity 169

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

02(2082) MPC 00 No4 No6 OSA2080 (QDIO data) Isolated SIU ALL
VLAN 10 (IPv4)

Group Address Multicast Address

01005E000001 [Link]

VMAC IP address
HOME 02004F776872 [Link]
HOME 02004F776872 [Link] Y

Note: The ARP takeover function still works as expected if we start a second device on the
same subnet in the same stack. ISOLATE does not alter this function.

Test 2: effect of ISOLATE and NOISOLATE for multiple stacks

We next test to see if the shared OSA on CHPID2 will allow TCPIPC and TCPIPD to
communicate directly or not with each other and with TCPIPA and TCPIPB. We also test to
confirm that the stack routes still allow us to reach TCP/IP stacks in the external network
across the OSA ports, even if they have been coded with ISOLATE. Examine Figure 4-21.

SC30 SC31 SC32 SC33

TCPIPA TCPIPB TCPIPC TCPIPD

PROFA30X PROFB31X PROFC32X PROFD33X

VIPA1L [Link]/24 VIPA1L [Link]/24 VIPA1L [Link]/24 VIPA1L [Link]/24

OSA2080X [Link]/24 OSA2080X [Link]/24 OSA2080X [Link]/24 OSA2080X [Link]/24

OMPROUTE OMPROUTE OMPROUTE OMPROUTE

ISOLATE ISOLATE QDIO OSA

[Link]/24

X3
X1 X2
[Link] [Link] [Link] [Link]
4
[Link] [Link] [Link] [Link]
[Link] [Link] [Link] [Link]

5 Switch 5
[Link] Router [Link]
[Link]/24

1, 2, 3 = Direct routes from TCPIPA or TCPIPB to any other stack are unsuccessful.
4 = Direct route between TCPIPC and TCPIPD is successful.
5 = Routes from any stack to terminals reached through the router are successful.

Figure 4-21 Available paths when ISOLATE has been defined and dynamic routing is enabled

Those tests show that the existing basic routing table at each of the stacks allows us to
communicate with TCP/IP networks reached through the external router (5).

170 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

The routing tables also permit TCPIPC and TCPIPD to communicate with each other (4);
Note: ISOLATE prevents only direct communication across the OSA port to any stack that
has coded the ISOLATE keyword. However, it does not prevent communication between
the stack defined with ISOLATE and any destinations beyond the OSA port. Therefore, we
continue to be able to use the stack’s routing table to TELNET or FTP from our
workstations into any of the TCP/IP stacks, regardless of the coding of ISOLATE or
NOISOLATE.

NOISOLATE is either coded or defaulted on the INTERFACE in the two stacks. However,
TCPIPC and TCPIPD cannot communicate at all with either TCPIPA or TCPIPB, and TCPIPA
and TCPIPB cannot communicate with each other over the internal OSA path (1, 2, 3). In
Example 4-28, we see the typical responses when a target cannot be reached.

Example 4-28 Unsuccessful attempts to reach TCPIPB from TCPIPC

===> tracerte [Link] (tcp tcpipc V srcip [Link]
CS V1R12: Traceroute to [Link] ([Link]):
1 * * *
2 * * *

Unfortunately, the only path that TCPIPC and TCPIPD for reaching TCPIPA and TCPIPB is
the direct route through the OSA port, but this port prevents internal routing because the
parameter ISOLATE has been coded at TCPIPA and TCPIPB. Examine the routing table in
Example 4-24 on page 167, where you see that the table points to a network route for
[Link]/24 that happens to be reached by way of a directly attached next-hop router
([Link]):
[Link]/24 [Link] UO 0000000000 OSA2080X

There is no route for any of the stacks to reach each other over the external router.

Again, the issue is that the dynamic routing table knows nothing of the ISOLATE feature
because ISOLATE is not a Layer 3 function. The dynamic routing protocol is working per the
protocol standards. So, how do we rectify this situation if we really want the stacks to
communicate with each other, but just not directly over the OSAs? It will be a matter of
adjusting the routing table by adding some non-replaceable static routes.

Recall that we earlier gave you options for dealing with this situation:
 Bypass the direct path by using Policy Based Routing (PBR).
 Bypass the direct path by coding static routes that supersede the routes learned by the
dynamic routing protocol (see Figure 4-16 on page 160).
 Adjust the costs or weights of connections to favor alternate interfaces over the interfaces
that have been coded with ISOLATE (see Figure 4-17 on page 161).

We tested only one of these options: coding static routes to supersede the dynamically
learned routes.

Chapter 4. Connectivity 171

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

Test 3: effect of ISOLATE: alternate path is present in routing table

In this test, we assume that we want to establish connectivity between TCPIPA and the other
stacks on System z, but not over the direct OSA path. We also want to establish connectivity
between TCPIPB and the other stacks, but not over the direct OSA path. In short, we want to
leave our dynamic routing with OSPF in place, but we need to ensure that the learned routes
over the direct OSA path are not always used. However, in doing so, we need to ensure that
the direct paths between TCPIPC and TCPIPD stay in place. We accomplish all of this by
adding non-replaceable static routes to the TCP/IP stacks, as shown in Figure 4-22.

SC30 SC31 SC32 SC33

TCPIPA TCPIPB TCPIPC TCPIPD

PROFA30X PROFB31X PROFC32X PROFD33X

VIPA1L [Link]/24 VIPA1L [Link]/24 VIPA1L [Link]/24 VIPA1L [Link]/24

OSA2080X [Link]/24 OSA2080X [Link]/24 OSA2080X [Link]/24 OSA2080X [Link]/24

OMPROUTE OMPROUTE OMPROUTE OMPROUTE

ISOLATE ISOLATE QDIO OSA

[Link]/24

X3
X1 X2
[Link] [Link] [Link] [Link]
4
[Link] [Link] [Link] [Link]
[Link] [Link] [Link] [Link]

5 Switch 5
[Link] Router [Link]
[Link]/24

1, 2, 3 = Direct routes from TCPIPA or TCPIPB to any other stack are unsuccessful.
4 = Direct route between TCPIPC and TCPIPD is successful.
5 = Routes from any stack to terminals reached through the router are successful.
6 = Indirect routes among the stacks through external router are successful if present in routing table.

Figure 4-22 Making paths available through an external next-hop router

The routes to the remote TCP/IP nodes (5) through the OSA ports continue to be successful
in our scenario; no changes are necessary here. The routing table between TCPIPC and
TCPIPD continues to function as expected to permit direct routing between the two stacks (4);
changes to the routing table are unnecessary here as well.

The routing paths indicated with 1, 2, and 3 in Figure 4-22 continue to be unsuccessful in this
test because we want to enforce ISOLATE. However, we can make the two-hop paths through
the external router (6) available if we code non-replaceable static routes. These routes will
supersede the dynamically learned routes in the stack’s routing table.

172 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Coding non-replaceable static routes with the BEGINROUTES statement block

We add the static routing statements shown in Example 4-29 to TCPIPA. Note that TCPIPA’s
VIPA should not be present in the table.

Example 4-29 Static non-replaceable routes at TCPIPA to override direct route through OSA port
;[Link](ROUTA30X)
BEGINRoutes
; Direct Routes - Routes that are directly connected to my interfaces
; Destination Subnet Mask First Hop Link Name Packet Size
;;;;;;;;;;;;;;;;;;;;;;;;;BELOW IS FOR TESTING ISOLATION;;;;;;;;;;;;;;;;
ROUTE [Link]/24 [Link] OSA2080X mtu 1492 1
ROUTE [Link]/24 [Link] OSA2080X mtu 1492 2
ROUTE [Link]/32 [Link] OSA2080X mtu 1492 3
ROUTE [Link]/32 [Link] OSA2080X mtu 1492 3
ROUTE [Link]/32 [Link] OSA2080X mtu 1492 3
;;;;;;;;;;;;;;;;;;;;;;;;ABOVE IS FOR TESTING ISOLATION;;;;;;;;;;;;;;;;;
ENDRoutes

In Example 4-29, we see, at 1 and 2, the indirect route to both the native OSA port IP subnet
and the VIPA IP subnet. In our scenario, these two statements do not suffice, because our
OSPF configuration indicates that we are advertising HOST routes for the VIPAs. As a result,
we also need the statements you see at 3, that is, the statements that point to a route over the
external router in order to reach the specific host VIPA addresses. If we do not code these
statements, OSPF will advertise HOST routes and our stack will always try unsuccessfully to
reach the target VIPAs over the OSA port.

We add the static routing statements shown in Example 4-30 to TCPIPB. The only difference
to the statements at TCPIPA is the absence of TCPIPB’s VIPA and the presence of TCPIPA’s
VIPA address.

Example 4-30 Static non-replaceable routes at TCPIPB to override direct route through OSA port
;[Link](ROUTB31X)
BEGINRoutes
; Direct Routes - Routes that are directly connected to my interfaces
; Destination Subnet Mask First Hop Link Name Packet Size
;
;;;;;;;;;;;;;;;;;;;;;;;;;BELOW IS FOR TESTING ISOLATION;;;;;;;;;;;;;;;;
ROUTE [Link]/24 [Link] OSA2080X mtu 1492
ROUTE [Link]/24 [Link] OSA2080X mtu 1492
ROUTE [Link]/32 [Link] OSA2080X mtu 1492
ROUTE [Link]/32 [Link] OSA2080X mtu 1492
ROUTE [Link]/32 [Link] OSA2080X mtu 1492
;;;;;;;;;;;;;;;;;;;;;;;;ABOVE IS FOR TESTING ISOLATION;;;;;;;;;;;;;;;;;
ENDRoutes

We are going to test only a subset of all addresses available at the four stacks, that is, the
connectivity with the VIPAs and the native OSA port addresses. Therefore, we have limited
our BEGINROUTES coding only to these two address types.

Note: If you also need connectivity to other addresses, such as CTC or HiperSockets, then
you might have to add more routes to your list of non-replaceable routes.

Chapter 4. Connectivity 173

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

To simplify the test, we stop all interfaces but OSA2080X and take a snapshot of the current
routing table (shown in Example 4-31).

Example 4-31 Routing table built by OMPROUTE (OSPF) at TCPIPA

D TCPIP,TCPIPA,N,ROUTE
IPV4 DESTINATIONS
DESTINATION GATEWAY FLAGS REFCNT INTERFACE
DEFAULT [Link] UGO 0000000000 OSA2080X
[Link]/32 [Link] UH 0000000000 VIPA1L
[Link]/32 [Link] UGHO 0000000000 OSA2080X A
[Link]/32 [Link] UGHO 0000000000 OSA2080X A
[Link]/32 [Link] UGHO 0000000000 OSA2080X A
[Link]/24 [Link] UO 0000000000 OSA2080X B
[Link]/32 [Link] UH 0000000000 OSA2080X
[Link]/32 [Link] H 0000000000 OSA20A0X
[Link]/24 [Link] UGO 0000000000 OSA2080X
[Link]/24 [Link] UGO 0000000000 OSA2080X
[Link]/32 [Link] H 0000000000 OSA20C0X
[Link]/32 [Link] H 0000000000 OSA20E0X
[Link]/24 [Link] US 0000000000 IQDIOLNK0A01070B
[Link]/32 [Link] H 0000000000 EZASAMEMVS
[Link]/32 [Link] UH 0000000000 IQDIOLNK0A01070B
[Link]/32 [Link] UHS 0000000000 IQDIOLNK0A01070B
[Link]/32 [Link] UHS 0000000000 IQDIOLNK0A01070B
[Link]/24 [Link] UGO 0000000000 OSA2080X
[Link]/32 [Link] UH 0000000002 LOOPBACK
[Link]/24 [Link] UGO 0000000000 OSA2080X
[Link]/24 [Link] UGO 0000000000 OSA2080X
[Link]/24 [Link] UGO 0000000000 OSA2080X
IPV6 DESTINATIONS
DESTIP: ::1/128
GW: ::
INTF: LOOPBACK6 REFCNT: 0000000000
FLGS: UH MTU: 65535
23 OF 23 RECORDS DISPLAYED
END OF THE REPORT

At A in Example 4-31, we see that OSPF reaches the VIPAs in subnet [Link]/24 over the
OSA port. At B, we see that OSPF has informed the stack that the network [Link]/24 is
directly attached.

174 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

We place OBEYFILE in the BEGINROUTES block. The new routing table is depicted in
Example 4-32.

Example 4-32 Routing table at TCPIPA with static routes inserted

D TCPIP,TCPIPA,N,ROUTE
IPV4 DESTINATIONS
DESTINATION GATEWAY FLAGS REFCNT INTERFACE
DEFAULT [Link] UGO 0000000000 OSA2080X
[Link]/24 [Link] UGS 0000000000 OSA2080X
[Link]/32 [Link] UH 0000000000 VIPA1L
[Link]/32 [Link] UGHS 0000000000 OSA2080X C
[Link]/32 [Link] UGHS 0000000000 OSA2080X C
[Link]/32 [Link] UGHS 0000000000 OSA2080X C
[Link]/24 [Link] UGS 0000000000 OSA2080X D
[Link]/32 [Link] UH 0000000000 OSA2080X
[Link]/32 [Link] H 0000000000 OSA20A0X
[Link]/24 [Link] UGO 0000000000 OSA2080X
[Link]/24 [Link] UGO 0000000000 OSA2080X
[Link]/32 [Link] H 0000000000 OSA20C0X
[Link]/32 [Link] H 0000000000 OSA20E0X
[Link]/24 [Link] US 0000000000 IQDIOLNK0A01070B
[Link]/32 [Link] H 0000000000 EZASAMEMVS
[Link]/32 [Link] UH 0000000000 IQDIOLNK0A01070B
[Link]/32 [Link] UHS 0000000000 IQDIOLNK0A01070B
[Link]/32 [Link] UHS 0000000000 IQDIOLNK0A01070B
[Link]/24 [Link] UGO 0000000000 OSA2080X
[Link]/32 [Link] UH 0000000003 LOOPBACK
[Link]/24 [Link] UGO 0000000000 OSA2080X
[Link]/24 [Link] UGO 0000000000 OSA2080X
[Link]/24 [Link] UGO 0000000000 OSA2080X
IPV6 DESTINATIONS
DESTIP: ::1/128
GW: ::
INTF: LOOPBACK6 REFCNT: 0000000000
FLGS: UH MTU: 65535
24 OF 24 RECORDS DISPLAYED
END OF THE REPORT

TCPIPB’s routing table looks similar after the changes are made. Both tables now have HOST
routes that point directly to the VIPAs and to the native OSA port addresses; however, the
route statement now sends any packets destined for those addresses through the router with
an IP address of [Link]. See the lines marked with A and B in Example 4-32.

Chapter 4. Connectivity 175

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

We also need to make routing changes at TCPIPC. See the statements we have added to this
stack in Example 4-33.

Example 4-33 TCPIPC: non-replaceable static routes to other TCP/IP nodes on System z
;[Link](ROUTC32X)
BEGINRoutes
; Direct Routes - Routes that are directly connected to my interfaces
; Destination Subnet Mask First Hop Link Name Packet Size
;;;;;;;;;;;;;;;;;;;;;;;;;BELOW IS FOR TESTING ISOLATION;;;;;;;;;;;;;;;;
ROUTE [Link]/32 [Link] OSA2080X mtu 1492 1
ROUTE [Link]/32 [Link] OSA2080X mtu 1492 1
ROUTE [Link]/32 [Link] OSA2080X mtu 1492 2
ROUTE [Link]/32 [Link] OSA2080X mtu 1492 2
;;;;;;;;;;;;;;;;;;;;;;;;ABOVE IS FOR TESTING ISOLATION;;;;;;;;;;;;;;;;;
ENDRoutes

At TCPIPC and TCPIPD, we need to override the routes learned from OSPF that point to the
addresses at TCPIPA and TCPIPB. In Example 4-33 at 1, we have defined host routes to the
native OSA port IP addresses at TCPIPA and TCPIPB that point to the external router. Note
how we have not explicitly coded any static routes for the TCPIPD stack. At 2, we add routes
to the host VIPAs that are in TCPIPA and TCPIPB, but not in TCPIPD.

Of course, we need to make the same types of routing changes at TCPIPD. See the
statements we have added to this stack in Example 4-34.

Example 4-34 TCPIPD: non-replaceable static routes to other TCP/IP nodes on System z
;[Link](ROUTD33X)
BEGINRoutes
; Direct Routes - Routes that are directly connected to my interfaces
; Destination Subnet Mask First Hop Link Name Packet Size
;;;;;;;;;;;;;;;;;;;;;;;;;BELOW IS FOR TESTING ISOLATION;;;;;;;;;;;;;;;;
ROUTE [Link]/32 [Link] OSA2080X mtu 1492 1
ROUTE [Link]/32 [Link] OSA2080X mtu 1492 1
ROUTE [Link]/32 [Link] OSA2080X mtu 1492 2
ROUTE [Link]/32 [Link] OSA2080X mtu 1492 2
;;;;;;;;;;;;;;;;;;;;;;;;ABOVE IS FOR TESTING ISOLATION;;;;;;;;;;;;;;;;;
ENDRoutes

At 1, we have defined host routes to the native OSA port IP addresses that point to the
external router. Note how we have not explicitly coded any static routes for the TCPIPC stack.
At 2, we add routes to the host VIPAs that are in TCPIPA and TCPIPB, but not in TCPIPC.

176 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

In Example 4-35, we show you the new routing table structure at TCPIPC. (The routing table
at TCPIPD resembles the one at TCPIPC, and so we do not illustrate it here.)

Example 4-35 Routing table at TCPIPC with entries provided by OSPF and by static routes
D TCPIP,TCPIPC,N,ROUTE
IPV4 DESTINATIONS
DESTINATION GATEWAY FLAGS REFCNT INTERFACE
DEFAULT [Link] UGO 0000000000 OSA2080X
[Link]/32 [Link] UGHS 0000000000 OSA2080X A
[Link]/32 [Link] UGHS 0000000000 OSA2080X A
[Link]/32 [Link] UH 0000000000 VIPA1L
[Link]/32 [Link] UGHO 0000000000 OSA2080X B
[Link]/24 [Link] UO 0000000000 OSA2080X C
[Link]/32 [Link] UGHS 0000000000 OSA2080X D
[Link]/32 [Link] UGHS 0000000000 OSA2080X D
[Link]/32 [Link] UH 0000000000 OSA2080X
[Link]/32 [Link] H 0000000000 OSA20A0X
[Link]/24 [Link] UGO 0000000000 OSA2080X
[Link]/24 [Link] UGO 0000000000 OSA2080X
[Link]/32 [Link] H 0000000000 OSA20C0X
[Link]/32 [Link] H 0000000000 OSA20E0X
[Link]/24 [Link] US 0000000000 IQDIOLNK0A01071F
[Link]/32 [Link] UHS 0000000000 IQDIOLNK0A01071F
[Link]/32 [Link] H 0000000000 EZASAMEMVS
[Link]/32 [Link] UH 0000000000 IQDIOLNK0A01071F
[Link]/32 [Link] UHS 0000000000 IQDIOLNK0A01071F
[Link]/24 [Link] UGO 0000000000 OSA2080X
[Link]/32 [Link] UH 0000000002 LOOPBACK
[Link]/24 [Link] UGO 0000000000 OSA2080X
[Link]/24 [Link] UGO 0000000000 OSA2080X
[Link]/24 [Link] UGO 0000000000 OSA2080X
IPV6 DESTINATIONS
DESTIP: ::1/128
GW: ::
INTF: LOOPBACK6 REFCNT: 0000000000
FLGS: UH MTU: 65535
25 OF 25 RECORDS DISPLAYED
END OF THE REPORT

Look more closely at Example 4-35. The entries marked with A were statically added to
override learned routes from OSPF. The entries at B and C remain as OSPF originally
advertised them. These are for addresses in TCPIPD or for other [Link]/24 addresses that
are not to be found in TCPIPA or TCPIPB. The entries marked with D were statically added to
override learned routes from OSPF.

Chapter 4. Connectivity 177

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

Testing with the non-replaceable static routes and OSPF

We use traceroute to determine whether we are now taking a one-hop or two-hop route
through the router. See the output in Example 4-36.

Example 4-36 traceroute tests from TCPIPA to TCPIPB

TO NATIVE OSA PORT ADDRESS AT TCPIPB:

===> tracerte [Link] (tcp tcpipa V srcip [Link]

CS V1R12: Traceroute to [Link] ([Link]):

1 router1 ([Link]) 36 bytes to [Link] 1 ms 0 ms 0 ms A
2 [Link] ([Link]) 36 bytes to [Link] 0 ms 0 ms 0 ms
***

TO VIPA AT TCPIPB from NATIVE OSA PORT on TCPIPA:

===> tracerte [Link] (tcp tcpipa V srcip [Link]

CS V1R12: Traceroute to [Link] ([Link]):

1 router1 ([Link]) 36 bytes to [Link] 0 ms 0 ms 0 ms A
2 WTSC31B ([Link]) 36 bytes to [Link] 2 ms 0 ms 0 ms
***

TO VIPA AT TCPIPB from VIPA on TCPIPA:

===> tracerte [Link] (tcp tcpipa V srcip [Link]

CS V1R12: Traceroute to [Link] ([Link]):

1 router1 ([Link]) 36 bytes to [Link] 0 ms 0 ms 0 ms A
2 WTSC31B ([Link]) 36 bytes to [Link] 0 ms 0 ms 0 ms
***

As you can see in Example 4-36, our command executions are successful and point to a
two-hop route across the router (A) between the two isolated TCPIP stacks (TCPIPA and
TCPIPB).

Our tests to the external terminals from TCPIPA are also successful. (See Figure 4-22 on
page 172for a diagram of where the terminals reside.) Our test in Example 4-37 shows a
verbose ping to the terminal at address [Link]:

Example 4-37 Connectivity through the ISOLATED OSA to the remote network
===> ping [Link] (tcp tcpipa V srcip [Link]
CS V1R12: Pinging host [Link]
with 256 bytes of ICMP data
Ping #1 from [Link]: bytes=264 seq=1 ttl=127 time=1.28 ms
***
Ping #2 from [Link]: bytes=264 seq=2 ttl=127 time=0.37 ms
Ping #3 from [Link]: bytes=264 seq=3 ttl=127 time=0.91 ms
Ping statistics for [Link]
Packets: Sent=3, Received=3, Lost=0 (0% loss)
Approximate round trip times in milliseconds:
Minimum=0.37 ms, Maximum=1.28 ms, Average=0.85 ms, StdDev=0.46 ms
***

178 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Again, notice how our test is successful. The ISOLATE parameter did not inhibit us from
reaching our external network over the OSA port.

We must now test our connectivity from TCPIPA to TCPIPC and TCPIPD to see if the two-hop
route is successful now that we have updated the routing tables at all four stacks. See the
indications of a two-hop route (2) in Example 4-38.

Example 4-38 traceroute tests from TCPIPA to TCPIPC

TO NATIVE OSA PORT ADDRESS AT TCPIPC from NATIVE OSA PORT at TCPIPA:

===> tracerte [Link] (tcp tcpipa V srcip [Link]

CS V1R12: Traceroute to [Link] ([Link]):

1 router1 ([Link]) 36 bytes to [Link] 0 ms 0 ms 0 ms
2 [Link] ([Link]) 36 bytes to [Link] 0 ms 0 ms 0 ms 2
***

TO NATIVE OSA PORT ADDRESS AT TCPIPC from VIPA at TCPIPA:

===> tracerte [Link] (tcp tcpipa V srcip [Link]

CS V1R12: Traceroute to [Link] ([Link]):

1 router1 ([Link]) 36 bytes to [Link] 0 ms 0 ms 0 ms
2 [Link] ([Link]) 36 bytes to [Link] 0 ms 0 ms 0 ms 2
***

TO VIPA AT TCPIPC from VIPA on TCPIPA:

===> tracerte [Link] (tcp tcpipa V srcip [Link]

CS V1R12: Traceroute to [Link] ([Link]):

1 router1 ([Link]) 36 bytes to [Link] 0 ms 0 ms 0 ms
2 WTSC32C ([Link]) 36 bytes to [Link] 0 ms 0 ms 0 ms 2
***

Chapter 4. Connectivity 179

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

Finally, we test the connectivity between TCPIPC and TCPIPD to ensure that we are still
taking the direct path through the OSA port despite the addition of our static routes. See
Example 4-39.

Example 4-39 Static and dynamic routes at TCPIPC and TCPIPD

TO NATIVE OSA PORT ADDRESS AT TCPIPD from NATIVE OSA PORT at TCPIPC:

===> tracerte [Link] (tcp tcpipc srcip [Link]

CS V1R12: Traceroute to [Link] ([Link]):

1 [Link] ([Link]) 0 ms 0 ms 0 ms A
***

TO VIPA AT TCPIPD from VIPA on TCPIPC:

===> tracerte [Link] (tcp tcpipc srcip [Link]

CS V1R12: Traceroute to [Link] ([Link]):

1 [Link] ([Link]) 0 ms 0 ms 0 ms A
***

You see in Example 4-39 that we are indeed taking the one-hop route (A).

4.5.6 Conclusions and recommendations: best practices for isolating traffic

Our experience shows that the ISOLATE function is not necessary in order to segregate traffic
that is flowing across a shared OSA port. We have also seen that the ISOLATE function
requires careful consideration, especially when you have implemented dynamic routing
protocols in order to simplify the maintenance of valid routes in your network.

If you are using static routing protocols at z/OS and must isolate traffic over shared OSA
ports, then either deploy a VLAN implementation with separate VLAN IDs assigned to
separate IP subnets or exploit the ISOLATE feature and remember to disable ICMP redirects.

If you are using a dynamic routing protocol at z/OS and must isolate traffic over shared OSA
ports, use a VLAN implementation with separate VLAN IDs assigned to separate IP subnets
for each of the sharing TCP/IP stacks.

If you are using a dynamic routing protocol at z/OS and must isolate traffic over shared OSA
ports but are reluctant to deploy VLANs in the System z TCP/IP stacks, use the OSA
Connection Isolation feature. When doing so, plan a strategy to include some
non-replaceable static routes in the TCP/IP stack’s routing table that will force a hop over an
external router. Create a robust testing plan to ensure that you are permitting only the type of
routing that you desire.

4.6 HiperSockets connectivity

HiperSockets provides very fast TCP/IP communications between different logical partitions
(LPARs) through the system memory of the System z server. The LPARs that are connected
this way form an internal LAN, passing data between the LPARs at memory speeds, and
thereby totally eliminating the I/O subsystem overhead and external network delays.

180 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

To create this scenario, we define the HiperSockets, which is represented by the IQD CHPID
and its associated devices. All LPARs that are configured to use the shared IQD CHPID have
internal connectivity, and therefore have the capability to communicate using HiperSockets.

In our environment we use three IQD CHPIDs (F4, F5, and F6). Each will create a separate
logical LAN with its own subnetwork. Figure 4-23 depicts these interfaces to our scenario.

A11 (SC30) A13 (SC31) A16 (SC32) A18 (SC33)

TCPIPA TCPIPB TCPIPC TCPIPD

IUTIQDF4L [Link]/24 IUTIQDF4L [Link]/24 IUTIQDF4L [Link]/24 IUTIQDF4L [Link]/24

IUTIQDF5L [Link]/24 IUTIQDF5L [Link]/24 IUTIQDF5L [Link]/24 IUTIQDF5L [Link]/24
IUTIQDF6L [Link]/24 IUTIQDF6L [Link]/24 IUTIQDF6L [Link]/24 IUTIQDF6L [Link]/24

HiperSockets CHPID F4 Devices E800-E81F IPADDR 10.1.4.x1

HiperSockets CHPID F5 Devices E900-E91F IPADDR 10.1.5.x1
HiperSockets CHPID F6 Devices EA00-EA1F IPADDR 10.1.6.x1

Figure 4-23 HiperSockets implementation scenario

4.6.1 Dependencies
The dependencies are:
 The HiperSockets must be defined as CHPID type IQD to the server using HCD or IOCP.
This CHPID must be defined as shared to all LPARs that will be part of the HiperSockets
internal LAN (see Example 4-1 on page 137).
 When explicitly defined, a correspondent TRLE must be created in VTAM using a port
name IUTIQDxx, where xx is the CHPID number.
 When more than one IQD CHPID is configured to a specific LPAR, the VTAM start option
IQDCHPID must be used to specify which specific IQD CHPID this LPAR should use.

Note: In both cases, the TRLE is dynamically built by VTAM. The IQDCHPID VTAM
start option controls the VTAM selection of which IQD CHPID (and related devices) to
include in the HiperSockets MPC group (IUTIQDIO) when it is dynamically built for
DYNAMICXCF connectivity.

For additional details regarding how to configure a user-defined HiperSockets device or

interface, refer to z/OS Communications Server: IP Configuration Reference,
SC31-8776.

Chapter 4. Connectivity 181

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

4.6.2 Considerations
For isolation of IP traffic between LPARs through HiperSockets, consider using VLANs, which
means that you can logically subdivide the internal LAN for a HiperSockets CHPID into
multiple virtual LANs. Therefore, stacks that configure the same VLAN ID for the same CHPID
can communicate over that given HiperSockets, while stacks that have no VLAN ID or a
different VLAN ID configured cannot.

For HiperSockets, the VLAN ID applies to IPv4 and IPv6 connections. HiperSockets
VLAN IDs can be defined using the VLANID parameter on a LINK or INTERFACE statement.
Valid VLAN IDs are in the range of 1 to 4094.

4.6.3 Configuring HiperSockets

The steps to implement HiperSockets are basically the same as with an OSA-Express
interface. What changes is that there is no external configuration to be done, and the TRLE is
created dynamically by VTAM.

The steps in the TCP/IP profile are as follows:

1. Create a DEVICE and LINK statements for each HiperSockets CHPID.
2. Create a HOME address to each defined LINK.
3. Define the characteristics of each LINK statement using BSDROUTINGPARMS.

Create a DEVICE and LINK statements for each HiperSockets CHPID

When defining an MPCIPA HiperSockets, use the DEVICE statement to specify the IQD
CHPID hexadecimal value. The reserved device name prefix IUTIQDxx must be specified.
The suffix xx indicates the hexadecimal value of the corresponding IQD CHPID that was
configured with HCD or IOCP.

Define the device and link statements for each HiperSockets CHPID being implemented, as
shown in Example 4-40. A HiperSockets CHPID must be defined as an MPCIPA type of
device 1.

The link definition describes the type of transport being used. A HiperSockets link is defined
as IPAQIDIO 2.

Example 4-40 HiperSockets device and link definitions

;HiperSockets definition. The TRLE is dynamically created on VTAMs
DEVICE IUTIQDF4 MPCIPA 1
LINK IUTIQDF4L IPAQIDIO 2 IUTIQDF4
DEVICE IUTIQDF5 MPCIPA 1
LINK IUTIQDF5L IPAQIDIO 2 IUTIQDF5
DEVICE IUTIQDF6 MPCIPA 1
LINK IUTIQDF6L IPAQIDIO 2 IUTIQDF6

Important: The hexadecimal value specified here represents the CHPID, and it cannot be
the same value as that used for the dynamic XCF HiperSockets interface.

182 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

Create a HOME address to each defined LINK

Each link configured must have its own IP address. Our HiperSockets links are defined with
the IP addresses, as shown in Example 4-41.

Example 4-41 HiperSockets HOME addresses

HOME
[Link] IUTIQDF4L
[Link] IUTIQDF5L
[Link] IUTIQDF6L

Define the characteristics of each LINK statement using

BSDROUTINGPARMS
To define the link characteristics, such as MTU size (1) and subnet mask (2), we used the
BSDROUTINGPARMS statements (see Example 4-42). If not supplied, defaults will be used
from static routing definitions in BEGINROUTES or the OMPROUTE configuration (dynamic
routing definitions), if implemented.

If the link characteristics, BEGINROUTES statements, or the OMPROUTE configuration are

not defined, then the stack's interface layer (based on hardware capabilities) and the
characteristics of devices and links are used. This, however, might not provide the
performance or function desired.

Example 4-42 BSDRoutingparms statements

BSDROUTINGPARMS TRUE
; Link name MTU Cost metric Subnet Mask Dest address
VIPA1L 1492 0 [Link] 0
OSA2080L 1492 0 [Link] 0
OSA20A0L 1492 0 [Link] 0
OSA20C0L 1492 0 [Link] 0
OSA20E0L 1492 0 [Link] 0
IUTIQDF4L 8192 1 0 [Link] 2 0
IUTIQDF5L 8192 0 [Link] 0
IUTIQDF6L 8192 0 [Link] 0
ENDBSDROUTINGPARMS

Note: Static and dynamic routing definitions will override or replace the link characteristics
defined through the BSDROUTINGPARMS statements. Refer to Chapter 5, “Routing” on
page 205 for more information about static and dynamic routing.

4.6.4 Verifying the connectivity status

In this section, we verify the status of all devices defined to the TCP/IP stack or VTAM.

Chapter 4. Connectivity 183

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

Verifying the device status in TCP/IP stack

To verify the status of all devices being activated in the TCP/IP stack we use the NETSTAT
command with the DEVLIST option, as shown in Example 4-43.

Example 4-43 Using command D TCPIP,TCPIPA,N,DEV to verify the HiperSockets connection

DEVNAME: IUTIQDF4 DEVTYPE: MPCIPA
DEVSTATUS: READY
LNKNAME: IUTIQDF4L LNKTYPE: IPAQIDIO LNKSTATUS: READY
IPBROADCASTCAPABILITY: NO
CFGROUTER: NON ACTROUTER: NON
ARPOFFLOAD: YES ARPOFFLOADINFO: YES
ACTMTU: 8192
VLANID: NONE
READSTORAGE: GLOBAL (2048K)
SECCLASS: 255 MONSYSPLEX: NO
IQDMULTIWRITE: ENABLED (ZIIP)
ROUTING PARAMETERS:
MTU SIZE: 8192 METRIC: 80
DESTADDR: [Link] SUBNETMASK: [Link]
MULTICAST SPECIFIC:
MULTICAST CAPABILITY: YES
GROUP REFCNT SRCFLTMD
----- ------ --------
[Link] 0000000001 EXCLUDE
SRCADDR: NONE
[Link] 0000000001 EXCLUDE
SRCADDR: NONE
LINK STATISTICS:
BYTESIN = 196650
INBOUND PACKETS = 1647
INBOUND PACKETS IN ERROR = 0
INBOUND PACKETS DISCARDED = 0
INBOUND PACKETS WITH NO PROTOCOL = 0
BYTESOUT = 82841
OUTBOUND PACKETS = 670
OUTBOUND PACKETS IN ERROR = 0
OUTBOUND PACKETS DISCARDED = 0

Displaying TCP/IP device resources in VTAM

The device drivers for TCP/IP are provided by VTAM. When CS for z/OS IP devices are
activated, there must be an equivalent Transport Resource List Element (TRLE) defined to
VTAM. The devices that are exclusively used by z/OS Communications Server IP have
TRLEs that are automatically generated for them.

Because the device driver resources are provided by VTAM, you have the ability to display the
resources using VTAM display commands.

184 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

For TRLEs that are generated dynamically, the device type and address can be decoded from
the generated TRLE name. The format of the TRLE name is IUTtaaaa:
IUT Fixed for all TRLEs that are generated dynamically.
t Shows the device type, which indicates the following:
C Indicates this is a CDLC device.
H Indicates this is a HYPERCHANNEL device.
I Indicates this a QDIO device.
L Indicates this is a LCS device.
S Indicates this is a SAMEHOST device.
W Indicates this is a CLAW device.
X Indicates this is a CTC device.
aaaa The read device number. For SAMEHOST connections, this is a sequence
number.

To display a list of all TRLEs active in VTAM, use the D NET,TRL command, as shown in
Example 4-44.

Example 4-44 D NET,TRL command output

D NET,TRL
IST350I DISPLAY TYPE = TRL 468
IST924I ----------------------------------------------------
IST1954I TRL MAJOR NODE = ISTTRL
IST1314I TRLE = IUTIQDF6 STATUS = ACTIV CONTROL = MPC
IST1314I TRLE = IUTIQDF5 STATUS = ACTIV CONTROL = MPC
IST1314I TRLE = IUTIQDF4 STATUS = ACTIV CONTROL = MPC
IST1314I TRLE = ISTT3033 STATUS = ACTIV CONTROL = XCF
IST1314I TRLE = ISTT3032 STATUS = ACTIV CONTROL = XCF
IST1314I TRLE = ISTT3031 STATUS = ACTIV CONTROL = XCF
IST1314I TRLE = IUTIQDIO STATUS = ACTIV CONTROL = MPC
IST1314I TRLE = IUTSAMEH STATUS = ACTIV CONTROL = MPC
IST1454I 8 TRLE(S) DISPLAYED
The D NET,TRL,TRLE command that is used to obtain information about a HiperSockets device is shown in Example 4-45.

Example 4-45 D NET,TRL,TRLE=IUTIQDF6

D NET,TRL,TRLE=IUTIQDF6
IST075I NAME = IUTIQDF6, TYPE = TRLE 512
IST1954I TRL MAJOR NODE = ISTTRL
IST486I STATUS= ACTIV, DESIRED STATE= ACTIV
IST087I TYPE = LEASED , CONTROL = MPC , HPDT = YES
IST1715I MPCLEVEL = QDIO MPCUSAGE = SHARE
IST2263I PORTNAME = PORTNUM = 0 OSA CODE LEVEL = 4752
IST2337I CHPID TYPE = IQD CHPID = F6
IST2319I IQD NETWORK ID = 0720
IST1577I HEADER SIZE = 4096 DATA SIZE = 16384 STORAGE = ***NA***
IST1221I WRITE DEV = EA01 STATUS = ACTIVE STATE = ONLINE
IST1577I HEADER SIZE = 4092 DATA SIZE = 0 STORAGE = ***NA***
IST1221I READ DEV = EA00 STATUS = ACTIVE STATE = ONLINE
IST924I ------------------------------------------------------------
IST1221I DATA DEV = EA02 STATUS = ACTIVE STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST1717I ULPID = TCPIPA
IST2309I ACCELERATED ROUTING ENABLED
IST2331I QUEUE QUEUE READ

Chapter 4. Connectivity 185

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

IST2332I ID TYPE STORAGE

IST2205I ------ -------- ---------------
IST2333I RD/1 PRIMARY 2.0M(126 SBALS)
IST2305I NUMBER OF DISCARDED INBOUND READ BUFFERS = 0
IST1757I PRIORITY1: UNCONGESTED PRIORITY2: UNCONGESTED
IST1757I PRIORITY3: UNCONGESTED PRIORITY4: UNCONGESTED
IST1801I UNITS OF WORK FOR NCB AT ADDRESS X'0F191010'
IST1802I P1 CURRENT = 0 AVERAGE = 1 MAXIMUM = 1
IST1802I P2 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P3 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P4 CURRENT = 0 AVERAGE = 1 MAXIMUM = 2
IST924I -------------------------------------------------
IST1221I DATA DEV = EA03 STATUS = ACTIVE STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST1717I ULPID = TCPIPC
IST2309I ACCELERATED ROUTING ENABLED
IST2331I QUEUE QUEUE READ
IST2332I ID TYPE STORAGE
IST2205I ------ -------- ---------------
IST2333I RD/1 PRIMARY 2.0M(126 SBALS)
IST2305I NUMBER OF DISCARDED INBOUND READ BUFFERS = 0
IST1757I PRIORITY1: UNCONGESTED PRIORITY2: UNCONGESTED
IST1757I PRIORITY3: UNCONGESTED PRIORITY4: UNCONGESTED
IST1801I UNITS OF WORK FOR NCB AT ADDRESS X'0FC87010'
IST1802I P1 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P2 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P3 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P4 CURRENT = 0 AVERAGE = 1 MAXIMUM = 2
IST924I -------------------------------------------------
IST1221I DATA DEV = EA04 STATUS = ACTIVE STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST1717I ULPID = TCPIPB
IST2310I ACCELERATED ROUTING DISABLED
IST2331I QUEUE QUEUE READ
IST2332I ID TYPE STORAGE
IST2205I ------ -------- ---------------
IST2333I RD/1 PRIMARY 2.0M(126 SBALS)
IST2305I NUMBER OF DISCARDED INBOUND READ BUFFERS = 0
IST1757I PRIORITY1: UNCONGESTED PRIORITY2: UNCONGESTED
IST1757I PRIORITY3: UNCONGESTED PRIORITY4: UNCONGESTED
IST1801I UNITS OF WORK FOR NCB AT ADDRESS X'0F530010'
IST1802I P1 CURRENT = 0 AVERAGE = 1 MAXIMUM = 1
IST1802I P2 CURRENT = 0 AVERAGE = 0 MAXIMUM = 0
IST1802I P3 CURRENT = 0 AVERAGE = 1 MAXIMUM = 1
IST1802I P4 CURRENT = 0 AVERAGE = 1 MAXIMUM = 2
IST924I --------------------------------------------------
IST1221I DATA DEV = EA05 STATUS = RESET STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I --------------------------------------------------
IST1221I DATA DEV = EA06 STATUS = RESET STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I --------------------------------------------------
IST1221I DATA DEV = EA07 STATUS = RESET STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I --------------------------------------------------

186 TCP/IP Implementation Volume 1: Base Functions, Connectivity, and Routing

Draft Document for Review March 21, 2011 7:30 am 7896 [Link]

IST1221I DATA DEV = EA08 STATUS = RESET STATE = N/A

IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I --------------------------------------------------
IST1221I DATA DEV = EA09 STATUS = RESET STATE = N/A
IST1724I I/O TRACE = OFF TRACE LENGTH = *NA*
IST924I --------------------------------------------------
IST314I END

Chapter 4. Connectivity 187

7896 [Link] Draft Document for Review March 21, 2011 7:30 am

4.7 Dynamic XCF connectivity

The last connectivity scenario that we add to our environment connects all images within the
same sysplex environment through a dynamic XCF connection that is created by the
DYNAMICXCF definition in the TCP/IP profile.

After being defined, DYNAMICXCF provides connectivity between stacks under the same
LPAR by using the IUTSAMEH device (SAMEHOST) and between LPARs through
HiperSockets using a IUTiQDIO device. To connect other z/OS images or other servers, an
XCF Coupling Facility link is created.

In our scenario, we use DYNAMICXCF through HiperSockets with IQD CHPID F7. So, by
defining the DYNAMICXCF statement, we create the XCF subnetwork through HiperSockets,
as shown in Figure 4-24.

XCF
10.1.7.x1