DevOps from

APIs to z Systems
IBM Limited Edition

by Rosalind Radcliffe

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 DevOps from APIs to z Systems For Dummies®, IBM Limited Edition

Published by
John Wiley & Sons, Inc.
111 River St.
Hoboken, NJ 07030-5774
www.wiley.com
Copyright © 2017 by John Wiley & Sons, Inc.

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form
or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as
permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without the prior written
permission of the Publisher. Requests to the Publisher for permission should be addressed to the
Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011,
fax (201) 748-6008, or online at http://www.wiley.com/go/permissions.

Trademarks: Wiley, For Dummies, the Dummies Man logo, The Dummies Way, Dummies.com, Making
Everything Easier, and related trade dress are trademarks or registered trademarks of John Wiley & Sons,
Inc. and/or its affiliates in the United States and other countries, and may not be used without written
permission. IBM and the IBM logo are registered trademarks of International Business Machines
Corporation. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc.,
is not associated with any product or vendor mentioned in this book.

LIMIT OF LIABILITY/DISCLAIMER OF WARRANTY: THE PUBLISHER AND THE AUTHOR MAKE NO

REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE ACCURACY OR COMPLETENESS OF THE
CONTENTS OF THIS WORK AND SPECIFICALLY DISCLAIM ALL WARRANTIES, INCLUDING WITHOUT
LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE. NO WARRANTY MAY BE CREATED
OR EXTENDED BY SALES OR PROMOTIONAL MATERIALS. THE ADVICE AND STRATEGIES CONTAINED
HEREIN MAY NOT BE SUITABLE FOR EVERY SITUATION. THIS WORK IS SOLD WITH THE UNDERSTANDING
THAT THE PUBLISHER IS NOT ENGAGED IN RENDERING LEGAL, ACCOUNTING, OR OTHER PROFESSIONAL
SERVICES. IF PROFESSIONAL ASSISTANCE IS REQUIRED, THE SERVICES OF A COMPETENT PROFESSIONAL
PERSON SHOULD BE SOUGHT. NEITHER THE PUBLISHER NOR THE AUTHOR SHALL BE LIABLE FOR
DAMAGES ARISING HEREFROM. THE FACT THAT AN ORGANIZATION OR WEBSITE IS REFERRED TO IN
THIS WORK AS A CITATION AND/OR A POTENTIAL SOURCE OF FURTHER INFORMATION DOES NOT MEAN
THAT THE AUTHOR OR THE PUBLISHER ENDORSES THE INFORMATION THE ORGANIZATION OR WEBSITE
MAY PROVIDE OR RECOMMENDATIONS IT MAY MAKE. FURTHER, READERS SHOULD BE AWARE THAT
INTERNET WEBSITES LISTED IN THIS WORK MAY HAVE CHANGED OR DISAPPEARED BETWEEN WHEN
THIS WORK WAS WRITTEN AND WHEN IT IS READ.

For general information on our other products and services, or how to create a custom For Dummies book
for your business or organization, please contact our Business Development Department in the U.S. at
877-409-4177, contact [email protected], or visit www.wiley.com/go/custompub. For information about
licensing the For Dummies brand for products or services, contact BrandedRights&[email protected].

ISBN: 978-1-119-41023-2 (pbk); ISBN: 978-1-119-41024-9 (ebk)

Manufactured in the United States of America

10 9 8 7 6 5 4 3 2 1

Publisher’s Acknowledgments

Some of the people who helped bring this book to market include the
following:

Project Editor: Carrie A. Burchfield Business Development

Editorial Manager: Rev Mengle Representative: Sue Blessing

Acquisitions Editor: Steve Hayes Production Editor: Magesh Elangovan

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Table of Contents
INTRODUCTION ............................................................................................... 1
About This Book ................................................................................... 2
Icons Used in This Book....................................................................... 2
Beyond the Book .................................................................................. 3

CHAPTER 1: nderstandin t e alue of t e Mainframe ......... 5

Defining Mainframes and Their Importance .................................... 6
Calculating the Value of the Mainframe ............................................ 7
Placing Mainframe at the Heart of the Hybrid IT World .................. 9
Hybrid infrastructure .................................................................... 10
Hybrid data .................................................................................... 10
Hybrid applications ....................................................................... 10
Integrating Systems of Engagement with Systems
of Record ............................................................................................. 11
Transforming Organizations to Service Providers
with APIs .............................................................................................. 12

CHAPTER 2: nderstandin t e ypical Mainframe

Application Development allen es ...................... 15
Understanding Application Complexity ........................................... 16
Measuring Risk in Changing Business Critical
Applications......................................................................................... 17
System Availability.............................................................................. 18
Manual Test Processes ...................................................................... 19
Fragmentation of Skills and Tools .................................................... 19
Siloed Teams ....................................................................................... 20

CHAPTER 3: DevOps and t e Mainframe

Mission Possible ....................................................................... 21
Defining DevOps? ............................................................................... 21
Culture ............................................................................................ 23
Think ............................................................................................... 24
Code................................................................................................ 24
Deliver ............................................................................................ 27
Manage........................................................................................... 27
Learn............................................................................................... 28
Why DevOps Is Critical for Mainframe Environments ................... 28

Table of Contents iii

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 The Balancing Act: People, Processes, and Tools ........................... 31
People............................................................................................. 31
Processes ....................................................................................... 32
Tools ............................................................................................... 33
What Problems Does DevOps Solve?............................................... 34
Improving Application Development with DevOps ........................ 35

CHAPTER 4: Loo in at DevOps Best Practices

for z Systems ................................................................................. 37
Adopting DevOps Best Practices for z Systems .............................. 37
Modern development practices and tools................................. 38
Test environments ........................................................................ 40
Common deployment .................................................................. 43
Improving Productivity and Optimizing Systems ........................... 43
Improving productivity ................................................................. 43
Optimizing systems ...................................................................... 45

CHAPTER 5: Applyin Open Source ools to

DevOps and z Systems........................................................... 47
Open-Source Pipeline ........................................................................ 48
Git.................................................................................................... 48
Build system .................................................................................. 50
Pipeline orchestrator .................................................................... 50
Deployment system ...................................................................... 50
Build artifact repository ............................................................... 50
Open Source and z Development .................................................... 51
SCM for z/OS source ..................................................................... 51
Build system .................................................................................. 52
Pipeline orchestrator .................................................................... 52
Deployment system ...................................................................... 52
Built artifact repository ................................................................ 53

CHAPTER 6: Buildin for t e Di ital Economy .................................. 55

Understanding How Mainframe Is at the Heart of the Digital
Economy .............................................................................................. 55
Looking at Full Application Life Cycle Management....................... 58

CHAPTER 7: Seein DevOps Success Stories

in the Enterprise......................................................................... 61
Development Modernization ............................................................ 61
Large financial organization ........................................................ 62

iv DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Large financial organization in Europe ...................................... 63
CICS development organization .................................................. 64
Automated Testing and Deployment............................................... 65
Application Understanding ............................................................... 66
Seizing control of development .................................................. 67
Unleashing progress..................................................................... 67
Full Practice Adoption ........................................................................ 68

CHAPTER 8: Ma in a DevOps ransition ............................................ 71

Getting Started with DevOps ............................................................ 71
Guiding You through a DevOps Transformation Process ............. 72
Step 1: Pick the right set of applications .................................... 73
Step 2: Develop the vision ........................................................... 73
Step 3: Determine what’s holding you back .............................. 74
Step 4: Transition to DevOps ....................................................... 74
Step 5: Measure the ROI .............................................................. 74
Step 6: Repeat ............................................................................... 76

CHAPTER 9: Understanding Where DevOps

an a e ou................................................................................. 77
How Cloud and DevOps Come Together in the
Enterprise ............................................................................................ 77
The problem .................................................................................. 78
APIs: A paradigm shift .................................................................. 79
Building the Next Generation Enterprise IT Skills .......................... 82
Fit for Purpose Platforms .................................................................. 83
Myth Busters for Mainframe DevOps .............................................. 84
DevOps is for born-on-the-web companies .............................. 84
DevOps implies continuous deployment into production ...... 84
DevOps and ITIL don’t go together ............................................. 85
DevOps and separation of duties don’t mix .............................. 85
DevOps is only for small companies .......................................... 85
DevOps is only about automated deployment ......................... 86
DevOps is all about improving cycle time .................................. 86
DevOps is a tool ............................................................................ 87

Table of Contents v

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 vi DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Introduction
D
evOps is a way of doing business; it applies to all aspects of
an organization. DevOps brings Agile and Lean IT practices
to the entire organization to allow for quick reaction to
business demands. De O s a lies to all or ani ations new,
born-on-the-web companies as well as traditional small and
large businesses that have been around for many years.

With the drive today toward an application programming inter-

face (API) economy and the need to respond even more quickly
to business re uirements, com anies must chan e their internal
practices. This is why DevOps has become the de facto standard
for high-achieving businesses. The ability to respond quickly
to new ideas, e eriment with di erent ossible solutions, and
deli er e istin business with reater reliability and security
requires this fundamental change.

In some ways the term DevOps is misleadin , im lyin only

De elo ment and O erations are in ol ed, but really it s all o
the organization from the business through the running system.
Everyone in the organization must be responsible for delivery.
The organizations that embrace this overall change have seen the
greatest value from the transformation.

In lar e scale or ani ations this trans ormation can be harder,

but it s more critical or ust those com anies. The number o
silos is enerally reatly increased in lar e or ani ations, and
removing the silos is critical to DevOps. These large-scale orga-
ni ations ha e e istin systems that ro ide si nificant alue but
have been built up over years. These systems sometimes referred
to as the systems of record have valuable business assets that need
to be e osed to new business ossibilities and need to be able to
be updated rapidly. DevOps brings this ability to these systems
and improves the ability to integrate these systems with the new
mobile interfaces or systems of engagement.

DevOps is the cultural change to help achieve the value promise

o the API economy, a lyin to all or ani ations and all arts o
the or ani ation. With De O s, com anies transition their oli-
cies and procedures to support this new culture as well as bring in
the right tools to support this change.

Introduction 1

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 About This Book
DevOps from APIs to z Systems For Dummies, IBM Limited Edition, is
desi ned or e ecuti es, decision ma ers, and ractitioners who
are in organizations that have to deal with mainframe systems
or anyone who wants to learn more how DevOps can work within
lar e scale e istin or ani ations.

This boo also hel s com anies with e istin de elo ment and
o erations or ani ations that are se arated and siloed, includ-
ing those that are outsourced. You also discover a little history
behind how these silos developed and how now is the perfect time
to transition to a DevOps culture.

DevOps from APIs to z Systems For Dummies, IBM Limited Edition,

contains some e am les o how other com anies ha e ained
business value in their transitions to DevOps. DevOps is an ongo-
in ourney or com anies, and in this u dated edition, the e am-
ples show the greater value that can be achieved by continuing
with the transformation and by truly building a DevOps culture.

Icons Used in This Book

You find se eral icons in the mar ins o this boo . Here s what
they mean.

The Tip icon points out helpful information on various aspects of

DevOps.

Anything with the Remember icon is something to keep in mind

as you adopt DevOps.

The Warnin icon alerts you to ossible side e ects and/or im li-
cations of adopting DevOps.

Technical Stu oes into s ecific details that may be more rel-
e ant to a ractitioner than to an e ecuti e.

2 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Beyond the Book
This boo ro ides an introduction to De O s and how it fits in
organizations that have cross-platform environments including
main rames. But these a es can hold only so much, so or more
in ormation beyond what s co ered here, chec out the ollow-
ing links:

» IBM Enterprise DevOps: http://ibm.biz/DevOps4z

» IBM DevOps for Enterprise Systems POV: http://ibm.
biz/IBMDOESPOV
» Building an Enterprise API Strategy: https://ibm.biz/
BdrjtH
» DevOps on z Systems eGuide: http://ibm.co/1QWTCDa

Introduction 3

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 4 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 IN THIS CHAPTER
» De nin mainframes and y t ey re
important

» Seein t e mainframe s value

» De nin t e mainframe s role in t e

ybrid I orld

» Discoverin o to inte rate systems of

en a ement it systems of record

» sin APIs to transform or anizations

Chapter 1
Understanding the
alue of t e Mainframe

W
hen you hear the word mainframe, many di erent
thoughts may come to mind. To some, the mainframe
is an old machine no longer in use. To others, main-
frames are, by far, the most reliable computing solution available.
There was a time when it was “common knowledge” that the
mainframe was dead. To paraphrase Mark Twain, news of the
mainframe’s demise was greatly exaggerated.

Most consider the mainframe’s birth to coincide with the April 7,

1964, announcement of the IBM System/360 line of computers.
After more than 50 years of evolution and technological advances,
the mainframe continues to run most of the world’s businesses.
According to IBM market development and insights

» The mainframe is essential to 44 out of the top 50 worldwide

banks.
» 70 percent of the United States’ largest retailers use z
Systems.

CHAPTER 1 nderstandin t e alue of t e Mainframe 5

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 » 10 out of 10 of the world’s largest insurers use the
mainframe.
» 90 percent of the world’s largest airlines use z Systems.
» 1.3 million CICS transactions are processed every second,
every day. In comparison, there are 68,542 Google searches
every second globally.

In this chapter, I give you the background for the mainframe and
how it’s used today.

De nin Mainframes and

eir Importance
Large scale computing started out with built-for-purpose
machines, tar eted to s ecific use cases, such as the business class
machines and the scientific class machines. This di ision o labor
even gave rise to two separate user groups, GUIDE and SHARE:
GUIDE or the Business and SHARE or the Scientific. O er the
years, these groups merged as well. In the United States, SHARE
continues to be the premier user group for enterprise organiza-
tions. There are equivalent groups all over the world.

Some 53 years ago, on April 7, 1964, IBM introduced a new line of

general-purpose computers to serve the needs of businesses, uni-
versities, governments, and, in fact, any organization that needed
large-scale transaction performance. These computers and their
successors became known as mainframes.

A number of operating systems have been developed to run on the

mainframe hardware:

» For ultra-high-speed transaction processing, there’s z/TPF or

Transaction Processing Facility.
» z/VM is used to provide virtualization environments to
allow other operating systems to share the mainframe
platform.

6 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 » For more general purpose enterprise production workloads,
there’s z/OS.
» The most recent addition is Linux for z Systems, completing
the Linux story across the server lines.

These di erent o eratin systems ro ide di erent ca abilities

for organizations while taking advantage of the underlying reli-
ability of the hardware.

The current generation of the mainframe is the z Systems z13.

The can rocess . billion transactions a day the e ui a-
lent of 100 cyber Mondays every day of the year. The z13 is the
most modern mainframe hardware available, with a new proces-
sor desi n able to address TB o memory, with aster I/O and u
to 141 processor units in one machine. A single footprint can run
as many as 8,000 virtual servers.

alculatin t e alue of t e
Mainframe
The mainframe was designed from the casters up to be the ideal
lat orm or traditional online transaction rocessin OLTP wor -
loads. The was s ecifically enhanced to handle the e losi e
growth that today’s mobile users are driving into the mainframe.
The z13, with built-in encryption technology and the capability to
do inline analytics, can be the basis for key business solutions such
as real-time analytics and in-transaction fraud detection.

The z Systems machines provide this capability in the most

ener y efficient machines. The a era e business class main rame
s uses about the same amount o ower as ust our itchen
co ee ma ers, or less electricity than a clothes dryer. While
some ser er lines fill rooms and challen e the HVAC in rastruc-
tures that support them, with a single footprint, the z Systems
z13 provides the processing power required to run large-scale
a lications. With built in redundancy and efficient multi site
capabilities, a few of the machines can provide all the processing
power required for large-scale organizations in full high avail-
ably, disaster reco ery confi urations.

CHAPTER 1 nderstandin t e alue of t e Mainframe 7

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 With the Systems ca ability to run efficiently at ercent uti-
li ation or hi her , clients can ut essentially all the rocessin
ower they urchased to e ecti e roducti e wor . With shar-
in al orithms, honed to a fine ed e o er years o e erience,
mainframes provide the ability to consolidate many workloads
into very few machines while still ensuring the most important
workloads have the processing power they need. Mainframe users
et all this efficiency with a nearly seamless ability to scale with
their business.

The Linu ONE system or runnin Linu wor loads ro ides

another demonstration o the e ibility o hardware that s
desi ned and built with reliability at the core. By usin Linu ONE,
you can consolidate your various Linux servers onto a single sys-
tem with the built-in security. In addition, the Blockchain High
Security Business networ on Linu ONE is ro idin a new oun-
dation or the way business can run. By runnin on Linu ONE and
taking advantage of the built-in security and the secure services
container, businesses can trust the blockchain and have improved
performance and scalability.

In addition, z Systems are the most secure, with security built

into it from the very start. z Systems have the highest commer-
cial server security ratings in the industry. IBM z Systems are the
only commercial lat orm with EAL security classification. This
security classification hel s uarantee that the system has been
erified to meet s ecific security re uirements. Only the Systems
ser ers ro ide an inte rated encry tion in rastructure rom
the ser er, to the /OS o eratin system to rotect sensiti e
data over the Internet, on databases, and tape.

Looking at today’s mainframe, it’s better to see it as a large server

that has additional reliability and security built in. Thinking of it
as a lar e ser er with its own uality o ser ice QoS hel s osi-
tion it in the right way as you think about what applications or
parts of applications you would run on the mainframe and what
you would run on other systems. The term fit-for-purpose platform
is generally used to describe this concept of picking the type of
hardware most appropriate for the work being performed.

8 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 LOO I A OS OI E
MAI FRAME ORLD
To decipher the who’s who of the mainframe world, you must first
start with a little look back at history:

• A Semi-Automated Business Research Environment, better known as

SABRE, is still at the heart of most airline reservations systems.
SABRE has been running on z Systems since its conception.
• When Neil Armstrong and Buzz Aldrin became the first humans to
walk on the moon, forever changing the view of the world, the
System 360, the mainframe of the time, was used to process the
data for the lunar landing and lifto
• Today, z Systems are used primarily in financial, insurance, health-
care, and retail environments. IBM z Systems process more than
23 billion ATM transactions per year worth more than $1.4 trillion
(that’s more than $3 billion a day).
• First National Bank (FNB) of South Africa is bringing mobile bank-
ing to the unbanked in Africa through the use of z Systems.
• As part of the 50th Anniversary of the mainframe celebration in
2014, a number of companies discussed their use of mainframe;
companies like VisaNet have been running without an outage for
over ten years, and it continues to run without outage.
• In the FIFA 2014 World Cup, 14 cameras were set up to capture
goal mouth action. The data from these cameras was sent to a
mainframe computer that analyzed each goal and, in turn, sent a
signal to a watch worn by the referee to confirm that a goal had
been scored.

Examples like these demonstrate why people use z Systems. They’re

built with reliability and resilience at their core, and they provide built-
in encryption and the most secure system currently available.

Placin Mainframe at t e eart of t e

ybrid I orld
Hybrid cloud is the standard today or lar e com anies the
capability to run systems in whatever environment is appropriate:

CHAPTER 1 nderstandin t e alue of t e Mainframe 9

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 ublic cloud, ri ate cloud, or on remise. Di erent actors con-
tribute to the appropriate location for the systems, including
security concerns as well as country laws. When o ernments
require that personal data stay within the country boarders,
companies, depending on where they’re located and where the
country they’re working in, may need to add a new private cloud
or add their own data center.

But when you think of a cloud, you may be thinking of commodity

hardware and wonderin what Systems has to do with it. Critical
business applications run on z Systems so they need to be part of
the overall cloud solutions. And if you think about it, mainframes,
in many ways, were the first cloud systems, ro idin a time
sharing system for multiple applications, allowing the scaling up
of the application.

When thin in about hybrid cloud, di erent models e ist. These

are covered in this section.

ybrid infrastructure
Hybrid infrastructure is about mana in two or more di er-
ent clouds as one bi one. Consider the model where you want
to leverage Linux instances from within the enterprise and in a
public cloud. You want to have a single interface managing those
instances regardless of where they’re running. Businesses that
can e ecti ely utili e multi le cloud resources will ma imi e
their efficiency.

ybrid data
Hybrid data is about leveraging data from multiple sources to cre-
ate information. This is important to businesses today because
much of their traditional data is in relational structures like DB2
or Oracle, and much o the cloud data is in unstructured NoSQL
models li e Mon o or Couch. Businesses need to be able to unite
both to be successful.

ybrid applications
Hybrid applications are about splitting up applications and running
them across multiple environments. This is extremely important
to enterprises because this is about breaking up their monolithic
applications and turning them into services or application pro-
rammin inter aces APIs . These APIs can then be combined

10 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 in new ways, with new tools and technologies. Businesses that
unleash their business assets as APIs will not only create a new
agile technological environment but also enable a powerful agile
development model.

Inte ratin Systems of En a ement

it Systems of Record
In the past, information was aggregated in the data center. The
enterprise had applications running on the mainframe or dis-
tributed servers that took data and information and aggregated
it. These applications were accessed through terminals or clients
outside the data center. Even Internet applications followed the
model of accessing information within a data center via devices
that were accessed throu h a client a browser .

The proliferation of mobile devices radically changes this model.

Mobile devices represent a new way of turning data into infor-
mation. In fact, mobile devices create information from data and
other pieces of information. In short, these devices are the new
collection point for information. These devices are called systems
of engagement because they work with the users where they are.
They also provide data about the users in addition to the users’
direct input. They can provide location, contact, movement, or
event data that can be provided by the device itself in addition to
the input known by the users. They are meant to represent the
users and their needs.

These systems of engagement provide vast amounts of data

and, in some cases, information itself, but they still need to take
advantage of critical information available only in the data cen-
ter. Therefore, they need systems of record to perform business
unctions that are ust not suited to the systems o en a ement.
The systems of record are those systems, generally running on
/OS, that ro ide the actual data and transactions or rocessin
the data. These two systems work together and provide new ways
to understand, sell, and service the needs of the user commu-
nity. They also provide a completely new way to design and deploy
applications, which can change the way an enterprise interacts
with its users forever.

CHAPTER 1 nderstandin t e alue of t e Mainframe 11

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Ima ine this You o to wor in the mornin , define a new idea or
marketing your products to users (or perhaps a small demographic
o your user community , and ha e an a lication su ortin that
idea out in the users’ hands the same day. This is the promise of
systems of engagement working with systems of record. Instead
of carrying the burden of creating everything in one place, this
model exists by taking the core capabilities in the systems of
record and ro idin them to systems o en a ement as APIs that
can be easily consumed. These APIs then can be combined and
recombined in di erent ways to create new results rom the same
base data. Because the tools used on the systems of engagement
leverage tools like Ruby and JavaScript, they can create powerful
a s uic ly, and because they use systems o record APIs, they
can inherit enterprise class capabilities.

ransformin Or anizations to Service

Providers it APIs
The model of systems of engagement talking to systems of record
is another disruptive technology in hybrid cloud. Exposing exist-
in business unction ia APIs allows enter rises to com ete in a
world that’s beyond their reach and enables them to sell capabil-
ity that was once only available internally. The model also helps
enter rises turn their IT rom a cost center to a rofit center.

Re resentational State Trans er REST based inter aces are

bein added to e istin business lo ic to e ose it as an API. The
REST standard has helped transform the ease at which applica-
tions can interface with each other. The REST standard helps build
new applications through the combination of multiple existing
applications in ways never intended with the development of the
original application.

REST is an architectural style to provide a simple way to interact

between systems, providing a stateless interaction with simple
HTTP interactions o PUT, GET, POST, and Delete. The messa es
are self-descriptive allowing a simple interaction. Traditionally
web ser ices ha e had more structure and definition, but REST
ma es it much easier to find and use ser ices and is becomin the
most commonly used standard.

12 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Many of today’s mainframe applications contain transactions
that can easily be e osed as APIs by usin the REST standard,
and with ca abilities such as SWAGGER used to define these APIs,
it s easier or de elo ers to find and combine a set o APIs into
a new business or business function. Even if the applications
today are monolithic, ser ices can easily be created usin /OS
Connect to define the API. These can then be e osed internally
and e ternally by usin an API mana er. By e osin the e istin
main rame a lications business alue throu h APIs, or ani-
zations can more easily access this value to drive new business
opportunities.

Transforming to a service provider is a way to not only provide

better value to the business, but also to change the way IT is seen
within an organization. By becoming a revenue generator, the
ability to invest and innovate increase.

CHAPTER 1 nderstandin t e alue of t e Mainframe 13

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 14 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 IN THIS CHAPTER
» Understanding the application
complexity

» Seeing the risk in changing business

critical applications

» Knowing why system availability is key

» Looking at the manual test processes

» Discovering the implications of

fragmented skills and tools

» Working with siloed teams

Chapter 2
Understanding the
Typical Mainframe
Application
Development Challenges

A
pplication development for the mainframe has remained
the same for years (in updating this book, this is one of the
chapters I was hoping to change dramatically; however,
not much has changed in the last two years). In many environ-
ments, you see the same tools and processes in use that were in
place 30 years ago. In this chapter, I review the challenges that
have caused these processes to be put in place and how these tools
and processes are now limiting the capability to change as quickly
as today’s business requirements demand.

CHAPTER 2 Understanding the Typical Mainframe Application Development Challenges 15

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Understanding Application Complexity
Applications that have evolved over many years, by their nature,
will end up more complex than originally intended. Business criti-
cal applications, with their need to be highly available, drive even
more complexity. Developers, unwilling to update existing code
they didn’t write (and may not even understand), tend to add new
code that works around the existing code, making the result even
harder to maintain. At the core of the original applications are
usually sets of transactions performing some business task. For
example, there could be a transaction to create a customer record,
update a customer record, read that customer, or delete a customer
record. These transactions would get some input usually through
a screen, as dis layed in Fi ure , and then er orm the
action on the data. The best of the applications didn’t mix the dis-
play and presentation logic with the business logic but some did.
These transactions are now functions hidden inside the monolithic
application, but they’re what provide the business value.

The applications that form the systems of record have been built
and changed over the years by using tools with 3270 interfaces.
Those 3270 oriented tools have survived long after the demise of the
3270 hardware thanks, in part, to the 3270 Emulator. 3270 Emula-
tors turn modern workstations into 30-year-old user input devices.
Fi ure shows the standard Interacti e System Producti ity Facil-
ity ISPF rimary o tion menu dis layed on a modern wor station.

FIGURE 2-1: The standard ISPF primary option menu displayed on a modern
workstation.

16 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 WHERE IS THE DOCUMENTATION?
Developers don’t like to document their code, a reality that has not
changed over time. With applications that started 30 years ago, the
lack of documentation and clarity of design means the applications
have been modified, changed, and extended over the years without
clear understanding of the original design. Even when developers
documented the original design, it was usually not tied directly to the
code and rarely, if ever, updated as the code was changed. This left
the documentation with little semblance to the current code base.

Originally designed in the 1980s and optimized for data entry,

these interfaces were intended for a limited set of “power” users.
O er time, end user systems e ol ed, first as riendlier inter aces
for Windows workstations, then to browser-based interfaces in
many cases.

As part of the client server movement, application servers were

added into the environment to access the back-end systems of
record with new modern interfaces to hide the 3270 interfaces
from the users. These application servers added another layer into
the application design and additional complexity into the envi-
ronment. They also provided a second channel for interaction for
clients, allowing some users to use the traditional 3270 interface
with others using the new front ends.

Now with mobile front ends, applications need to support addi-

tional channels for interaction. Applications continue to increase
in complexity with the mobile app and browser interface, the
mid-tier providing its own processing, and the system of record
interacting with the data and providing the business transactions.

Measuring Risk in Changing Business

Critical Applications
Any time an application change goes into production, a problem
may cause an outa e. Sometimes, de loyin the chan e re uires
a short outage. You can plan for this. After the change is in place,
unless everything was done absolutely correctly (taking all the

CHAPTER 2 Understanding the Typical Mainframe Application Development Challenges 17

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 possible interactions into account), there could be problems. Many
production problems are the result of an incorrect “deployment”
o a chan e not that the chan e itsel wasn t coded correctly.

Because the fact that making a change could cause an outage, many
organizations reduced the number of windows where application
changes could be made. Reducing the number of outage windows
was seen as reducing the risk, and although more changes were
applied at once, they were grouped and hopefully tested together.
Reducing changes was seen as the key way to reduce risk, which
would then group up changes into large releases scheduled for
production in these planned outages. This grouping of changes,
although seen as reducing risk, actually increased the risk of
incompatible changes or required long testing cycles to ensure
compatibility.

System Availability
In Cha ter , I e lain that main rame systems ha e not only been
in use but also have provided continuous service for years. System
availability generally refers to the percentage of time the capability
provided by the system is available. For many systems, this avail-
ability is generally measured as 99 percent, outside of planned
outa es. Howe er, many clients measure /OS a ailability by the
number of years the system has been running without an out-
age. The basic assumption of the mainframe is that its hardware
and software are designed to be both reliable and highly avail-
able. There is redundancy built into every part of the system. The
software is written to support multiple workloads in a shared
environment with failure isolation and dynamic recovery facilities
that at least mask if not avoid outages as perceived by the users.

S ecifically /OS su orts Parallel Sys le or a sys le en iron-

ment, allowin multi le /OS systems in lo ical hardware artitions
LPARs to ro ide a seamlessly redundant en ironment or a li-
cations and their data. This allows updates to be made to hardware,
operating systems, subsystems, and even the applications them-
sel es without a ectin the o erall a ailability as seen by the user.
The same has been true of databases with data sharing groups cre-
ated to allow the data to remain available while systems elements
are o ine or maintenance. Pressured to deli er more reliable
solutions, distributed systems are now adopting the concepts the
mainframe has had in place and in production for decades.

18 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Manual Test Processes
Manual testing is the norm for many mainframe applications.
With little application understanding, determining what needed
to be tested for each change was hard. For many organizations,
this resulted in taking months to manually test any change that
was going into production, running entire batch runs, and hav-
ing the business analysts manually verify the application changes.

The promise of automation hasn’t delivered when it comes to

testin /OS a lications. Buildin automated tests was diffi-
cult due to lack of automated testing frameworks designed for
the dynamic, highly resource shared, multi-system environment
that is /OS. As testin ramewor s such as Unit were de elo ed
or distributed a a systems, no e ui alent was a ailable or /OS
until recently. With the latest updates in automated testing capa-
bilities, now is the time to look at these tools again.

Manual tests were the norm as the testing tools were lim-
ited and the processes had separate teams doing the testing in
a later phase. The concepts of Test Driven Development weren’t
used in the main rame. With the ad ent o Unit and with the
transition to more a lication ro rammin inter aces APIs to
expose business functions, automated testing can be more easily
accomplished.

Fragmentation of Skills and Tools

Looking at the application development environment available
today, you see an unfortunate diversity of tools and practices. The
/OS de elo ment teams are enerally still usin based tools
that have been around for the last 30 years. The distributed teams,
however, have advanced their development environments, taking
advantage of the latest and greatest tools as they build applica-
tions. While the /OS teams stay with the same tools year a ter
year, the distributed teams experiment and develop new capa-
bilities to im ro e the de elo ment rocess. Startin with the
individual development environment (IDE) that provides individ-
ual productivity, distributed teams have had the value of program
ow, code com letion, and a wide ran e o other roducti -
ity improvements just for the developer. Then comes modern

CHAPTER 2 Understanding the Typical Mainframe Application Development Challenges 19

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 source code mana ers SCMs that allow or reater sharin and
isibility to the code. The SCMs ha e e ol ed o er the years to
su ort arallel de elo ment, ro idin efficient ways or clearly
understanding the state of the code for any level. Modern build
tools ha e de elo ed that aren t art o the SCM but allow users
to easily select the ri ht build tool and the ri ht SCM or the wor .
These build tools have developed to provide complex build pro-
cesses without complex scripting requirements. The list goes on
for the new advancements the distributed teams have had while
the z teams continued to use the same tools with very limited
enhancements o er the years. Some e am les include code or
iPhone, Android Studio or Android de elo ment, IBM De el-
o er or Systems or /OS de elo ment, or web de elo ment
environments associated with cloud-based platform-as-a-service
PaaS en ironments, such as Bluemi .

With the distributed teams modernizing and taking advantage

of modern tools and practices, they’ve been able to deliver code
faster and share skills across teams. The mainframe team, by
staying with its old tools, has limited its ability to share capa-
bilities and increase speed. The distributed teams have learned
through this that changing tools isn’t so hard and not something
to worry about. The tools can change to improve the process
without breaking the application.

Siloed Teams
The fragmentation of tools and skills discussed in the preceding
section has led over the years to the siloed teams you see today.
These organizational silos are based on one of the platforms the
solution is tar eted or. Some e ce tions e ist to this where teams
are buildin a a a lications that mi ht run on di erent lat orms
or some de elo ment teams s ecifically build code to su ort mul-
tiple platforms with C/C++. But this is much less common.

Silos enerally e ist or the mobile teams, distributed teams, and

se arate /OS teams. The a lications the business needs cross
all these teams, but the teams continue to work separately; there-
fore, collaboration, if any even exists, is limited. The interfaces
between these teams are where the hardest problems develop,
largely because the interfaces are more afterthoughts than
intended design.

20 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 IN THIS CHAPTER
» nderstandin at DevOps is

» Loo in at t e importance of DevOps for

mainframe environments

» oin t rou t e balancin act People,

processes, and tools

» Solvin problems it DevOps

» Improvin application development it

DevOps

Chapter 3
DevOps and the
Mainframe: Mission
Possible?

I
f you’ve been reading this book straight through, you’ve read
about the challenges in the mainframe environment as well as
the need for speed in addressing business needs. This require-
ment for change is why this chapter addresses this need, and
De O s is the way to do it. In this cha ter, I define De O s and
how it applies to complex environments.

De nin DevOps
DevOps is about bringing the principles of Agile and Lean IT to the
entire enterprise, from the initial business users, to development
and test, through to operations. The term DevOps comes from the
merging of Development and Operations, but it’s important to
understand it’s more than that. DevOps is about an organizational
strategy to manage the entire life cycle from requirements to a

CHAPTER 3 DevOps and the Mainframe: Mission Possible? 21

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 running system, across all platforms and teams. DevOps is about
the people and processes used. Tools help support the processes
and people.

DevOps includes such practices as continuous planning, collab-

orative development, continuous integration, continuous delivery,
continuous testing, and continuous feedback. DevOps is all about
bringing automation, transparency, and collaboration to the entire
organization.

Notice that I said it’s a strategy as well as about people and pro-
cesses. DevOps is a transformation in the way people work and
how they work together. It’s about breaking down the silos that
ha e built u between di erent rou s o er the years. Also, it s
important to understand that while tools help, use of tools doesn’t
make an organization DevOps.

To understand DevOps, it’s important to understand a little of the

history behind how the development and operations organiza-
tions have developed. Over the years a virtual brick wall devel-
oped between the development and operations teams, though not
always a virtual one. Development had its own set of practices
and tools, which evolved over time. Operations had its own prac-
tices and tools, e ol in in di erent ways. De elo ment ocused
on producing more function faster, while Operations focused on
controlling the environment to improve availability. A goal of
DevOps is to break down the wall and instead have development
and operations working together, collaborating to provide more
and more timely business alue. This is where Fi ure comes
into play. One side shows a developer with a machine, while the
other side is users with servers showing operations. The brick wall
needs to disappear for collaboration to exist.

FIGURE -1: The brick wall between Development and Operations.

DevOps is a broad topic. To more clearly understand it, you can

break it down into a set of areas:

22 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 » Culture
» Think
» Code
» Deliver
» Run
» Manage
» Learn

Ta e a loo at Fi ure . It shows the continuous loo with Cul-

ture in the middle. Culture is at the heart o all De O s.

FIGURE -2: DevOps Continuous feedback loop.

ulture
Culture is at the heart o any or ani ation the culture dictates i
the transformation brings value or you just change tools. Setting
the right understanding across the organization and backing it up
with practices and rewards is critical to gain the business value.
Those companies that have transformed indicate that cultural
change is what gave the organizations new life and excitement.

CHAPTER 3 DevOps and the Mainframe: Mission Possible?

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Think
Think is the first area o De O s and re resents the areas related to
driving the overall business direction and receiving user feedback.
Businesses need to be more agile, responding to the ever-changing
client desires and government regulations. Achieving this goal of
agility requires the ability to deliver quickly, but both high quality
and appropriate feedback ensures what’s being delivered is actu-
ally going to satisfy the requirements. This need to deliver fast
change with immediate customer feedback as well as high quality
presents a real challenge.

The current de elo ment en ironment consists o di erent teams

operating in silos, making it harder to collaborate. The siloed
environment forces extra processes, many of which are manual
and slow down the delivery of function. The information required
to plan and replan appropriately is fragmented across the teams,
o ten in di erent tools. In addition, many teams see this lan-
ning and replanning as governance overhead that slows the teams
down instead of an activity to help them deliver value with speed.

DevOps helps reconcile the competing perspectives of deliver-

ing with speed and reducing outages while providing high-value
solutions through multiple activities. It focuses on continuous
eedbac into the business lannin cycle, identifies the ey areas
for development, and removes waste in unnecessary development.

Code
The code area focuses on all the practices related to development
and test, including collaborative development and continuous
integration.

Collaborative development
Software development practices, which have evolved over the
years, ha e been ormali ed into many di erent disci lines, such
as development, test, quality assurance, performance, and busi-
ness analysis. In larger shops, separate teams often handle these
disci lines. This leads to many di erent rou s ha in to wor
together to deliver the end result. In addition to the teams orga-
nized by discipline, the advent of geographically dispersed teams
and outsourcin o arts o de elo ment ha e si nificantly com-
plicated the situation.

24 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Collaborati e de elo ment s oal is to brin all these disci lines
to ether in cross unctional teams to more efficiently roduce
business value. This cross-functional team is supported by tool-
ing to allow people, no matter their location around the globe,
to truly collaborate together, sharing ideas and providing the
transparency needed for the entire organization. In the best of
all worlds, this cross functional team, including operations and
business analysts, would sit physically together to take advantage
of the natural collaboration possible through colocation. How-
ever, in today’s world that’s rarely possible, so tooling needs to
be used to bring the team together virtually.

Building cross-functional teams for collaboration doesn’t neces-

sarily imply or require an organizational reorganization, nor does
it imply the removal of the separation of duties requirements.
Di erent indi iduals need to be res onsible or di erent tas s
they ust wor to ether in this cross unctional team. Chan -
ing the organizational structure from the very beginning isn’t
required, but over time, changing the organization will become
critical to the continued expansion of DevOps across the entire
organization.

A core concept of collaborative development is continuous inte-

ration, shown in Fi ure . Continuous inte ration im lies
individual changes are integrated early to drive out integration
roblems sooner. Continuous inte ration is su orted throu h
tooling to allow the teams to deliver and easily build, deploy, and
test the solution through automation.

FIGURE - : Continuous integration representing the bringing together

of the build for each team and final integration between teams

CHAPTER 3 DevOps and the Mainframe: Mission Possible? 25

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 ontinuous testin
Continuous inte ration is hel ul only with continuous testin .
Continuous testin ro ides the ollowin

» Early validation that the new changes don’t break existing

behavior
» Validation of new capabilities through automated unit
testing
» Validation of integration between the various changes being
made throughout the team
» Continuous testing of applications as changes are introduced
to more uickly and easily identify the change with a problem

Continuous testin re uires automated testin that s de elo ed

and maintained. Automated interface testing also helps introduce
the ability to test the interfaces between parts of the application
sooner and more frequently. The concept of “shift left testing”
or the testing of interfaces earlier in the development cycle helps
identi y those harder to fi roblems earlier and hel s im ro e
the inter ace definition so that both the caller and called ro-
rams ha e more clearly defined interactions.

Automation needs to be available at all levels of testing from unit

test, to function test, system test, and to performance and scal-
ability testing.

The first or lowest le el o testin is unit testin , which should

be used to make sure all new lines of code are covered in the test.
Tools such as jUnit or zUnit can be used to build up individual
tests that can be run as part of new development with concepts
such as Test Driven development. They can also be added to the
regression bucket that’s run with each build.

Function test, the next level of testing, should begin with function
tests being built based on the requirements. This ensures tests
are created as function is built and they can be tested together in
the same iteration. Automating function tests at the API or inter-
face level can simplify the process of creation of the tests and
the maintenance of the tests. These tests can also be run with
code coverage reports to provide additional coverage of the exist-
ing code.

26 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 System testing sometimes referred to as integration test, per-
formance testing, and scalability testing are generally done in
environments that are similar to the production environment.

Deliver
Deliver is the area of DevOps that focuses on the automated release
and de loyment rocess. Continuous de loyment is one area that
many or ani ations mo in to De O s ocus on howe er, it s
important to consider that continuous deployment doesn’t have
to imply deployment into production. The deployment needs to
include all environments within the organization.

Many of the tools that are so called DevOps tools are created for this
area of continuous deployment. The goal is to develop a DevOps
i eline, shown in Fi ure , that allows the or ani ation to
move changes through the pipeline in an automated manner as far
as the business requires.

FIGURE - : The DevOps pipeline.

Mana e
The ne t area, which is mana e, includes both Continuous Moni-
toring and Optimization, both providing feedback into the contin-
uous lannin rocess. Continuous monitorin ro ides the data
and metrics to operations, quality assurance, development, and
line-of-business. It’s important to remember this monitoring
isn’t done just in production, but it’s done in all environments to
provide feedback as early as possible. It may not be reasonable to

CHAPTER 3 DevOps and the Mainframe: Mission Possible? 27

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 run production-monitoring tools on each development machine,
but the same monitoring tools that run in production should be
run on the test environments.

The most important data a development organization can receive

about an application is the data on how users actually use the
application and any direct customer feedback. This information
needs to be captured and not just used by Operations in tuning
the system. It should also be feedback to the application design to
improve the overall user experience.

One simple example would be noticing that on days when employ-

ees might logically be receiving paychecks, the number of requests
or current balances si nificantly increases. This can indicate to
development that providing a simple way of always showing the
actual current balance through something such as the Passbook
on the iPhone or PassWallet on the Android provides the func-
tion the users are looking for easier, without the same load on the
back-end system.

Learn
The last area is learn in many ways learn should ust be an o erlay
of the entire process. One key goal of DevOps is to get feedback as
quickly as possible and as often as possible. Do whatever it takes
to experiment to provide the best possible solutions for your users.

y DevOps Is ritical for Mainframe

Environments
To understand why DevOps is critical for mainframe environ-
ments, you need to start with the development side. DevOps is
about breaking down the silos, and nowhere are the silos more
separated than between the mainframe teams and the distributed
or mobile development teams. Separation develops for many rea-
sons, but in the case of the distributed and mainframe teams, the
se aration was caused artially by di erent tools, lan ua es, and
development practices.

Over the years the tools, languages, and practices continued to

e ol e or the distributed teams thin OS/ , smalltal , C/C++,

28 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Java, Javascript, Python, and so on. Practices such as Agile and
Test Driven development evolved in the distributed space, while
the mainframe development teams generally continued to prac-
tice water all methodolo ies and de elo in COBOL and PL/ .
Because the practices and languages were not changing, neither
did the tools. De elo ers became ery roficient at buildin code
in Interactive System Productivity Facility (ISPF).

For those of you who don’t know ISPF, it’s the green screen menu
system a ailable or /OS, shown in Fi ure .

FIGURE - : The ISPF.

ISPF provides the editor and the interface that allow others to
build the tools for development and operations. Many vendors
built library management systems and production control sys-
tems to control the source code and load modules for deployment
into production. These tools were generally built for Operations to
use to manage the environment and were then used by develop-
ment as the tool was already available.

When I started in ISPF de elo ment years a o, I used the same

tools that many /OS de elo ers are still usin today. Thin about
that. What else in this world is the same as years a o I you re
a mainframe developer, you may be thinking, “Yes, but it works,
so why chan e It may wor or you, but does it wor or your
child Remember these systems ro ide critical business alue
and need to be maintained and updated. The mainframe can be
the system of innovation for the company, but does your child

CHAPTER 3 DevOps and the Mainframe: Mission Possible? 29

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 want to use the tools and process you use, or does she want to use
the modern tools she is used to Main rame or ani ations need
to take advantage of the DevOps transformation in order to stay
relevant.

Current /OS de elo ers are com ortable with ISPF and the cur-
rent practices, and, yes, it’s possible for new developers to learn
them as well, but to continue to row and maintain the /OS
development, you must modernize the tools. The current products
weren’t built to support modern development practices, and new
developers expect basic functions provided in modern tools that
ust aren t a ailable with the traditional /OS de elo ment. This
is a si nificant chan e, but the alue in not ha in to worry about
lack of future workforce alone makes that investment worth it.

The other important part is the move from waterfall to agile.

Waterfall process increases delays and reduces quality by separat-
ing the people doing the individual activities and delaying testing
among many other things. The move to agile processes improves
the quality of delivery by getting feedback faster and having an
integrated team working together. For the traditional mainframe
applications, you may be wondering how this is possible with
the monolithic code base, but in fact, this makes agile even more
important. Making small changes and having a test suite to test
those changes quickly reduces risk by quickly verifying the exist-
ing function.

Now loo at the o erations side. A ain you will find se aration o
teams between mainframe and distributed. The mainframe oper-
ations teams have built up procedures over the years to ensure
availability and reliability. In order to do this, generally they have
reduced the times when changes are introduced into the sys-
tem and have created release windows. During these windows,
the changes that have been approved by the appropriate date will
be deployed. These change windows are sometimes called release
parties, where all participants involved are on a bridge call for the
entire time, coordinating each part of the change to ensure all
related changes are timed in the right order, across all platforms.
Most mainframe applications today have some front-end system
or system o en a ement that ro ides the inter ace. Chan es
to these parts need to be coordinated to ensure the system cor-
rectly functions. This coordination is done through the bridge call.
Because the teams ha e di erent rocesses and tools, eo le ha e
to do the coordination.

DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Many times there is a third team in ol ed in this icture as well
the Quality Assurance team. Chec out Fi ure .

FIGURE - : A look at the virtual brick walls between the Dev, QA, and
Ops teams

These teams can be named di erent thin s, but they re the

groups responsible for testing the changes before they go into
production. This third group has its own tools and processes. This
leads to more silos and more virtual brick walls as shown in the
fi ure Now you ha e bric walls between De elo ment, QA, and
OPS and the a lication is fi urati ely thrown o er the wall to the
next team.

e Balancin Act People,

Processes, and ools
The key point to a DevOps transformation is that it’s truly about
people and processes, where tools are used to help. Adopting the
best collaboration and automation tools is useless without the
cultural change associated with DevOps.

Cross unctional teams ocused on deli ery characteri e a De O s

culture. These teams focus on business objectives and place high
value on early feedback, transparency, and automation.

People
Many eo le find chan e hard. De O s is about chan in the way
people work to be more collaborative and cross teaming. Building
a De O s culture isn t li e ado tin a tool or a chan ed rocess
it’s changing the focus of the organization to the end delivery and
building trust across the various groups.

CHAPTER 3 DevOps and the Mainframe: Mission Possible?

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 People have to be measured appropriately. Traditionally, devel-
opment has been measured based on new function availability
while operations has been measured based on availability of the
systems. These can be seen as competing metrics. In DevOps,
everyone should be measured on delivery of new function with
availability of the environment. Everyone is therefore focused on
the same end oal and can wor to ether more efficiently.

Building a DevOps culture requires change throughout the orga-

nization, from the top leaders on down. One way to help with this
change is to start with a cross-functional team on a project, and
have this team be the example to learn and develop the practices.
As this team ains the benefits o the De O s trans ormation, it
can begin the grassroots campaign to change the overall culture.
Winning examples are always the best way to promote organiza-
tional change.

Sometimes eo le are unwillin to chan e i so, they may need to

be moved to other opportunities within the organization.

Processes
In the preceding section, I discuss people and the cultural aspects
o De O s. Processes define what the eo le actually do. A De O s
culture is the first critical as ect, but you also need to chan e
the rocesses to fit with this new culture. Numerous rocesses
are associated with the entire DevOps model (too many for this
book to cover). I’ve addressed continuous integration, continuous
deployment, and shift-left testing. One additional process I want
to discuss is the o erall Chan e Mana ement rocess.

Chan e Mana ement is a ey rocess or all businesses, and there

are many aspects to change management. The primary objec-
ti e o Chan e Mana ement is to enable beneficial chan es to be
made, with minimum disru tion to IT Ser ices. Chan e Mana e-
ment addresses the entire scope of the change process. Within
this there are various aspects of change that should be addressed.
A ter readin the definition o Chan e Mana ement rom ITIL,
it’s important to understand that DevOps helps organizations
satisfy this requirement through the automation and early valida-
tion to ensure changes are successful and provide business value.

DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Tools
DevOps is not tools, but tools do help support the transformation.
Tools are required to support the automation, transparency, and
collaboration required as part of DevOps. When determining what
tools will be used, it’s necessary to take an enterprise view. Most
organizations today use multiple languages and run on multiple
platforms. The tools selected need to support all those languages
and platforms in use. If the tools can’t support all the languages
and platforms then those teams are left out of the collabora-
tion and silos remain. Collaboration tools need to s an the entire
organization providing the transparency and integration with the
other change management tools or help desk related tools. The
deployment tooling should support all the platforms, allowing a
coordinated deployment without the need of a set of people to
coordinate between platforms.

S ecifically addressin the tools or the main rame ortions o

the organization will be important to move to modern tools to
support the new development practices.

DevOps has evolved over the last number of years. At the begin-
ning, many organizations thought allowing teams to select their
own toolin would ro ide the best solution others loo ed or
more standardization. Now it has become clear through many
di erent com anies trans ormations that some tools need to be
standardi ed, while others can ha e more e ibility. Se eral tools
are needed for standardization:

» e Source ode Mana er S M This tool is one that’s

critical to standardize on. Getting to one organizational SCM
should be the goal. This provides a single central repository
to share organization assets, search for reuse, and build
tooling around.
» e ontinuous Inte ration ontinuous Deployment
I D coordinator The CI/CD coordinator is another of
the critical tools for standardization. Having a standard
pipeline coordinator allows all teams to focus on their own
processes and not supporting multiple tools, as well as
allows the collection of central metrics.

CHAPTER 3 DevOps and the Mainframe: Mission Possible?

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 » e or trac in system This single tool tracks work
across teams, has a managed backlog, and coordinates
collaboration all of which helps organizations maintain a
clear understanding of business work to be accomplished.
Having a single standard tracking system across the entire
organization, including operations, may not be realistic. For
example, there will be a help desk system that collects
problems as well. However, having the systems integrated so
the help desk tickets are replicated into the problem tracking
system used ensures a single managed backlog of defects
and enhancements Having di erent and separate tracking
systems makes managing the work in progress for a team
much more di cult and causes confusion on priorities
» e test mana ement tool The test management tool
provides a central repository for tracking and management
of tests, which allows for reuse and tracking.
» Deployment capability Application deployment for all
re uired platforms simplifies the overall process and
management. This is critical as a single managed system for
the organization. This integrates with the CI/CD coordinator
to provide clarity of deployment.
» IDE and speci c test automation tools These need the
most exibility because they’re tied more to the language
being developed and the system. This area is where
individual preference can make a significant di erence I like
to say it’s not worth fighting the editor battle People have
their favorite editor; let them use it.
» ode covera e tools Code coverage tools are re uired and
should be part of the toolbox provided to the developer, but
an appropriate code coverage tool for each language and
platform is re uired This is an example of where multiple
tools are re uired but all for the same functionality

These tools are co ered in more detail in Cha ter .

What Problems Does DevOps Solve?

Many organizations have understood that the number of changes
should be reduced, so process and procedures have been put in
place over the years to reduce the number of times that changes

DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 are made in the production environment. This delays the deploy-
ment of business value and gives the false impression that the
mainframe is slow. This causes organizations to implement func-
tion on other platforms, which then causes additional complexity
in the application environment.

By adopting DevOps practices across the entire organization,

changes can be delivered for the appropriate platform with the
speed required for today’s businesses.

Increased automated testing with proper code coverage indica-

tions can help reduce the time spent testing while decreasing the
risk of those changes. Introducing fewer smaller changes into the
environment more often brings business value faster.

Building function in smaller pieces, by focusing on user-centric

function and getting early feedback, and validating that function
to ensure it satisfies the user needs early and o ten, allows or
smaller incremental change delivering business value faster.

Improvin Application Development

it DevOps
DevOps with the principles of automation and incremental
improvements help transition the overall development process.
This helps all teams improve their ability to deliver business value
faster. For the mainframe teams in particular, as comfort with the
automated interface testing increases, the ability to make changes
in the back-end application design can also increase. This auto-
mated inter ace testin allows de elo ers to more confidently
make major structural changes to back-end applications.

Many a lications that ha e been de elo ed o er the last years

have become large and monolithic. With complete automated
testin , de elo ers can ha e the confidence to ma e structural
changes. With application understanding tools, the developers
can understand how the arts fit to ether, understand the de en-
dencies between applications, and ensure the tests are covering
what is required. Also with application understanding capability,
applications can now be split up, redesigned, and the no-longer-
used code can be removed. These improved applications become

CHAPTER 3 DevOps and the Mainframe: Mission Possible?

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 easier to maintain and enhance. This allows developers to focus
more on the delivery of business value.

Another key principle of DevOps, building the smallest increment

of value and validating it with end-users, can now more easily
be accom lished with the confidence o bein able to re ression
test the existing capability. This small incremental value is many
times referred to as the Minimum Viable Product (MVP). It’s impor-
tant not to be confused by the use of the word product. The goal is
to build the smallest change that has incremental value that can
be demonstrated to the end-user.

Not only is DevOps on the mainframe Mission Possible, but

also it s Mission Critical in order to enable continuous so tware
delivery to respond to business requirements and build competi-
tive advantage. This transformation allows your business to gain
the value from the resources that are seemingly locked in the
mainframe.

DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 IN THIS CHAPTER
» Looking at the DevOps best practices for
z Systems

» Using adoption for greater productivity

and optimization of systems

Chapter 4
Looking at DevOps Best
Practices for z Systems

I
you read Cha ter be ore comin here, you disco ered what
De O s is and why it s im ortant to main rame de elo ment.
Don t worry You don t ha e to read that cha ter be ore this
one, but in this cha ter, I ocus on some best ractices or this
transition.

Adopting DevOps Best Practices

for z Systems
When ado tin De O s, you should consider a number o areas.
I co er those areas in this section with some recommended best
ractices.

De O s is about cultural chan e, brea in down the silos, and

mo in to A ile and Lean ractices across the or ani ation. Some
o these best ractices, such as buildin minimum iable roducts
MVPs , I discuss earlier in the boo . The e am les in this cha ter
are some o the best ractices, but they aren t a com lete list that
would re uire a much lon er boo .

CHAPTER 4 Looking at DevOps Best Practices for z Systems 37

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Modern development practices
and tools
Probably the most im ortant ste in the De O s trans ormation
or or ani ations with Systems is to a ly modern tools and
de elo ment ractices within the de elo ment teams. Tryin to
brea down the silos without this ste is ery difficult.

This transition to modern tools and ractices can also be a reat

way to start the transition to the De O s culture in act, your
teams may be the leaders in trans ormation in the or ani ation
throu h their si nificant trans ormation. They ha e the arthest
to o, so startin with the modern ractices and tools is a ey
actor

» The first important practice is agile development Notice that

I didn’t capitalize the a in agile that’s because it’s not
that you have to be exactly Agile it’s that adopting agile
practices help improve software uality and improve speed
There have been many papers written about the death of
waterfall, but unfortunately it’s still practiced in many shops
There are many studies that show that waterfall hasn’t
helped improve software development processes, and
actually it has done the opposite Having silos between
teams and trying to define everything upfront isn’t possible
The re uirements demanded by the business today may not
be the top re uirements for the business even a few weeks
later Having a managed backlog and working through items
one by one as small enhancements help ensure that the
correct functions are built, and time isn’t spent on extras that
aren’t needed anyway Agile incorporates many di erent
practices, too many to discuss here, but the practices are the
first place to start
» The next modern tool is the Integrated Development
Environment IDE Having modern editors and tools that
help with application understanding allows z developers to
work more e ciently and increases their productivity This
helps the existing team understand the code, as well as
helps new members of the team more easily understand the
existing monolithic systems Providing a modern IDE also
doesn’t re uire a cultural change or any process change it
provides a set of productivity helpers and allows new
developers to come up to speed on the existing code
much faster

38 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Providing a modern IDE is critical, but don’t push it as a new
editor People have their own favorite editor, so let them use
it The key to the IDE is the new capabilities it should provide
for application understanding and easy access to the z
System Using z/OS explorer as a base for all users of z/OS is
a good start It’s provided for anyone who has a z system, so
it’s not an extra purchase, but it’s the provided modern
interface for developers and system programmers The
additional development tools should fit into z/OS explorer
» Another area is a modern source code manager SCM
Traditional host-based SCM solutions are generally more
library managers than SCM solutions Modern SCM solutions
allow developers to more easily work with their changes and
merge those changes with other changes also in progress
Supporting parallel development e ciently is key to these
modern SCM solutions The best modern SCM for your
organization will be one that can support all the languages
and platforms for the organization
On z/OS, you can work in a variety of languages, including
COBOL or PL/I but also ava or possibly Swift With this
language variety, it’s critical to have an SCM that easily works
with all these languages in the way users expect Having
COBOL source in the same SCM as the ava code allows
developers to see the interfaces used between the two and
makes available to everyone the logic of the business rules
Another consideration is the file systems on z/OS most
traditional host SCMs were designed and built for partitioned
data sets PDSs However, as the system has evolved, the
hierarchal file system HFS , which is a more natural file
system for most people because it’s ust like any other system
they work on, has developed and needs to be supported as
a first-class resource The modern SCMs support the HFS
naturally
» Collaborative development tooling is the next area of focus
and should again support all languages and platforms All
teams should be able to work within the same solution so
they can collaborate across the organization I describe this
capability in Chapter 3 The key from a mainframe perspec-
tive is that it’s included Providing all of this in dashboards
available to the entire company helps facilitate the goal of
everyone being responsible for delivery

CHAPTER 4 Looking at DevOps Best Practices for z Systems 39

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 » For mainframe development, another re uired capability is
application understanding Existing mainframe applications
are generally monolithic and not well documented Having
application understanding tools helps teams understand the
dependencies between applications as well as understand
the ow of individual programs Clearly understanding the
application makes it much easier for anyone to update the
application and estimate the time re uired to do so
» Modern testing tools, such as unit test capability, code
coverage, and code rules enforcement, are also critical to
this process improvement With tools to help teams in these
areas, the adoption of automated testing increases
» In addition, tooling to help generate interface tests and
virtual service definitions allows the teams to work indepen-
dently but together at the same time

These tools su ort the transition to the modern de elo ment

ractices, allowin teams to wor to ether across the lat orm
boundaries. With these tools, teams can more easily ado t rac-
tices such as Test Dri en De elo ment, de elo ment o MVPs,
and iterati e de elo ment.

Test environments
Pro idin sufficient test en ironments or indi idual teams to be
able to do their de elo ment and test, without delays, is the ne t
ey or the De O s trans ormation. It doesn t hel to build unction
in small increments or im ro e the roducti ity o the de elo -
ment team i there isn t an en ironment or them to de elo and
run their automated tests. Today, many or ani ations ha e shared
test en ironments that are scheduled between the multi le teams.
Fi ure shows a ty ical testin en ironment today with multi le
teams sharin the same en ironments.

Test data management

Test data mana ement is an area o ten le t out in the discussion
o automated testin . As automated tests are created, the associ-
ated test data must also be de elo ed to be able to run the test and
alidate the results. Test data should be created with all le els o
tests, unit, unction, er ormance, and scalability.

40 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 FIGURE -1: A typical z/OS testing environment showing sharing of
resources

There are many ways to create test data, and di erent creation
and amounts o data are re uired or di erent le els

» For unit and function testing, test data should be fabricated,

and it should be as small as possible As the environments
come closer to production, more test data may be re uired
» For performance and scalability testing, use production data
that has been obfuscated or masked
Never ust use production data in ust about every case, it
will include some private information that shouldn’t be
accessible to the development and test individuals

One area enerally lac in is ne ati e testin or ailure testin .

This is es ecially true when usin roduction data that has been
scrubbed. Additional data must be added or the data that can t
et into the system. Many teams build the ha y ath tests but
ail to ocus on the bad data or boundary conditions. Both or unit
testin and or unctional testin , ha in these ne ati e tests is
critical to ensure com lete code co era e. The other area o ne a-
ti e testin is or security testin addin e treme data and tryin
or bu er o er ows are critical arts o testin that many times
et le t out o the main rame testin . With the ocus on brin in

CHAPTER 4 Looking at DevOps Best Practices for z Systems

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 main rame a lications to be directly e osed to mobile a lica-
tions, security becomes more critical. All o this test data needs to
be art o the test data mana ement lan.

The oal o the automated testin should be to ro ide er-

cent code co era e. The ercent oal may ne er be achie able
with today s e istin systems because some ercenta e o it is
robably ne er reachable. It s im ortant to come u with the er-
centa e you re com ortable with, and understand what isn t bein
tested. Realistically, com anies start with current code co era e
o ercent and add such oals as ercent code co era e
o the new code, then slowly increase the co era e o e istin or
herita e code.

Interface testing and virtual services

One other ractice is to use inter ace tests to disco er inte ration
roblems sooner and to im ro e code co era e. Testin ia a li-
cation ro rammable inter aces APIs where er ossible hel s
im ro e the test co era e without buildin the more ra ile user
inter ace tests that may need to be u dated with each chan e in
the end user inter ace. Fi ure shows how irtual ser ices can
be used to test ortions o an a lication inde endently.

FIGURE -2: The use of virtual services for di erent parts of an application

42 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 For a Re resentational State Trans er REST inter ace that will
be used between a mobile a lication and a bac end system, the
teams can define the a aScri t Ob ect Notation SON schema
to be used or the REST call. With this definition, a irtual ser-
ice can be created to allow the mobile team to de elo the ront
end without ha in access to a runnin bac end system. The
inter ace can also be used to create a test case that s used by the
bac end team as it de elo s the bac end ca ability. By usin
this same definition, the li elihood o inte ration roblems si -
nificantly decreases.

Common deployment
Common de loyment ca ability is another critical area to be
addressed. With the com le nature o a lications, ma in sure a
sin le automated de loyment ca ability is ro ided or all arts o
the a lication and used in all hases o the de elo ment rocess
si nificantly hel s reduce de loyment issues. This a lication
de loyment should also include the confi uration re uired or any
middleware com onent as art o the de loyment. Ima ine ha -
in a sin le tool that does the de loyment o the mobile, mid tier,
and bac end system to ether in e ery hase o de elo ment. By
the time the ca ability is de loyed to roduction, it may ha e been
de loyed a hundred times in earlier en ironments, thereby disco -
erin any de loyment issues earlier. This also ensures that com-
onents actually tested to ether were actually de loyed to ether.

Improving Productivity and

Optimizin Systems
The first hal o this cha ter discusses some o the best ractices
in De O s ado tion. In this section, I let you now how ado -
tion leads to reater roducti ity and allows the o timi ation o
systems.

Improving productivity
One o the sim lest roducti ity ains occurs with the ado tion
o automated testin . Many or ani ations see a si nificant ain
in the trans ormation rom mostly manual testin to automated
testin . The first actor can be the chan e in s ills re uired.

CHAPTER 4 Looking at DevOps Best Practices for z Systems 43

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Instead o ta in the time rom the line o business or the basic
manual erification testin , these indi iduals can ocus on eri-
fication o the deli ered alue, with already tested ca ability.
They can now ocus on the alue ro ided or erification and can
s end most o their time on business acti ities.

With this transition to automated testin , de elo ment s ills are

enerally re uired to build the tests. In the test dri en de elo -
ment and a ile de elo ment strate ies, de elo ers themsel es
build more automated testin , and members o the team may
cycle throu h buildin automated tests and buildin business
ca ability. While the indi idual unit tests or unction should be
de elo ed by the de elo er creatin the ca ability, unctional
erification should be created by di erent indi iduals within the
team so the tests are written based on the re uirements, not the
im lemented code.

With automated testin bein de elo ed into the builds, de elo -

ers roducti ity increases by recei in almost immediate eed-
bac on their code. In addition to the roducti ity ained throu h
automated testin , modern tools brin si nificant indi idual ro-
ducti ity ains or de elo ers who e been usin ISPF and ISPF
based tools.

Another area or roducti ity ains come rom a lication under-

standin tools. By ro idin a tool that enerates a lication ow,
im act analysis, and data ow, users can more uic ly under-
stand where to ma e chan es and how to ma e the chan es. With
im act analysis, you can also identi y which ro rams are related
to each other, allowin you to understand what areas need test-
in when ma in a chan e. This reduces the testin re uired and
allows the initial testin to ocus on the s ecific areas o im act to
et eedbac uic ly. A isual o an a lication disco ery browser
is shown in Fi ure .

Producti ity ains aren t seen immediately. Education is re uired

to learn to use the new tools and methods. Most indi iduals will
decrease their roducti ity at first in the transition to the mod-
ern tools, but their roducti ity will si nificantly increase as their
amiliarity with the new tools rows. Indi idual e erience ar-
ies, but in all studies that ha e been er ormed, the roducti ity
increases are measurable.

44 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 FIGURE - : An application discovery browser

Pro ide o to e erts on the new tools or the team. Ha e some-

one who has already learned the new tools assist others within a
rou to learn the ca abilities.

Optimizing systems
With the new de elo ment tools, ractices, and automated test-
in there comes a reat o ortunity to o timi e the en ironment.
For main rame en ironments, the sim lest first ste is to ta e
ad anta e o the o timi ations by usin the latest releases o the
com liers. Each release o the main rame hardware has de elo ed
new ca abilities, and many o these ca abilities re uire the use o
the new com liers to ain the ull ad anta e. Recom ilin ro-
rams that ha en t been com iled or years is the first bi ste .

You may as why these ro rams ha en t been recom iled. Due to

the desire to decrease ris , many main rame or ani ations build
only the chan ed ro rams. I some e istin ro rams ha en t
needed to chan e, they ha en t been recom iled. Now, with the
automated testin ca ability, recom ilin these ro rams intro-
duces less ris .

E en with automated testin , it may not be acce table to recom-

ile all the ro rams at once. Focus on those ro rams that are
most o ten run. This is another area where de elo ment and
o erations need to come to ether, ro idin the data rom the
o erational en ironment on how o ten ro rams are actually

CHAPTER 4 Looking at DevOps Best Practices for z Systems 45

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 bein run o er time and how they indi idually er orm. This data
can be used to ma e the best decisions on where to o timi e first.

Another common ocus area or o timi ation is identi yin the

du licated solutions. Many or ani ations ha e rown throu h
ac uisition. Throu h the ac uisitions, multi le ersions o the
same unction e ist within the or ani ation. By identi yin the
arious du licated ca abilities, business can decide how best to
standardi e on common ca abilities, remo in the cost and e ort
o maintainin the multi le solutions.

46 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 IN THIS CHAPTER
» Looking at open-source tooling for
DevOps pipelines

» Understanding the pros and cons of

using open-source tools in an enterprise
environment

Chapter 5
Applying Open-Source
Tools to DevOps and
z Systems

W
hen you see presentations or conferences about DevOps,
you see many di erent o en source tools. Many
or ani ations ha e ocused on creatin o en source
i elines or their distributed de loyment rocess. Why is o en
source ha in such an e ect O en source is so tware with source
code that anyone can read, chan e, and enhance. The oal o
o en source so tware is to ro ide an o en e chan e, su ort
collaborati e artici ation, ro ide trans arency, and encoura e
community oriented de elo ment. Instead o a sin le or ani a
tion bein res onsible or all arts, indi iduals with an interest
can re iew and submit their own u dates or enhancements or
chan es.

O en source ro ects ha e been created or many di erent inds

o ro ects. O en source or de elo ment and De O s tools has
ained si nificant interest due to the number o eo le contribut
in to the solutions.

CHAPTER 5 Applyin Open Source ools to DevOps and z Systems 47

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 In this cha ter, I discuss some o the ey as ects and tools that
are bein used, and then you loo at the a licability to the Sys
tem. This cha ter also co ers o en source tools that ha e seen
si nificant ado tion in the mar et lace. This study isn t scien
tific it isn t intended to list all the ossible o en source o tions,
but it s based on my customer e erience and the con erences
I e attended.

Open-Source Pipeline
Most De O s i elines ha e a set o ey ca abilities see Cha ter
or more details . They include a Source Code Mana er SCM , a
build tool, a i eline orchestrator, a de loyment ca ability, and
a built arti act re ository. There are a number o other standard
tools such as a code uality mana er, test mana ers, test automa
tion tools, and the IDE used by the de elo er, but those ary much
more o ten and aren t enerally included in the standard i eline
re uired or use. Chec out Cha ter or more in ormation.

Git
Git is a o ular o en source source code mana er SCM , and
many di erent endors ro ide su orted ersions. This SCM has
become the de acto standard or the industry. It s ocus on shar
in code between users, and it s distributed su ort ma es it ideal
or de elo ment o o en source solutions themsel es.

A distributed SCM ta es a eer to eer a roach to the solution,

re licatin the entire source tree, includin all ersion in orma
tion to any system in a networ unli e a centrali ed SCM where
there s a central location that s the master with all the ersion
in ormation and indi idual users or build systems ullin the
ersion they want to wor with.

In Fi ure , you see the di erence between the centrali ed and

distributed SCM.

With com anies tryin to de elo a model o internal o en source,

this has increased the demand. Also Git has been laced at the
center o de elo ment toolin such that ust about any de elo er
already understands how it wor s.

48 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 FIGURE -1: A comparison of centralized and distributed SCMs.

Git does ha e its limitations. Because Git is a distributed SCM,

re licatin the entire code base to a local machine is easier, which
ma es it also easier or someone to wal o with the entire code
base o an a lication or or someone to ha e a la to stolen with
the entire code base. For many or ani ations, this security ris
limits their use o Git to non critical or ro rietary so tware. Oth
ers reduce the ris by re uirin irtual des to solutions, so the
code always resides within the data center. Most or ani ations
also re uire la to s to be encry ted, which reduces the ris i a
la to is lost.

Many eo le also ha e used their e istin SCM to store the build

out ut build output includes the e ecutables and many times
listin s enerated by com ilin the source code. This out ut is
lar e and not needed by de elo ers in their local co ies. By the
nature o Git re licatin e erythin , this ractice is no lon er a
ood idea, and the build out ut must be stored in a build out ut
re ository and lin ed to the source code. The build out ut re osi
tory can be tied to the de loyment system or can be a se arate
re ository. The ey is to ro ide a ro riate lin s between the
built out ut and the source code that was used to create it.

When usin Git as the SCM, store your build out ut in an arti
act re ository instead o the SCM and ro ide the ro er lin a es
between the systems.

CHAPTER 5 Applyin Open Source ools to DevOps and z Systems 49

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Build system
The build system is res onsible or com ilin the source code
and has been a tar et or a lar e number o o en source tools.
The most common build tool has been Ant. Ma en came alon to
im ro e Ant, and now Gradle ro ides e en more e ibility and
ca ability. Each o these tools ro ides the ca ability to define
how the source code should be rocessed by the com ilers. Tools
such as en ins are many times used as the build coordinator as
well as the i eline orchestrator usin Ant or another tool as the
actual build scri t.

Pipeline orchestrator
The i eline orchestrator is used to coordinate all the other acti
ities. For the i eline orchestrator, en ins is the most common
o en source coordinator, but Tra is CI has become more o ular
with smaller teams. en ins and Tra is CI ro ide the ca ability to
define a rocess ow and call other tools to er orm those actions.
They also ull the in ormation bac rom those tools to ro ide
dashboards o in ormation about the status o the i eline.

Deployment system
For actual de loyment tools, there hasn t been a si nificant
introduction o o en source tools. Howe er, containers such as
Doc er ha e e ol ed to sim li y de loyment. Doc er is an o en
source tool that ma es it easier to de loy a lications by creatin
a container that includes the a lication and re uired libraries. By
ac in all the arts to ether, all the de endencies are de loyed
to ether so the de elo er and o eration sta don t ha e to worry
about the ossible di erences in an o eratin system. This con
tainer can now run on any system.

Build artifact repository

A build artifact repository houses all the out ut rom the build
rocess, such as the e ecutables and the listin s roduced. This
arti act re ository will store all the arious ersions o the build
out ut and can also be used to store the e ecutables ro ided
by e ternal sources that may need to be de loyed. The arti act
re ository can also be used as in ut to the build rocess when
build out ut rom one art o an a lication is re uired as in ut
to another art o the a lication.

DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Arti actory and Ne us are the two most commonly discussed
o en source re ositories. Ha in an arti act re ository has
become a re uirement or all customers usin Git see the earlier
section, Git, in this cha ter or more in ormation to ha e a
stora e or the built a lications.

Open Source and z Development

O en source lays a lar er and lar er role in a lications and
in de elo ment tools. As the ecosystem e ands, Systems has
ocused on le era in o en source. When thin in about Sys
tems, howe er, there are di erent as ects to ee in mind

» The first is Linux on z, and when it comes to development

tooling, it should be the same as the rest of your Linux
development There is nothing about Linux on z that makes
the re uirements di erent
» The second is ava on z/OS, and for that environment, it’s the
same tooling as ava on any other platform

For both o these rior systems or lan ua es, many or ani ations
ha e chosen to use rimarily the same o en source toolin , i
their com any choice is o en source.

When you re thin in about traditional /OS de elo ment, answer

these uestions

» Is open source appropriate?

» Is it available?

In this section, you see these choices in detail and how they could
be used to wor or your or ani ation or com any.

SCM for z/OS source

On /OS traditional source code is in E tended Binary Coded Deci
mal Interchan e Code EBCDIC code a e and has a ery di erent
rocess or buildin . This code a e di erence is im ortant or
any SCM to understand and why o en source toolin that didn t
understand this has been more difficult to use. Than s to the
e orts o Roc et So tware, Git has been orted to /OS and now

CHAPTER 5 Applyin Open Source ools to DevOps and z Systems

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 has code a e su ort when runnin on the lat orm. It s been
done in a way that the code can be in EBCDIC on /OS but will be in
American Standard Code or In ormation Interchan e ASCII or
Unicode Trans ormation Format UTF or all other lat orms,
which allows /OS code to ully artici ate in the o en source
SCM s ace and to ma e it easier to o en source ser ices, usin
traditional /OS lan ua es such as COBOL and PL/I.

From an SCM stand oint, now Git can be used or /OS de elo
ment i you re interested in the distributed SCM ca abilities.

Build system
en ins has a /OS connector that allows or a build rocess on
/OS that can use the Git su ort. Other o en source build tools
ha en t ro ided /OS s ecific su ort. By usin this su ort, you
can submit traditional ob Control Lan ua e CL to do a tradi
tional /OS com ile or ic o other /OS based tools or the build
rocess.

Pipeline orchestrator
De endin on your build and de loyment system, your i eline
orchestrator ust has to be able to connect to them, so s ecific
/OS su ort isn t re uired. Com anies ha e used en ins as a
i eline orchestrator or /OS a lications. Tra is CI doesn t ha e
/OS su ort at this time.

Deployment system
De loyment re uires s ecific /OS su ort, and en ins ro ides
the connector but no s ecific de loyment ca ability or /OS
because no o en source tools e ist that su ort the traditional
/OS lan ua es. Doc er ima es aren t directly su orted on /OS
today howe er, Doc er li e unction is bein de elo ed to allow
or the easy creation o containers on /OS or a lications. As o
the time o this writin , the first ca ability is comin out or CICS
a lications to et early eedbac on the conce t or traditional
/OS a lications. This Doc er li e container builds on the act
that /OS is desi ned or runnin multi le a lications within a
sin le system and handlin the wor load a ro riately.

DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Built artifact repository
As lon as your build rocess can create an arti act that can be
mo ed o /OS or you create a secondary rocess that creates
such an archi e, any arti act re ository should be able to handle
/OS arti acts. The difficultly will be in creatin the archi e to
trans ort the files.

/OS e ecutable, nown as load modules that reside in PDSEs,

cannot sim ly be co ied to an arti act re ository they must be
e tracted in a orm that s trans ortable by usin the XMIT unc
tion, and then they can be com ressed to ether in a tar or a file
that can then be stored in an arti act re ository. Ha in a ac
a in system that understands how to do this ma es usin an
e ternal re ository much easier, but it s ossible to do with any
build system and custom scri tin .

CHAPTER 5 Applyin Open Source ools to DevOps and z Systems

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 IN THIS CHAPTER
» Getting to the API ecosystems in the
hybrid cloud

» Taking advantage of full application life

cycle management

Chapter 6
Building for the Digital
Economy

T
he digital economy is transforming the way business pro-
vides value to its end-users. Multiple interfaces are required
to reach the di erent ossible audiences. This chan e dri es
the need to expose more of the business value through application
programming interfaces (APIs) to not only support internal
demands, but also to support new external opportunities. In this
chapter, I discuss the implications for the mainframe applications
and how they can fully participate in this new environment.

Understanding How Mainframe Is at

t e eart of t e Di ital Economy
Teams can wor to ether in di erent ways. Common rocesses
and toolin are the first as ect o ha in teams wor to ether.
With a common source code environment, the teams can share
common arti acts such as inter ace definitions easily. Automated

CHAPTER 6 Building for the Digital Economy 55

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 inter ace testin is another as ect o how the teams can wor
to ether. By ha in inter ace definitions and buildin test cases
and irtual ser ices based on those inter aces, the teams can wor
inde endently o each other s schedules. This may seem at first
li e a counter oint to wor in to ether, but by ha in this inde-
pendence, teams can collaborate without having any one side hold
up the progress of the other.

The common toolin , rocesses, and inter ace testin are tools
and rocesses to hel , but how do they actually wor to ether to
ro ide alue This is where a lication desi n and collaboration
really come into play.

Essential to this discussion is understanding how the existing

a lications actually wor . There is si nificant alue loc ed in
the mainframe applications, but it’s sometimes hidden inside
a monolithic system. You can t ust o find the documentation,
because it s not there or not accurate. This is one lace where
a lication understandin tools are critical. These tools such
as Application Discovery help you understand the relationships,
understand the dependencies, and, most importantly, identify the
areas that should be exposed as an API or service to be used by
other applications.

Thin in bac to when these a lications were ori inally written

(something I can almost do, but some actually were started before
I was wor in in the industry , they were built as transactions
in either In ormation Mana ement System IMS or Customer
In ormation Control System CICS . These transactions er ormed
some s ecific unction, and the transactions were connected
to ether. These core transactions many times ro ide the actual
API now needed by a front-end application. Finding these trans-
actions and identi yin the arts that ma e u the ca abilities are
ey in bein able to e ose them or new business ur oses.

The ront end a lication, many times now a mobile a lica-

tion, provides the interaction with the end-user, but the mobile
a lications re uire data rom the bac end systems. The mobile
a lication wants to sim ly call a Re resentational State Trans er
REST ser ice to et the data or er orm the action re uired or
the end-user.

56 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 With mobile a lications, REST inter aces are the most common
way o connectin . These REST ser ices are ro ided in multi le
ways:

» Directly with REST services from CICS

» Through the use of CICS transaction gateway
» IMS gateway
» MQ
» z/OS Connect Extended Edition

/OS Connect ro ides a sim le way to add a ser ice inter ace to
an e istin a lication without chan in that a lication. The
ey here is without chan in the a lication. /OS Connect can
easily be used in combination with application understanding to
expose an existing set of capabilities to new business needs.

These ser ices that are e osed ia /OS Connect or other meth-
ods are many times a more IT centric iew o the ca ability. Some
of these may actually be business services that you want to expose
to a wider audience. For this, an API mana er such as API Connect
can be used to expose the services, and then secure, manage, and
meter them.

Mobile applications will change frequently in order to satisfy user

demand for constant change. Users expect updates for mobile
a lications, so the bac end systems ha e to su ort multi le
versions of the mobile application and browser-based applica-
tions at the same time. In order to do this with the least impact,
well defined inter aces or APIs are created. By wor in u ront
collaboratively on the design of the APIs, the APIs can evolve over
time without brea in the ront end a lications.

By definin and maintainin well defined APIs, the challen e o

deployment is also decreased. While it’s critical to be able to deploy
the full application together at any time, it’s just as important to
be able to deploy the individual components at their own pace.

These defined APIs ro ide the base or all the di erent inter-
actions with the systems of record, allowing organizations to
ro ide the arious inter aces re uired by their customers. These
APIs can also be provided as services to outside organizations
to ro ide additional business alue and enable efficient cloud
environments.

CHAPTER 6 Building for the Digital Economy 57

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 LEVERAGING DEVOPS SERVICES IN
THE CLOUD
Bluemix is a platform-as-a-service PaaS o ering from IBM that pro-
vides an e cient way to build the systems of engagement connected
to back-end services Bluemix is a combination of IBM, third-party,
and open-source technologies IBM Bluemix DevOps Services pro-
vides the application development environment for these systems of
engagement that’s based on the same technologies that can be used
for the in-house development

Consider a well defined and mana ed set o APIs rom the bac
end systems; now mobile developers using platforms such as
Bluemix can easily experiment and develop very robust appli-
cations based on the information available from the systems of
record. By providing these services or APIs to the Bluemix envi-
ronment, your organization can more easily and rapidly adapt to
chan es in the mar et lace. You can also use the same irtual
service capability to allow the developers in test to build against
those virtual services without having to have access to the real
service early on.

Looking at Full Application Life Cycle

Management
A lication Li e Cycle Mana ement ALM is a well defined
process for applications from idea, development, and mainte-
nance through retirement of the application. DevOps is a way of
in uencin the ALM rocesses to ro ide reater trans arency,
collaboration, and automation. The ALM rocesses need to be
u dated to ro ide the continuous eedbac cycle and ensure ull
transparency within the organization.

As I mention in Cha ter , De O s is about brin in the conce ts

o A ile and Lean IT to the ull or ani ation. This means your
ALM rocesses and rocedures need to be u dated to re ect these
changes. In the current ALM practices, it’s important to be mea-
suring the overall practice. In order to remove waste, you have to

58 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 understand where that waste is. Some possible measurements to
be capturing are

» Time to deploy the application or application changes in

each stage, development, test, and production, along with
the number of people re uired
» Time to find defects from when the developer checks in the
code Is it found within a few minutes with the build, or does
it take weeks before it’s even tested?
» Time to complete a regression test How much of it is
automated? How many people does it take to run the tests?
» Defects found by stage and type By stage I mean develop-
ment, test, or production by type I mean the kind of
defect was it a defect in design, misunderstanding of the
re uirements, coding defect, integration defect, and so on?
» Average time to get a new business re uirement into
production, from the initial re uirement identification
» Mean time to recovery when a problem occurs in
production
» Wait times, how long teams have to wait for test environ-
ments or systems

Select the a ro riate metrics or your or ani ation and ma e

sure you have some history for those metrics before the transi-
tion. These hel you clearly show the return on in estment in the
transition. For e am le, i it ta es fi e days to de loy the en i-
ronment today, but you can do it in fi e hours a ter automated
deployment tooling, it’s easy to see that return. Not all measure-
ments will be as si nificant as this, but they hel demonstrate
the value.

One im ortant note in the definition is retirement o the a lica-

tion. Retiring applications at the right time is almost as important
as is buildin the ri ht ca ability in the first lace. As or ani-
zations develop multiple channels for user access, the need for
clearly defined and mana ed ALM rocesses across all as ects o
the ca ability becomes e en more critical. The ALM rocesses and
procedures must be clearly updated to be supportive of the cul-
tural transformation.

CHAPTER 6 Building for the Digital Economy 59

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 60 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 IN THIS CHAPTER
» Starting with development
modernization

» Working toward automated testing and

deployment

» Practicing full adoption

Chapter 7
Seeing DevOps Success
Stories in the Enterprise

D
i erent or ani ations ta e di erent a roaches to their
De O s trans ormation. In this cha ter, you learn rom a
number o enter rise clients how they decided to ta e on
this trans ormation and disco er what they e ained. I ha e
selected ust a ew stories or this boo to ro ide some re resen-
tati e sam les. More are a ailable rom the lin s ro ided in the
be innin o the boo .

Development Modernization
Many e am les o or ani ations e ist that start their De O s
ourney with de elo ment moderni ation. This can be one o the
least or ani ational chan e related chan es and is a start or
many or ani ations as the first re uired ste or the main rame
art o the or ani ation. De elo ment moderni ation includes
the ado tion o modern tools and ractices or the main rame
de elo ers.

CHAPTER 7 Seeing DevOps Success Stories in the Enterprise 61

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Lar e nancial or anization
An e am le I would li e to start with is a lar e scale financial
or ani ation rimarily located in the United States. It has both
distributed and /OS de elo ment teams. De elo ment is located
in multi le sites, includin o shore howe er, teams are ener-
ally located to ether rather than s lit across the sites.

As in most ty ical main rame de elo ment or ani ations, this

lar e scale financial or ani ation was usin Interacti e System
Producti ity Facility ISPF with an e istin host source code
mana er SCM and de loyment ca ability. This tool set had been
used in basically the same way or the last years. The /OS
teams are com letely se arate rom the distributed teams. This
client has some small a lications with a ew de elo ers, and a
ery lar e a lication with a lar e de elo ment team. Honestly, I
would say the lar e a lication is fi e di erent a lications, but
the client has so many shared com onents the client wor s as i
it s one a lication. It has to be built and de loyed to ether.

This or ani ation needed to u date its SCM and build structure
due to the lar e si e o its critical a lication. In order to ma e
this transition, the com any would re uire a si nificant redesi n
and u date to the e istin en ironment, and when it was finished,
it would still be in the same toolin with the same old rocess.

The client assessed the e ort or the chan e to the e istin

en ironment and the e ort to mo e to a modern SCM and build
solution. Both e orts re uired si nificant e ort and time, but at
the end o the moderni ation ro ect, the com any would ha e
mo ed orward and ha e the ability to standardi e across the dis-
tributed and /OS de elo ment e orts on a sin le de elo ment
en ironment.

To sol e its roblem, the com any chose Rational Team Concert
RTC alon with IBM De elo er or Systems ID , ormerly
nown as Rational De elo er or Systems, as the solution and
lanned a sta ed mi ration. The mi ration lanned to mo e each
a lication team at a time, lea in the lar est a lication or
the last. In this case, the com any decided to start with the least
acti e ro ects in order to learn rom early de loyments. With the
scale o the mi ration and the number o users, the a lication
and team mi rations ha ened o er a cou le o years.

62 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 While the /OS teams were mi rated, distributed teams were also
mo ed into RTC to ro ide the a ile lannin toolin re uired or
their a ile trans ormation. A ter the /OS teams com leted their
mi ration, additional ilots were created to mo e to continuous
inte ration, startin with some o the distributed teams.

As with any u date, as the years ha e one by, additional chan es

ha e been made. The rior e am le was in the first edition o this
boo , and now a ew years later, the story has e ol ed. The teams
ha e u raded ersions o RTC and are ado tin new ca abili-
ties o ID , includin ca abilities such as Unit. They will also be
mo in to UrbanCode De loy or the de loy acti ities currently
done by RTC.

As the distributed side mo es to UrbanCode De loy as well, this

ro ides a sin le common de loyment ca ability. For the distrib-
uted side, as it mo es to more cloud de elo ment, it has decided
to im lement Github Enter rise, en ins, and UrbanCode De loy
inte rated with RTC. This will mo e the team entirely out o Clear
Case and ee RTC as its lannin , dashboardin , and audit oint.
RTC will be the system o record or all as ects o de elo ment,
test, and de loyment. Github Enter rise was selected to allow
de elo ers to re licate the entire code base with history to the
cloud en ironment but still ee an enter rise mana ed central
re ository.

This trans ormation is si nificant, so allow the teams a time in

reduced roducti ity while they et used to the new tools and
rocesses.

Lar e nancial or anization in Europe

Another e am le o trans ormation is a com any in Euro e that
has a new de elo ment ro ect on /OS. For this ro ect, it wanted
to start out ollowin De O s ractices and wor to build a ull
end to end modern i eline in su ort. In this case, the com-
any has decided to ee the i elines se arate or distributed
and and are ocused on ro idin the ull stac or /OS de el-
o ment, includin the mana ement o the middleware as art o
the solution.

The or ani ation selected Rational Team Concert RTC and

UrbanCode De loy UCD as the base or its i eline. RTC ro ides
the ull SCM build inte rated with UCD or the de loyment ste s.

CHAPTER 7 Seeing DevOps Success Stories in the Enterprise 63

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 As a ey art o this acti ity, not only is the source code mana ed
but also the confi uration or the middleware, such as any MQ
u dates and Data ower confi urations. UrbanCode De loy ro-
ided the ri ht lu ins in su ort o each o the systems re uired.

As art o this new a lication de elo ment, this or ani ation

is creatin ser ice inter ace definitions or the ca ability so that
it can be e osed to the ront end de elo ment acti ities. These
ser ice definitions are defined in CICS with the use o ID . The
rocess uses RTC to store these new ty e arti acts in com onents
alon with the COBOL a lication, uses RTC to build the arts,
and then uses UCD to de loy them to ether.

This com any lans to continue to e and the De O s ractices

to database u dates, scheduler u dates, and any other system
u dates re uired as art o an a lication de loyment. It also has
a ull ractice around test automation so it can recei e eedbac
uic ly on de elo ment acti ities.

CICS development organization

The IBM de elo ment team res onsible or buildin the CICS
Customer In ormation Control System roduct had been doin
de elo ment in the same way since the be innin o CICS. The
CICS team wanted to be more iterati e because it was usin the
same internal tools that all /OS teams had been usin or at least
years the toolin is robably at least years old . It was an
old VM based de elo ment en ironment. I you had rown u on
the toolin , you lo ed it, but it was not hel in attract new talent.
It also couldn t be used or the new a a de elo ment the CICS
team was wor in on, so a di erent set o tools was used or the
a a de elo ment teams.

In order to acilitate its trans ormation, the CICS team started

with RTC s wor items and lannin ca ability. This ro ided the
collaborati e de elo ment en ironment that all teams could use
and a sin le SCM they could all mo e to. O er time the teams
mo ed their ull source out o the internal VM based library
system and ully into RTC. CICS also de elo ed new automated
testin ca ability that allowed indi idual testers to easily create
new test cases to test the new unction bein de elo ed.

This combined rocess chan es and modern toolin a roach has

mo ed the CICS team to a more De O s culture. The team reali ed
business alue in the orm o

64 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 » Increased communication with a standard toolset for
consistency and a platform for dialogue
» Increased speed with automation and cutting release cycles
by 50 percent
» Increased responsiveness by incorporating feedback into
daily development and builds
» Increased innovation through reduced waste and greater
collaboration
This area can easily be seen through the updates the CICS
team releases, including application bundles and the z/OS
cloud work. The CICS team has, through its transformation,
become an example within IBM as to what can be done
through modern practices.

Automated Testing and Deployment

A second ocus area or startin a De O s trans ormation is that
o automated testin and de loyment. Many or ani ations find
automated de loyment to be one o the ma or roblems causin
outa es and delays in ettin inno ations to mar et.

Another lar e financial institution ro ided this ne t e am le.

This or ani ation has many se arate units that ro ide di erent
business ocus areas. The rou is s read throu hout the coun-
try and has ma or locations around the world as well. A senior
de elo ment mana er ro ided the ollowin descri tion o its
trans ormation

Our firm is e eriencin a ma or chan e in how we thin about

our main rame systems. Historically, these systems ha e had
ri orous, recise but un ortunately slow u rade cycles. New
a lications were installed at s ecific times and ro ects were
ma ed out in a detailed water all ashion. A ile methodolo ies
ha e started to chan e how we thin about our systems. The firm
has started to thin about a minimum iable roduct and how
test automation can be le era ed to roduce a lications aster,
test earlier and more re uently, and use automation to ut those
a lications into roduction.

CHAPTER 7 Seeing DevOps Success Stories in the Enterprise 65

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 The firm is le era in a lar e array o new technolo y in this
chan e, and two technolo ies stand out as si nificant

» IBM Development and Test for z Systems (formerly

Rational Development and Test for z/OS) D&T allows the
firm to create virtual z/OS instances on distributed hardware
The zD T and systems serve a dual purpose The first is
they’re easy to create just like any cloud system, so there’s
no harm in throwing it away after running a battery of tests
and starting with a fresh copy for the next test run. The
second purpose is as a learning system. What better way to
learn how a complex system like a mainframe works than by
actually using one and what better way to find out what that
button does” than by pressing it and not fearing any negative
consequences because the system itself was virtual to begin
with?
The ability to create virtual z/OS images has been combined
with a cloud deployment model to give fully automated API
access to create the systems The combination gives the firm
a direct path from code checked into source code control,
through build, unit testing, and functional testing on a z/OS
system in a fully automated fashion.
» UrbanCode Deploy: UrbanCode Deploy allows the firm to
create environments needed to test applications in a fully
automated fashion. Taking the manual approach out of the
environment creation allows for faster testing, but it also
allows a verifiable way to create the exact same environment
multiple times; that way the only thing that is changing is the
application code.
The UrbanCode Deploy environment creation and automa-
tion can be used on the virtual RD&T images and also on real
mainframe systems, which gives consistency between test
environments, QA, and production.

Application Understanding
As technolo y disru ts the insurance sector, ro iders are find-
in that traditional business models may no lon er be enou h to
han on to mar et share. Customer e ectations are on the rise
and ersonali ed roducts and ser ices can be an all im ortant

66 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 source o com etiti e ad anta e. Howe er, many leadin insurers
find their ability to inno ate is slowed down by com le , in e -
ible systems o record.

The Senior Director o Quality Assurance or a leadin UK insur-

ance com any elaborates We now how critical it is to e ol e the
a lication ort olio on our IBM Systems lat orm to su ort
new roducts and ser ices. Howe er, rowth throu h mer -
ers and ac uisitions had le t us with an e tremely com licated
a lication landsca e, with some arts mana ed by outsourcin
ro iders. We reali ed that o er time, the uality o our a lica-
tion code had de raded, and our lac o control made it difficult
to chan e.

Seizing control of development

The insurance com any trans ormed its a roach to de elo ment
by definin and rollin out a set o best ractices or codin , su -
orted by the IBM A lication Disco ery and Deli ery Intelli ence
solution. Initially, the firm ocused on its COBOL based olicy
a lications, and used these as a baseline or the rest o its a li-
cation ort olio.

The IBM solution a e us the insi ht into our a lication land-

sca e that we needed to de elo codin best ractices that ta e
uality and com le ity metrics into account, adds the com a-
ny s s o es erson.

The firm s de elo ers can now trac com liance with the new
standards early in the testin cycle. The s o es erson also added
that by monitorin how well our new code com lies with our
tar ets, we can reduce the number o de ects once new a lica-
tions ha e been released into roduction. Any de ects we do find
can be ed into a rocess lat orm or remediation, hel in us fi
them aster.

Unleashing progress
By increasin control and s eed o de elo ment, the insurance
com any has ained the tools to deli ht customers, oster inno-
ation, and out maneu er com etitors to win new mar et share.

IBM A lication Disco ery and Deli ery Intelli ence hel s us
dri e u the s eed and uality o our de elo ment cycles, com-
ments a s o es erson. Earlier and more accurate testin i es

CHAPTER 7 Seeing DevOps Success Stories in the Enterprise 67

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 us the reedom to inno ate without wastin resources. As a result,
we can et new roducts and ser ices out in the mar et lace ahead
o the com etition, i in us an ed e in a crowded industry.

The insurance com any has also succeeded in dri in down de el-
o ment costs Since de loyin the IBM solution, our de elo ers
can wor much more efficiently and need less hel rom third
arty ro iders, allowin us to ma e better use o resources.
Armed with IBM tools, we can loo to uture challen es with more
confidence, sa e in the nowled e that we can ada t aster than
e er be ore.

Full Practice Adoption

In this section, I ha e included Nationwide Insurance Co., whose
ocus was on creatin the ull De O s culture. It started with the
oal o transitionin a ull ro ect with a cross unctional team
and then e andin team by team.

Nationwide is a To Fortune Com any, head uartered in

Columbus, Ohio. It em loys more than , associates and is
a leader in se eral lines o business Auto, Li e, Home, Financial,
Ban in , Retirement Plans, Pet Insurance.

Nationwide has o erationali ed a ile ractices across IT, lines o

business, and the entire deli ery li e cycle, as well as across tech-
nolo y domains that include the e istin main rame, distributed
systems, and business intelli ence data solutions. Usin a De O s
a roach, Nationwide can now er orm continuous inte ration o
its code and continuous de loyment into its de elo ment en i-
ronment se eral times a day. Teams can also er orm acce tance
testin o customer re uirements in the same iteration with
de elo ment. They can show the customer, in near real time,
what de elo ers are roducin . This almost immediate eedbac
hel s ensure that what s bein roduced is oin to meet the cus-
tomer s needs.

To su ort collaboration, Nationwide or anically orms cross

unctional teams that sit to ether hysically at ods, com risin
members rom the business and multi le IT disci lines, includ-
in de elo ment and in rastructure. Collaboration has become
an e ected art o the culture and is built into the office s ace

68 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 and ractices. The com any has embraced chan e and nows that
it won t now all the answers at the be innin o the so tware
deli ery li e cycle.

The De O s a roach has trans ormed so tware de elo ment

at Nationwide. Some o the s ecific outcomes seen are hi her
roducti ity and hi her uality. It s not uncommon to de elo
code that, when deli ered to system test, has ero de ects. Fi ty
ei ht ercent o the Nationwide teams are in the to uartile o
the industry based on lines o code roduced. The com any has
im ro ed so tware uality by ercent o er the last three years,
and it has reduced user downtime by ercent.

Outside or ani ations ha e reco ni ed Nationwide s achie e-

ments. Nationwide is one o the first a ile at scale or ani ations
to achie e a Ca ability Maturity Model Inte ration CMMI Le el
ratin . Further, the com any s Des Moines, Iowa, de elo -
ment center won a Iowa technolo y award or best use o
technolo y.

Althou h the com any continues to mature its De O s ractices,

it s reali in business alue rom a rowin ado tion model.

Nationwide is buildin o these ains by e tendin a ile and lean

ractices across the entire deli ery li e cycle. Its current ocus is
on ro idin the ca ability or inte rated continuous ow and
isibility. By creatin this deli ery hi hway, Nationwide ro ides
business areas with the ability to deli er as ast as they need to,
based on their own determination o cost, ris , and alue. Time
consumin manual rocesses based on out o date in ormation
are re laced with automated de loyment olicies that use real
time in ormation synced rom systems o record. The combination
o dashboard and dials i es the business the isibility and control
to dri e at a much aster ace than was ormerly ossible. Lac o
trust in de elo ment outcomes has historically been an im edi-
ment to s eed. The new automated a roach ro ides confidence
to mo e aster to ser e the business without com romisin the
uality o deli eries.

CHAPTER 7 Seeing DevOps Success Stories in the Enterprise 69

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 IN THIS CHAPTER
» Getting started with DevOps

» Guiding you through the DevOps

step-by-step process

Chapter 8
Making a DevOps
Transition

I
n this chapter, I describe the steps to get started with DevOps.
This chapter uses examples and experiences based on a number
of customer engagements.

Getting Started with DevOps

DevOps is hard. Enterprise DevOps is harder yet. It not only rep-
resents a complete transformation of your software development
life cycle (SDLC), but also requires a change in both mindset and
organizational practices. Furthermore, it’s not just about tooling
and technology or infusing speed, but represents a more holistic
way of managing software development. This transformation is
no easy tas , and there are many di erent arts o the or ani a-
tion that need to artici ate. There are also many di erent silos
that need to be bro en down between the di erent teams.

DevOps is a journey. It’s not something any large existing orga-

nization can take on all at once. The important point is to get
started. Start somewhere with some part of the organization.
Some organizations may already have parts of the culture with
di erent terms. It s im ortant to reco ni e what your current
culture is to start with.

CHAPTER 8 Making a DevOps Transition 71

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Today, however, very few organizations haven’t started a DevOps
journey in some way. Some don’t use the term DevOps, but they’re
moving to a culture of small batches, automation, and continuous
improvement through removing waste. Most of the organizations,
however, have started with the distributed or mobile applications
and have left the mainframe side out. Now they’re looking at the
difficulties that ha e increased due to the trans ormation o the
front end, while leaving the systems of record process unchanged.
Enterprises are confused on how to improve without breaking
their systems of record.

As a result, many IBM clients have asked for help with “how to
get started” and/or creating an enterprise roadmap. Many times
this starts with a De O s wor sho , which can be the first time
parts of the organization have truly talked together. These work-
shops need to be cross-functional and cross business areas so
a clear icture o the current roblems can be identified. Really
this workshop helps point out the problems with the current pro-
cesses and the di erences between teams in those rocesses. So
this section gives you the steps you need to make that transition
into DevOps.

Don’t do these steps in a silo or department-only environment.

Reach out to sta eholders in e ery area that the a lication
touches. Invite them to the analysis, gain their input, get their
opinion, and eventually their buy-in. Doing it just for your depart-
ment is a recipe for disaster. Also, this exercise is meant to be
iterative. Play around with all the steps. Tinker and move things
around, chan e sco e, u date solutions, redefine ision, and so
on until you come up with an objective that’s both transformative
and achie able. Remember, De O s is a ourney, which can be two
steps forward and one step back.

Guiding You through a DevOps

Transformation Process
The ste s in this section are iterati e, as the last ste says Re eat.
These are a set of steps that have been developed with many
di erent client en a ements.

72 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Step 1: Pick the right set of applications
Not all applications are created equal. Some adjust themselves
more easily to DevOps than others. Some require DevOps more
than others. To be successful in DevOps, especially in the enter-
prise space, picking the right set of applications for the initial
trials is key. IBM has a Digital Transformation Model for Applica-
tions to help you categorize enterprise applications and chart a
roadma or your De O s ourney. Remember, this is an or ani-
zational transformation; all applications will need to change. The
first selected a lications are im ortant because you use these to
learn how best to transition and provide an important example for
a successful transformation.

Assuming you’ve already begun the transformation in the

distributed space, pick a back-end application associated with
a distributed application that’s the farthest along in the trans-
formation. The distributed team can help the mainframe team
understand the transformation and help with the processes and
procedures.

Another way to start is with a mainframe team that’s most inter-

ested in transforming. Never underestimate the value of existing
culture and the team dynamics. A good team that’s interested in
trans ormin will hel su ort the chan es and hel ma e your first
changes the most successful. Using application-understanding
tools to understand the impact between the applications can also
hel which to choose first. It may be easier to start with an a li-
cation that’s less interconnected with other systems.

Step 2: Develop the vision

Many organizations focus solely on speed of delivery. While
it is a ood start, ran ly it s incom lete. What use is code ut
into production quickly if it’s full of defects and takes an army
to delivery it? Along with speed, think about your quality and
productivity as well.

An example of a more holistic vision would be, “move from a

quarterly release cycle to monthly, with only less than 5 percent
defects slipping into production, and taking less than 50 percent
of the current resources to deploy.” In many ways, the three
dimensions provide checks and balances against each other. Any
vision without the other two will eventually fall apart in time.

CHAPTER 8 Making a DevOps Transition 73

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Step 3: Determine what’s
holding you back
I you now where you want to o, the ne t ste is to fi ure out
what’s holding you back. Analyze the overall cycle time (lead +
processing time), quality (number of defects at various stages),
and productivity (number of resources involved) metrics for
the a lications in uestion. A holistic study, across all three
dimensions, gives you insights into not only where your true
bottlenecks are, but also their severity and their impact on the
software delivery life cycle.

Ran the bottlenec s by time delays/im act/se erity to hel you

build the foundations of your DevOps roadmap. Perform this
exercise over a variety of applications to get an organization plus
an application view. Don’t skimp here. Yes, it will take time, but
the results uncovered will drive the rest of your DevOps strategy.

Step 4: Transition to DevOps

Resist the ur e to um to technolo y first control your inner
nerd; I know it’s hard). Instead focus on the three guiding prin-
ciples of a DevOps solution: automation, standardization, and
frequency. Determine where and how you would like to apply
these principles across your bottlenecks. Don’t be afraid of choos-
ing more than one. Case in point, if environment management is
an issue, don’t just stop at automation to help you with speed.
Also use standardized platforms and process to improve quality
and increase frequency of provisioning/de-provisioning plat-
forms to impact productivity and costs.

Go deeper and pick a set of solutions for each bottleneck. Again,

think in three dimensions: people, process, and technology. While
picking options, make sure all the solutions can talk to each other.
Thin eo le and rocess first and then technolo y thin ilot
not big bang, and for a fast start, consider partnering.

Step 5: Measure the ROI

It’s amazing how many organizations, after going through the
first our ste s, sim ly s i this ste . First, do a uic sanity
check. If you eliminated all the bottlenecks, can you still meet
your objectives? If not, go back to the drawing board. If yes, zero
down on what bottlenecks, areas, or solutions you must solve
first. Use the out ut rom here to rioriti e your roadma .

74 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Second, determine if this initiative is worth doing by building a
return on in estment ROI model. Loo or a minimum o a two
times the return o er a three to fi e year hori on. Stic with
s eed, uality, and roducti ity or benefits. For costs, consider
both real costs (software, services, consulting, and so on) and
hidden costs (people and process change, disruptions, and more).

Enterprise DevOps solutions save money in many dimensions:

» By increasing sta capacity through productivity

improvements
» By discovering defects earlier in the life cycle, thus reducing
the resolution costs and improving uality
» By automating deployment across the parts of the applica-
tion, thus removing deployment errors

In order to calculate the ROI or your De O s trans ormation, the

first as ect is to measure what you re doin today. Understand-
in your current acti ities hel s in calculatin the benefit to the
transformation.

Key areas to measure include the following:

» The time it takes to test

» Time spent waiting for resources or the next step
» Defects found by phase
» The time it takes to deliver an idea to the end-user, from
re uirement to delivery for small and large changes
» Time to complete deployment in each environment

All these metrics should be reasonable to calculate in an existing

environment.

The next step is to monetize the value of improvements in the

velocity of delivery and the increase in application quality. These
metrics should be continuously measured, then as the transfor-
mation takes place, these updated metrics can be compared to the
original metrics to show actual return values.

Many examples exist. For instance, in one client example, the

deployment times went down from 49 to 99 days to 3 to 5 days.
That ROI is easily seen.

CHAPTER 8 Making a DevOps Transition 75

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Step 6: Repeat
De O s is a trans ormation. Once you ha e trans ormed the first
few applications and gained knowledge and understanding of how
it works in your organization, it’s time to repeat the cycle through
all applications. It doesn’t end once all application teams are
transitioned. Now comes the continual improvement. Continue to
look for additional bottlenecks and areas for improvement, and
continue to improve. It’s important to keep the metrics in place to
see the continual improvement.

All applications need to be transformed. If you have some appli-

cations that you think aren’t worth the investment to transform
because of limited or infrequent updates, think about a few years
from now. All the rest of your development and operations teams
work in a DevOps manor, and you have these other applications
usin a di erent rocess and set o tools. You ha e to maintain
those other tools just for these infrequent updates, and you have
to have people remembering the other processes, and you open
yourselves up for additional errors. There are no applications that
should just be left behind.

76 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 IN THIS CHAPTER
» Bringing Cloud and DevOps together in
the enterprise

» Building the next generation enterprise

IT skills

» Loo in at t for purpose platforms

» Seein t e top myt busters for

mainframe DevOps

Chapter 9
Understanding Where
DevOps Can Take You

I
n this chapter, I discuss the additional value a DevOps culture
can ro ide an or ani ation. This cha ter ro ides s ecific
examples of how cloud relates to the mainframe in this new
DevOps environment. In this chapter, I also discuss a number of
myths that have developed around DevOps for the enterprise and
how you should determine the best fit lat orm or your uni ue
development needs.

How Cloud and DevOps Come

Together in the Enterprise
Now that you’ve started your DevOps transformation, how do you
deal with the increasing demands for capabilities as well as the
disruptive technology advancements? This section goes into more
detail of mobile and cloud and how the enterprise can deal with
the technology challenges to come out ahead.

CHAPTER 9 Understanding Where DevOps Can Take You 77

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 The problem
Most enterprises are facing competition today that didn’t exist
fi e or ten years a o. Bac then their com etition was other enter-
prises. Today an enterprise also competes with small start-ups
that provide only a small piece of what the enterprise provides.
These start-ups begin with a DevOps culture and don’t have the
existing organizational structures or systems to slow them down.
While none of these start-ups really competes with the enterprise
in its entirety, to ether they re resent an attac that a ects the
enterprise’s viability.

What ma es these start u s so attracti e to users is that they use

new technologies and methodologies to connect to the user per-
sonally, and they ha e the a ility to chan e their roducts uic ly
to meet the user’s changing demands. The start-up can have a
new app out and on a user’s phone in the time that the enterprise
ta es to determine which systems ha e to chan e to su ort the
demand. The complete DevOps culture is what allows the small
start u s to uic ly res ond.

What has given these start-ups so much power? Mobile devices.

These tools bring new power to the user in a new unprecedented
way. In the past, the collection point for creating information was
in the data center. Whether you had distributed servers or main-
frame systems, the focus was having server infrastructures in the
data center that turned data into information, and you had clients
or terminals outside the data center that would communicate to
them. Even when the web began to provide browser front ends to
in ormation, the real wor was always done in the data center.
Today, smartphones allow users to obtain information outside
the data center. They ta e in ormation rom multi le sources and
combine it with data that only exists on the phone to build new
in ormation tailored to this s ecific user. Start u s don t ha e
the myriad services that exist in the enterprise. They don’t have
the business assets that the enterprise has constructed over the
last 50 years, but they do have the ability to morph and change to
meet client demand immediately, and they le era e that to ta e
business away from the enterprise.

The belie by some eo le in the cloud field is that cloud mar s the
end of the enterprise. They believe this because they feel that there’s
no way for the enterprise to move with the speed and agility that
these start-ups move. They feel that the services that the enterprise

78 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 provides are the very inhibitors to success because these services are
monolithic a lications that inherently can t mo e uic ly enou h
to su ort this new model. But what i there was a way to retain
those business assets and be agile too?

If an enterprise could embrace a model that allowed those busi-

ness assets to be used in an agile way that would be a disruptive
technolo y, it would brea the con entional cloud belie that the
future belongs to a bunch of small components playing together.
This model is, at its core, a DevOps culture.

APIs A paradi m s ift

As cloud starts to focus on providing application programming
interfaces (APIs) via services instead of applications, it provides
new opportunities for the enterprise. If an enterprise can express
the business assets that are currently combined in an application
as APIs, it can combine those APIs in a more dynamic way down-
stream. This brea s u the alue chain that s currently loc ed
to ether in an a lication and ma es each lin a ailable to be
used in new ways.

For example, imagine you have a store with a loyalty program.

You have an application in the data center that alerts the cashier
that the customer whose card has just been scanned at purchase
time will get a 20 percent discount on any item in the store on
his next purchase. This application not only informs the cashier
to mention it to a client, but also it prints it out on the receipt
and maybe even sends an email reminder to the customer’s
registered email.

Compare that to a mobile app that the customer runs as he enters

the store or perhaps at home before he gets to the store. If the
mobile a loo ed at what the customer wanted to urchase, it
could ascertain that it would give loyalty point value immediately.
Because it isn t only tied to the urchase rocess, it could be used
to mar et other items to the customer that it nows he is inter-
ested in buying. It could actually show the customer how buying
that item at the same time would be chea er because it could ta e
advantage of the savings at the time of purchase. The idea is to
separate that loyalty program from the purchasing process and be
able to tie it to other shopping activities. Now the store can use the
loyalty program as a way to sell more items immediately instead
o ust tryin to et the customer to come bac to the store later.

CHAPTER 9 Understanding Where DevOps Can Take You 79

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Of course it could still be used at purchase time the way it had
always been used, but now the functionality can be reused in new
and di erent ways.

This is at the heart o what needs to be built. Smart hones and

other mobile devices connect to users in a much more intimate
way than even laptops. Have you ever decided against reaching
for your laptop in favor of doing something on your smartphone
or tablet? Over 2.6 billion smartphones are in use worldwide with
the preponderance of users below the age of 55. In fact, as the
demographic gets closer to 25, the more highly dependent on
smartphones it becomes. These systems of engagement are table
sta es or businesses today. Li e websites in the s, busi-
nesses need to be accessible via smartphone technology. The user
community of these systems is sensitive to the usefulness and
ower o a lications and is eenly aware o how uic ly it ta es
advantage of new technologies.

With this in mind, the focus of systems of engagement is to provide

the ability or the creation o new a s that can ta e ad anta e
of new trends and technologies. While the apps on the phones are
o ten written in lan ua es that fit their o eratin systems Ob ec-
ti e C or Swi t on iPhone and a a or Android , they o ten rely on
another tier to ro ide s ecific content. These layers are written
in lan ua es li e Ruby or a aScri t and de elo in with Swi t,
which allow developers to create and deploy new application
unctionality uic ly that connect to users and ro ide content
and capability that are constantly changing. They facilitate rapid
development and allow the programmer to control the ability to
provide new content in ways that can be augmented with tech-
nolo ical ca abilities a ailable rom the de ices. Businesses that
can ta e ad anta e o this ra id de elo ment ca ability become
radically more responsive to their user communities.

Of course these systems of engagement need some support. They

normally live much closer to the edge of the enterprise and provide
functionality that’s focused on the activities that are currently of
interest to the user. They don’t normally have access to much of
the rich business function that’s at the heart of most enterprises
today. To be truly successful in this space, an enterprise needs
to ta e ad anta e o the in ormation that is held in the systems
that have run the business for the last 50 years. These systems of
record have evolved over the course of a business’s life and have
focused on solving complex business problems with an eye on

80 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 sharing data securely in a high-performance platform with high
availability.

Because these systems o record ha e e ol ed o er the last

50 years, they often have components that still survive those
early years. In fact, mainframe systems are so sophisticated, they
allow programmers to use the latest technologies while still being
able to run code that was de elo ed in the s. These systems
have been traditionally more focused on providing support for
the business processes that have provided business success over
the years.

Systems o record mean more to the business than ust a set o

data that’s securely accessed over the life of the company. They’re
e en more than the transactional benefits that ha e been the
hallmar o their success. The benefit o current Systems is
that they can provide a way for systems of engagement to access
information in ways that have been unavailable in the past.

Notice that the discussion isn’t around data but information.

Information can be created without necessarily exposing any of
the secure data re uired to create the in ormation. For e am-
le, I could create an API that determines a atient s financial
res onsibility or a s ecific medical rocedure without e os-
in the underlyin health care data re uired to ma e that
decision. Therefore, if a system of record provided an API that
too as in ut a atient name and a articular medical rocedure,
it could res ond with that atient s financial res onsibility or
the rocedure ac is res onsible or ercent o the cost o this
ER isit without e osin to the caller any o the atient s health
care in ormation ac s account in ormation, re ious medical
history, other medical treatments, and so on).

Enter rises are ocusin on how to ta e all the business assets

that currently exist in their systems of record and free them to
ma e them a ailable on systems o en a ement. As they mo e
from the monolithic applications that have grown over the last
50 years into business and technology APIs that can be called, the
traditional business models of the enterprise get a new life. After
the enter rise brea s u the alue chain o business ca abilities
on the systems o record, the lin s o that chain can be combined
in new ways as di erent lines o business ic and choose the ele-
ments that enable them to reach new clients in new ways using
system of engagement and their user centric capabilities.

CHAPTER 9 Understanding Where DevOps Can Take You

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 The value of systems of engagement leveraging systems of record
doesn’t end with the business being able to relate to user commu-
nities better. It can actually sell this functionality as hybrid cloud
business APIs to other businesses. Suddenly those niche com-
petitors become value added resellers of the enterprise capability.
The enterprise gains a new revenue stream, as IT becomes a new
way to earn revenue. Traditional niche providers are able to grow
by providing a more complete portfolio of business capability
tailored to the community it ser es. Users also benefit as ser ices
become more ersonali ed and ocused on their s ecific wants
and needs.

This lin between systems o en a ement and systems o record

re resents a new disru ti e technolo y in cloud. In the first
wa e o cloud, the ocus was on how to ma e use o low cost
infrastructures to lower the barrier to entry of a business into a
mar et or to lower the o eratin cost o an e istin business or
enterprise. This second wave of cloud is much more focused on
providing enterprise business capability to any business. It brings
a new business model to existing enterprises, allowing them to
become managed service providers to other businesses. It allows
smaller businesses to ta e ad anta e o enter rise ca ability at
a low cost and provide new capabilities to their users. It allows
end-users to have unprecedented access to myriad information
that was previously unavailable. In the new hybrid cloud world,
the mobile device becomes that personal shopper, assistant, and
route to business and peers in an increasingly interconnected
world.

The De O s culture allows or ani ations to ma e these chan es

and e ol e into ro idin the business ser ices as re uired.

Building the Next Generation

Enterprise IT Skills
I I loo bac to when I started in the industry, it was lon
before the current proliferation of IT technology. There were no
cellphones, computers weren’t common in the home, and the
personal computer hadn’t even been released. In those days when
new talent was hired to wor in IT, they were trained on the com-
anies lan ua es and ractices. When I started in IBM, no one

82 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 assumed I new all the ins and outs o PL/X or assembler.
I learned them when I started. Over the years, the assumption
that talent could be hired with those e istin s ills has mor hed
the development landscape.

In order to deal with the desire to hire eo le trained with s ills

in a wide range of areas, the Academic Initiative was started years
a o. With the Academic Initiati e, IBM and other com anies
have partnered with universities and colleges to provide an
enhanced curriculum that includes topics appropriate for future
mainframe developers.

In addition, there are the Master the Mainframe contest and

additional amin contests ocusin on either buildin on /OS
or buildin ront end inter aces to /OS bac end a lications.
These challenges are driving interest among younger and younger
students and helping to develop the next generation of mainframe
developers.

Another ey actor in the ne t eneration enter rise IT s ills is

to remo e the se aration o tools based on lat orm. Sometimes
an o timi ed tool or a lat orm ma es sense, such as the IDE or
editor that understands the language or platform characteristics.
But beyond that, usin common modern toolin or the SCM i es
new de elo ers rom anywhere the ability to find and learn about
your code aster ha in a modern CI/CD i eline ro ides clear
isibility and understandin to the new de elo ers and, finally,
using common testing tools and practices allows individuals to
come up to speed much faster.

Fit for Purpose Platforms

One additional ey ad anta e o De O s with its cultural trans-
ormation and the brea in down o silos is that lat orm
selection is more easily based on the characteristics of that plat-
form instead of which development team can produce the solution
the fastest.

Fit for purpose is the process of selecting the platform based on

the actual business re uirements or an a lication or ser ice. For
some services, there aren’t characteristics that strictly determine
the best platform; for others there are. For example, functions

CHAPTER 9 Understanding Where DevOps Can Take You 83

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 that re uire hi h a ailability and ha e hi h transaction throu h-
ut are enerally seen as more a ro riate or /OS. Additional
attributes such as closeness to data and high volume are addi-
tional characteristics that mi ht lead to /OS.

In a DevOps culture, with everyone focused on delivery, the

platform selection can be made based on the actual business
re uirements, not on team ercei ed ability to deli er uic ly.

Myt Busters for Mainframe DevOps

Many common misunderstandings exist about what DevOps is
and what it im lies. In this section, I debun a ew o these myths
for you.

DevOps is for born on t e eb

companies
De O s as a culture is a licable to all or ani ations. Brea in
down the silos helps large-scale multiplatform organizations as
much as it does the born-on-the-web companies. The practices
of automation are well proven, and those large-scale enterprises
that have been transforming have shown the business value
received through the transformation.

DevOps implies continuous

deployment into production
DevOps culture facilitates the deployment of capability into the
end users or alidation, which doesn t im ly or re uire roduc-
tion. Deployment into production is to deliver business value;
deli erin or the sa e o deli ery ser es as little ur ose as hold-
in releases or monthly de loyments ust or the sa e o holdin
them. The pipeline should be created such that deployments hap-
pen to provide the business value.

One great example to use to understand the concept of delivery

being a measure of success is that every organization has a calen-
darin ser ice or API. This ser ice ro ides critical business alue
to all business applications. How often does the calendaring ser-
vice need to change to provide business value? Keep in mind that
the Gregorian calendar is still in use.

84 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 If you don’t recognize the reference to the Gregorian calendar,
it s really ust the calendar we use today with months, and
it s named a ter Po e Gre ory XIII, who introduced the calen-
dar in October . My oint is that the calendar hasn t chan ed
in hundreds of years, so there’s not a lot of need to change this
service.

This service needs to be updated for federal holiday changes,

for time zone changes, and to provide new interfaces, such as
REST. But other than that, there s no business alue to u datin
the unction, so re uent deli ery isn t a ood measure. Yes, this
is an extreme example, but it’s important to recognize that deliv-
ery or the sa e o deli ery isn t im ortant deli ery or business
value is what needs to be measured.

DevOps and ITIL don’t go together

DevOps complements Information Technology Infrastruc-
ture Library ITIL , a set o documented best ractices or IT
service management. DevOps practices of automation and vis-
ibility hel su ort the ITIL ractices, s ecifically the chan e
mana ement ractice by decreasin the li elihood o roblems
with deployment.

DevOps and separation of

duties don’t mix
One e am le o se aration o duties is the re uirement to ha e
someone who didn t wor on a chan e to a ro e that chan e to
be deployed. This principle can and should continue for produc-
tion deployments. The approval process is tied into the overall
DevOps pipeline. Approval could be based on passing some series
of automated tests or based on additional reviews.

DevOps is only for small companies

Lar e or ani ations benefit as much as small or ani ations
from a culture of collaboration, automation, and transparency.
In fact, large-scale development organizations with many silos
benefit most rom the trans ormation. This trans ormation can
bring new focus and life into more stagnant development orga-
nizations, returning the innovation and creativity that have been
suppressed due to processes.

CHAPTER 9 Understanding Where DevOps Can Take You 85

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 DevOps is only about automated
deployment
It’s easy to fall into this trap. After all, most of the DevOps prod-
ucts and ser ices currently a ailable in the mar et lace ocus on
de loyment usin so tware to automate the de loyment cycle.

Howe er, i you loo at the data, there are fi e ey im ediments

to reali in the ull benefits o De O s. They are in order o
importance) as follows:

» Environment availability
» Manual deployments
» Test and use case setup
» Governance
» Development practices

En ironment a ailability and confi uration cause more delays

waste or you lean rocess bu s than any o the other our.
It is no wonder that organizations that just focus on the auto-
mated deployment aspect then run into serious obstacles during
implementation.

So instead o ust sol in or de loyment, sol e or the first two

im ediments first. The two are closely lin ed to ether and doin
so will release the bottlenec at the end o the so tware deli ery
li e cycle, brin in you some benefits early and in turn uttin
pressure to improve on activities earlier in the life cycle.

DevOps is all about improving

cycle time
It is true, De O s is all about s eed. Leaders such as Faceboo ,
Net i , Etsy, and Ama on release new code into roduction multi-
le times a day. But what ood is s eed i it ta es an army to deli er
bad code? Inherent in DevOps value proposition isn’t just speed,
but also uality and roducti ity. In act, the three are closely
lin ed, and without any one o them, the other two all a art.

Start with roducti ity. Most IT leaders et this one a ter all, i
you’re using automation, you’re using fewer resources and that
rees u time to ocus on more alue added acti ities. But uality,

86 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 on the other hand, isn’t initially apparent. If you dig deeper, you’ll
find the our dimensions to uality im ro ement

» Making small code changes to production dramatically

decreases the risk of things breaking. Inserting one new line
of code is easier to test rigorously than 100 or 1,000 lines.
» Standardized, repeatable, and reliable processes executed
by software bring in a level of rigor and quality improve-
ments that’s usually unmatched by manual processing.
» Many defects arise due to a mismatch between the dev/test/
prod environments (how often have you heard “It worked on
my box”?). DevOps does away with these platform mismatch
bugs by automating the configuration so every environment
looks the same.
» Due to automation you can ping pong code between dev
and test teams frequently to the point where most bugs are
fixed, which in turn greatly increases the confidence that the
code will work in production.

So, while you may be chasin the se iness o s eed, ma e sure you
ee an eye on the nuts and bolts. I you do it ri ht, you should be
able to enjoy all three for the price of one.

DevOps is a tool
This last myth is the worst of all the myths. Many organizations
consider De O s to sim ly be a toolin e ercise where et-
tin the ri ht tool is the ey to the De O s u le and that once
it’s implemented correctly, the organization will be well on its
way for faster turnaround times. Unfortunately, the reality is a bit
more complicated.

Here are a few things you may not realize about DevOps:

» DevOps requires not just one tool, but an entire chain of

tools. Stitching tools from multiple vendors (open source and
paid) is not only painful, but also requires skills and a mindset
that’s usually foreign to the enterprise.

CHAPTER 9 Understanding Where DevOps Can Take You 87

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 » For the tools to work e ectively, the back-end infrastructure
has to change as well. In many cases, this dramatically
undoes years of organizational process, protocols, and
control.
» Arguably the hardest realization is changing the people and
processes to enable a seamless deployment process across
multiple silos that traditionally have had diametrically
opposite objectives.

It has been my experience that organizations that follow a

tool-led strategy ultimately end up either buying more tools,
bein stuc or worse rustrated, and throwin in the towel.
Always ee in mind De O s is a cultural trans ormation tools
are to support this.

88 DevOps from APIs to z Systems For Dummies, IBM Limited Edition

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Notes

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
 Notes

These materials are © 2017 John Wiley & Sons, Inc. Any dissemination, distribution, or unauthorized use is strictly prohibited.
WILEY END USER LICENSE AGREEMENT
Go to www.wiley.com/go/eula to access Wiley’s ebook EULA.