The Association of System

Performance Professionals

The Computer Measurement Group, commonly called CMG, is a not for profit, worldwide organization of data processing professionals committed to the
measurement and management of computer systems. CMG members are primarily concerned with performance evaluation of existing systems to maximize
performance (eg. response time, throughput, etc.) and with capacity management where planned enhancements to existing systems or the design of new
systems are evaluated to find the necessary resources required to provide adequate performance at a reasonable cost.

This paper was originally published in the Proceedings of the Computer Measurement Group’s 1981 International Conference.

For more information on CMG please visit http://www.cmg.org

Copyright Notice and License

Copyright 1981 by The Computer Measurement Group, Inc. All Rights Reserved. Published by The Computer Measurement Group, Inc. (CMG), a non-profit
Illinois membership corporation. Permission to reprint in whole or in any part may be granted for educational and scientific purposes upon written application to
the Editor, CMG Headquarters, 151 Fries Mill Road, Suite 104, Turnersville , NJ 08012.

BY DOWNLOADING THIS PUBLICATION, YOU ACKNOWLEDGE THAT YOU HAVE READ, UNDERSTOOD AND AGREE TO BE BOUND BY THE
FOLLOWING TERMS AND CONDITIONS:

License: CMG hereby grants you a nonexclusive, nontransferable right to download this publication from the CMG Web site for personal use on a single
computer owned, leased or otherwise controlled by you. In the event that the computer becomes dysfunctional, such that you are unable to access the
publication, you may transfer the publication to another single computer, provided that it is removed from the computer from which it is transferred and its use
on the replacement computer otherwise complies with the terms of this Copyright Notice and License.

Concurrent use on two or more computers or on a network is not allowed.

Copyright: No part of this publication or electronic file may be reproduced or transmitted in any form to anyone else, including transmittal by e-mail, by file
transfer protocol (FTP), or by being made part of a network-accessible system, without the prior written permission of CMG. You may not merge, adapt,
translate, modify, rent, lease, sell, sublicense, assign or otherwise transfer the publication, or remove any proprietary notice or label appearing on the
publication.

Disclaimer; Limitation of Liability: The ideas and concepts set forth in this publication are solely those of the respective authors, and not of CMG, and CMG
does not endorse, approve, guarantee or otherwise certify any such ideas or concepts in any application or usage. CMG assumes no responsibility or liability
in connection with the use or misuse of the publication or electronic file. CMG makes no warranty or representation that the electronic file will be free from
errors, viruses, worms or other elements or codes that manifest contaminating or destructive properties, and it expressly disclaims liability arising from such
errors, elements or codes.

General: CMG reserves the right to terminate this Agreement immediately upon discovery of violation of any of its terms.
 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

WORKLOAD CHARACTERIZATION AND CAPACITY PLANNING AT A LARGE IBM INSTALLATION

- A CASE STUDY -

Robert M. Jackson
MITRE Corporation

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
ABSTRACT

The Spacecraft Software Divi~.ion at the Jolmson Space Center in Houston, Texas is procuring new IBM-compat-
ible hardware. This paper presents MITRE's part in that procurement in the areas of workload characteriza-
tion and capacily planning.

1. INTRODUCTION sis. the tools and techniques that were used and the
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

results achieved.
The Software Development Laboratory (SDL) is the computer
facility of the Spacecraft Software Division (SSD) of the 2. WORKLOAD CHARACTERIZATION
Johnson Space Center (JSC) in Houston, Texas. The SSD is
responsible for the programming and verification of the The analyRi~ waR accomplished in iterations. MITRE per-
Space Shuttle's onboard flight software (FSW). formed the data gathering and organizing while IBM, the
FSW contractor. performed the subsequent data extra-
The SDL computers are five aging IBM 360/755 (5 Mb each) polations. By the time the Request for Proposal (RFP)
which use the Enhanced Operating System (EOS) for Rimu- was released. five iterations of this analysis had been
lations and an EOS/ASP configuration for development. accomplished.
The EOS is a version of 05/360 with real-time programming
extensions added. The Time Sharing Option (TSO) facility The data gathering was achieved through the use of the
is used extensively. The remainder of the hardware com- IBM System Management Facility (SMF) on the 360/75s.
prises disks. tapes, printers and unit record equipment. The SMF tapes were used to load a System 2000 (S2K) data
Most of the FSW is written in HALlS. a high level aero- base. UsinR; S2K, a broad range of "commonality analy-
space avionics language. sis" activities were performed. Literally hundreds of
job steps were eventually grouped into 13 job-type cate-
In March. 1981 SSD released a proposal to acquire new gories based on such parameters as: CPU utilization,
IBM-compatible hardware. Approximately two years prior I/O requirements, CPU to I/O ratio, program size. code
to that, extensive analysis was begun to: characterize to data ratios. requirements for tape and lines of out-
the then current workload, project how that workload may put. These parameters either cause the direct consump-
change in a high flight density environment. and accu- tion of computer resources or cause host overhead. In
rately estimate the computing capacity required to sup- either event. they determine the required capacity of a
port it. This p~per presents MITRE's role in that analy- computing system to support the applications they repre-
sent. The baseline list of the 13 job-type categories
found in the SDL is shown in Table 1 [1].

The work described in this paper was perf'nrmp.d fnr NASA/,TSC under contract Number T-4830H.

Table 1. SDL Job Categories

CATEGORY JOB TYPE REPRESENTATIVE JOB STEP

1 FEID Development and Verirication REFCNTRL REFCDARY. FElD
2 FEID Preventive Maintenance FEID
3 lCS/"SIM REFCNTRL REFCDARY
I. FSW Pre-Processors SPPILM3 SPP5LM3, DLEXEC DFWASGN
SDPPR T DFGPGM FORMATDA
5 SDL Pri!'- Processors SMDLPIPP SMDLER2 SMDLSPRE SMDLETBI
6 Mass Memory Build SMDLMBLD SMDLRCBP SMDLMTCI SMDLMTC2
SMDLMTBO, •.••• ,SMDLMTB9 SMDLMMCP
7 Standalone JSTEP SMDLRjSC
8 Post Processor SMDLPOST 5MBJMDJS SMDLPINT SDLPLOT
9 DASS Monitor XPLSM
10 RAS RASLOAD ALTERF
11 Build Utilities PLMDISKS QMXVTOC JCLGEN SMDLUTIL
PLMBLDR GENVERS SMDLRGMT PLMBLDI N
PLMLSTPR PLMHLOAD OLMMLT PBP SDLSLDP
PLMEDIT PLMDSTOR PLMSTRIO PLMBLDRP
COMTAPEI COMTAPEH PREBUILD SMDLRPRE
12 PMF PLMSUP IEWL IEV90 SIMPLE LINKIOl
13 Other

248

Find a CMG regional meeting near you at www.cmg.org/regions

 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

For each job-type category, MITRE developed histograms At this pDint. two steps still remained before simula-
which portrayed the percentage of occurrences (derived tion could begin. The first of these was data genera-
from observed frequencies) in 30 calculated intervals tion. This activity populates the simulation time frame
that the critical resources would be consumed. In this with a statistically distributed workload. The second
way, within a job-type category, statistical distribu- step was load balancing. This activity attempts to
tions could be drawn regarding resource consumption for shape the workload so that it appears to the system

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
any number of occurrences. under test (SUT) to be an instantaneous "slice of life".

3. CAPACITY PLANNING 5.1 DATA GENERATION

The shuttle program, under current schedules, should For the T50 portion, this determined how many sessions
reach its peak load in 1984. Under those conditions, (logons to logoff) were required for each simulated ter-
flights may occur as often as every two weeks. The prob- minal and how many compilations. executions, etcetera
lem for capacity planning for 1984 becomes one of extra- were to be accomplished in each session. This data was
polating 1979-1980 data into expected workloads. This then translated (coded) into TPNS message decks (scripts)
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

extrapolation was performed by IBM. The key componenL of and PATH operands. the actual network configuration
the extrapolation exercise was an "activity chart" which (lines. terminals, etc.) was set in accordance with the
resembles Lhe L:lassical PERT charlo As t.he prime FSW lu~al facility which was used for benchmark testing. It
developer, IBM defined over 400 distinct activities that should be noted here that the actual RFP-bid network con-
wuuld oceUI' duz>.Lng FSW reconfiguratlon. For each activi- flguraLion was the responsibility of the individual
ty, the amount of resources, expressed as a percentage of offeror.
one or more of the 13 job-type categories, that would be
necessary to perform that activity were estimated. (Note For the batch portion of the benchmark, the data from the
that this is contrasted with how much each OI the cate- reshaped histograms was inpuL Lo a locally developed
gories participate in the current SDL work.) Finally, a routine along with a specification of how many job steps
summation and normalization within each job category (a function of benchmark duration) from each of the 13
yielded a percentage of participation in the total work- job-type categories was to be produced. This program
load that a given job-type category could be expected to calculated a cumulative distribution curve I01' each job-
produce. Therefore. the future (1984) workload could be type category. Each curve was then statistically sampled
expressed in terms of the derived current job-type and the corresponding resource usage characteristics for
categories. the given number of job-type steps was produced. This
process is depicted in Figure 2 (3]. The example shows
The final reshaping was performed on the 13 categories how a sampling of three job steps was selected from the
themselves. 'I'his dealt wit.h the resource consumption mid-points of three equally-spaced intervals on the
characteristics of each. This reshaping was achieved by x-axis. The y-axis contains the observed interval values
shlftlng/altering the expected observation percentages for a glven resource.
within one or more intervals and by applying emphasis
factors based on the derived usage predictions. Figure The output of this program was subjected to processing by
depicts a typical histogram's data following reshaping. another routine which ultimately produced Fortran DATA
statements. These source statements were then included
4. THE BENCHMARK PROGRAM in the final build of the benchmark program and hence
ttloaded" it with the necessary parameters. This approach
The successor facility to the SDL is the Software Produc- was chosen to reduce the extraneous overhead processing
tion Facility (SPF). The old 360/75 SDL hardware is to of the benchmark and thereby keep t.he resources consumed
be scrapped and the new hardware will be installed in the during benchmark execution as much in line with the
SPF. Presuming to run three shifts a day for five days a characterization profile as possible.
week. the SPF workload is characterized as: on-line
(i.e •• TSO), background batch requiring four hour turn- 5.2 LOAD BALANCING
around, and overnight batch. The first two are collec-
tively called the prime shift workload and the third is As mentioned, it is desirable to present as even a load
called the non-prime shift workload. as possible to the SUT when simulating only a small por-
tion of real time. Accordingly. attempts were made to
The prime shift workload benchmark is implemented via the keep each of the benchmark jobs (three job steps per job)
IBM program product Teleprocessing Network Simulator more or less equal in the amount of service units they
(TPNS). The TSO portion of the work is carried out consumed. This was largely successfully achieved by a
through TPNS message decks. The background batch work- third program which examined all available job steps for
load is represented by executions of ttsmaller cases" OT a given portion of the benchmark (e.g., non-prime, no-
the batch benchmark envoked through the TSO SUBMIT tape) and constructed jobs from them.
facility.
For the ISO portion of the benchmark, execution of system
The non-prime shift workload is achieved through execu- processors (e.g., the Fortran compiler) was evenly dis-
tion of the "larger cases" of the batch benclunark. The tributed across all simulated terminal sessions. The
benchmark effectively uses CPU and I/O resources, caUhes SUBMIT commands were spread one per terminal, each in the
paging, swapping and contention, etcetera. and in gener- first session. This gave the SUT the maximum amount of
al, accurately (and repeatably) represents the required time to "work off" this short ·turnaround batch workload.
workload.
6. THE BENCHMARK IN THE PROCUREMENT
5. RUNNING THE SIMULATION
Prospective offerors to the SPF procurement had great
It is unreasonable to expect an offeror in a procurement latitude in setting up and running the benchmark and in
action to run any benchmark over an unduly protracted tun1ng their system to meet the required performance
period. Still, benchmark executions must be long enough specifications. This made simplified benchmark operation
to minimize the end effects of benchmark startup and an absolute requirement. This was achieved. by and
tailoff. For the SPF procurement it was decided that large. through MITRE providing wi th the benchmark tape a
one hour of prime shift workload and two hours of non- rigorously developed set of operating instructi on::; wi th
prime shift workload would be sufficient. specific references and examples of what an offeror may
change and what he may not change. lie was given the ~
benefit of the experiences of and the problems which

249

Find a CMG regional meeting near you at www.cmg.org/regions

 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
,",O~ nc.
St~~ Ay, M_ M .... M_ c.'u DASD '1"",... pOAl POA2 0'''' 1"
'""1"":':":1
34 0 0 26 25 10 ~
7 20 0 0 18 19 0 DASD Split
J 17 0 0 10 12 12
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

• 15 0 0 5 4 6
• 13 0 0 4 I 6
48.6 51.4
• 1 0 n 2 2 20
7 0 6 18 2 4 8
• 0 I 0 3 1 14
•
'0
0
0
7
6
5
8
2
2
2
I
3
2 .089
0 6 0 I I I
"
17 0 8 38 0 0 0
.045

0 5 0 0 n 1 .116
0 3 0 I 0 I PDA2""
'5 0 26 0 0 I 0
"17 n , n n n ,
,. 0
0
4
8
8
0
2
2
I
0
3
0
0 I 0 2 0 1
" I 0 2 8
0 3
71 0 0 0 A 12 r.
70
0 0 23 2 2 I
0 6 0 4 2 1
0 2 0 0 0 0
:IS n , n 7 , n
J' 0 0 0 0 0 0 """II<,
0 2 0 0 0 0
0 1 0 0 1 0
0 0 0 I 0 0
0 0 0 0 0 0

2.6 604.8 814.1 336.3 9021.8 1843

m~t~.}=*~ttm?:;~~;@~ ::"i:=~~~~~~1:;~~;~ ~:;~·:;~t;~*~~·:. ---eo'

CICt ~ lImil tuk,"..
:~~:~~t~t~~~~H{~~::itt~:~~ :H:~~:~~~;~n8 'Jf:f:;:~:~~~g:;: I::nn A". LRPel.
1+120 +300 XO. 7~ Ix,,. 1..2 OPSF_

• AlIlnterv"h Assumed Us. Of

M~~n Rit"Sl! Valu. From $2000
Dal.abaJ. Which Must B~ Modified By The- Assod.ed OPS Factors

Compu«:d ET (Pr.Ops) 14.6

(Obs.rv~ Attach Tim!' ..:::o:zn:..B-)

Figure 1. Modified Histrogram Data

250

Find a CMG regional meeting near you at www.cmg.org/regions

 Learn the basics and latest aspects of IT Service Management at CMG's Annual Conference - www.cmg.org/conference

Buy the Latest Conference Proceedings and Find Latest Computer Performance Management 'How To' for All Platforms at www.cmg.org
In 856.23
~
...
<
>
...
<
>
'""'....
Join over 14,000 peers - subscribe to free CMG publication, MeasureIT(tm), at www.cmg.org/subscribe

;z:
~

...:;:
In
Cl
260.75
"'>
'"
OJ

'"
~.
0

56.43

o 16.67 50.00 83.'33 100%

CU1o!U[.ATIVE PERCENTAGES

Figure 2. Data Point Selection

were encountered by the benchmark development team. The The ullimaL~ ~ucce~~ 01 the benchmark won't be entirely
end result was that very few questions regarding the proven until the actual 1984 projected workloads are
benchmark were received and of those, none of substance I'eached. In the meantime, further ref'inements of those
regarding its technical aspects or the procedures for projections are foreseen. It is anticipated that the
executing it. benchmark will be reconfigured to track those refine-
ments and thus play an important role in a continuous
In a similar vein, .it was known beforehand that the mem- capacity planning exerci5e.
bers of the source board for this procurement who would
be !'t:lsponslble for evaluating the results of the bench- REFERENCES
mark executions by the various offerors were not thor-
oughly versed in its technical aspects, nor were they [1] Linde, Shirley, "Analysis of' Sof'tware Production
necessarily experts in IBM computer systems. Accord- Facility (SPF) Predicted Resource Requirements",
ingly. MITRE developed and supplied. as part of the MITRE, March, 1980.
benchmark, post-processor programs. These programs
effectively reduced the output of the benchmark execu- [2J smith, D. M.; Sabat, J.; Siller, J.; Fleming,
tions to simplified reports. MITRE conducted training J. K. j "OPS Architecture Phase Five Loading
sessions for the source board technical team members and Report", IBM, July 25, 1980.
prepared check lists for them to use in reading the post-
processor reports. The result of these efforts was that [3J Bays, W.; Jackson, R. M.; Spitzer, J. F.j Williams,
the evaluation of the offeror's benchmark executions went J. N.; "Preliminary Design Review - Software
exceedingly smoothly, and the technical team required Production Facility (SPF) Benchmark", MITRE,
very limited MITRE assistance, all of which was conducted February 9, 1980.
over the telephone.

Careful preparation on both fronts had resulted in a

benchmark package that could be readily executed by the
offeror and accurately and easily evaluated by the
government.

7. RESULTS

The announcement of the winning offeror has not been made

as of this writing. However, it is known that more than
one proposal was received and that every proposal speci-
fied hardware of the capacity estimated by the govern-
ment to be required. In short, the procurement bench-
mark qualified exactly the class of hardware anticipated.

251

Find a CMG regional meeting near you at www.cmg.org/regions