Metrics “Goals—The first step towards getting somewhere is to decide that you are not going to stay where you are.” @ John Pierpont Morgan Metrics—In-process metrics and end-result metrics —The Metrics Roadmap—A typical metrics strategy —What to measure—Targets-variability—Acting on data—People and organisational issues—Common pitfalls —Implementation-checklists. INTRODUCTION Poza In this chapter, we cover Metrics. We looked up the word “Metric” in the various dictionaries. Webster's defined the word as an adjective, “based on the meter as a standard of measurement; of, relating to or using the metric system”. Obviously this was not the definition we were looking for! A little closer was the word “metrical” which meant of or relating to measurement which captured the real meaning that is normally associated with the word Metrics from a software project management perspective. Perhaps the best motivation for Metrics comes from a quotation by Lord Kelvin (of the Kelvin Temperature Scale fame): “When you can measure what you are speaking about, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in your thought, advanced to the stage of science.” Metrics in a project management context is about measurements—Measuring yout progress in order to know where you are and what mid-course corrections you59 Metrics Having said that metrics is about measurements, several important questions arise 1, What should be the ideal roadmap for metrics? 2. What should be the strategy for implementing metrics? 3, What are the individual steps that constitute metrics? 4, What are the common management issues and pitfalls to watch out for in implementing metrics? These topics get addressed in the following sections of this chapter. While we do give some examples of metrics in this chapter, we have consciously avoided going jnto a lot of detail on what the specific metrics of each of the various project phases chould be. Every activity—be it an Umbrella Activity like software configuration management or an in-stream activity like project initiation—will have relevant metrics that need to be identified. This is the reason why metrics is covered as the first Umbrella Activity in this book. In this chapter, we present some of the basic inciples in metrics in any activity. In each of the remaining chapters, we will dis- cuss in detail the specific metrics that apply to that particular step/activity. THE METRICS ROADMAP The Metrics Activity should be viewed like a roadmap. Basically, every Y project has a certain set of objectives it has to meet. There are short term objectives, viz., the specific deliverables for the particular project and there are long term objectives, e.g., how is this project tied to our com- pany vision or how does this project further the learning in the organisation? Any metrics program or measurements should provide gauges that measure our progress towards both short term as well as ong term objectives. 4 Check 0 BB Fig. 5.1 The Deming PDCA cycle for metricse Managing Global sop, —_ Ttoare By, The strategy for a successful metrics program falls in line with Deminy’s py Quality Model as illustrated in Fig. 5.1. The acronymn P-D-C-A Stanidg hoe fy 4 Do- Check- Act. For any project or activity, you Plan what you want to ach bf, based on your current position and your eventual destination. While makin Ou plan, you also establish measurement criteria (which includes what you mersan how you measure, how often you measure and what are your acceptable / border, line and danger zone parameters). Then you Do ot carry out the Plan. This entails carrying out the necessary steps for the actual execution and also carrying out the measurements for health check. The Check phase compares the measurements made against the expected norms set in the Plan phase. If things are found to be out of line, Corrective actions need to be taken. The corrective actions (or fine-tuning oy mid-course correction, to use other terms) are taken in the Act phase and we start all over again by refining the Plan. OSeety Stated in a different way, we can say that there are five questions one has to answer for a metrics program that fall in four steps, as given in Fig. 5.2. N Where do we want to 90?" “How dowe measure progress?” “What do we have todo" ofE]Fig. 5:2. The questions to ask in a metrics program 1, What is your goal or where do you want to go? 2. What is your current position? 3, Knowing where you are and where you want to go, what steps should you take? 4. How do you measure your progress? 5. What corrective actions do you take when you see that your progress devial . from the expected course?Metrics ik 61 Let us examine each of the questions in a slightly more detailed form. In managing a project suiccessfully, the first step is to know what the goal or the destination is. The goal has several dimensions like timeline, resource constraints, profitability, quality requirements, ete. While formulating the goals, it is important also to strike a balance between the project (ot tactical or short term) goals and the organisational (or strategic or long term) goals. Sometimes these two may be con- flicting and there should be clarity of understanding why we are shooting for a goal. For example, as a rule, every project usually strives for a certain profit margin. But, there may be some key customers whom you want to win over and may be willing, to sacrifice the margins. While undertaking such goals, the organisation (and the project manager) should be very clear on the goal they are shooting for and why. It is also important to communicate the goals and the vision to the rest of the team so that there is a shared vision and purpose, Once you know where you want to go, you should take stock of where you are today. Knowing where you are today comprises of asking ourselves questions like “What are our strengths and weaknesses?”; “What has been our track record so far?”; “Do we have the skill sets required to achieve the goals?” and so on. This step is like taking a health check of where the organisation stands today with respect to achieving the goals set forth in the first step. Knowing where you are today and where you want to go (the eventual goal) enables the construction of an action plan of “how to get there from here”. There are two important constituents to an action plan. The first comprises the actual action items or steps needed. For example, you may discover in your analysis of where you are and where you want to go, that you currently do not have competency ina certain technology area and that you need to acquire that competency. The action plan may be a training plan of how/ where you would get trained in that technology or where you would hire people from with that competency. The second constitu- ent of the action plan is comprised of the steps to measure whether or not we are straying from our path. In other words, while carrying out the actions, how do you ensure that we are making progress (and making sufficient progress)? In the above example of training, an essential step of the action plan would be ‘how do we meas- ure the effectiveness of the training received?” From the above steps, you have got a feel of the pulse of your progress if you have got your measurements right (and are measuring the right things!), you would have got data on whether or not you are on the right track; if not, you would have an idea as to what is needed to effect a mid-course correction. This constitutes the final step of the metrics program—acting on data gathered and minimising the deviation from desired results, 'A Real-Life Example of ‘Metrics Road To illustrate this with a non-software example, let uy visit the Fisherman’s Wharf in San Francisco, Hoy measurement program? Obviously, the first thiny S imaging Ww would vad Our 1B YOU shoud gp, start you are now! If currently you are in Bangalore, India, thy iste You urement ctiteria you would need would be very dierent re king nia * starting from Stanford, California. tn the fitst case, your pl torn the i Blan a urement criteria) would include factors like; a rehint * Wh toe yon at are the cheapest/most convenient ait-connection, Ks India to San Francisco? Ss that exigt fr Mra ® What is the budget for your hotel stay in the Bay Ateaz © Which hotel offers the most comfortable toom within th ", the Fisherman's Wharf? e budget nd Sq AN (ane ther On the other hand, if you are starting your journey at Stanford, (and therefore the measurement criteria) would not be concern, an 'Otnia fare from Bangalore to San Francisco or the best connections ec With the cnr Play focus on: Possible tt wou iy ® Would it be more practical, to take the public transportati " down? 19M OF shoulg ®@ \f driving, which are the best routes/exits | would take? Whe : much time should be allocated to travel given the traffic aan Id Park? availability). Mditons (ang mgt So, itis obvious that you would start with an assessment of where you where you want to go, before drawing the roadmap of what you ware ‘Urrenth toment®ad| In the above example, let us assume you are starting from Stanford and Fisherman's Wharf. Let us assume that you planned and took the US ia diving alight at the Golden Gate Avenue Exit. If, after dri half. there is a traffic backup and that the US 101 Hi you do? You would obviously have to abandor ably take some surface roads, reach a differe your final destination within time. In other words The last step in the above example illustrat program: an essential part of any measurer ‘enues for periodic health checks by measur will start off with some initial plan (e.g., t _ Or measurement criteria, carry out those int form to what was expected and if the values corrective action. Each one of the steps also very easy to miss! And when missed, i program totally ineffective.8 Metrics ATYPICAL METRICS STRATEGY BZ Having discussed the overall principles and the purpose behind metrics in the previous section, we will now focus our attention on an outline of the steps that constitute a sound metrics strategy © Decide what you want to measure and how you are going to measure it: It is very important for all the constituents to know the rules of the game, ie., what are the measurements that matter and how they are carried out. This helps everyone to work towards the same goals and have a common understanding of how to measure progress. » Set targets and track them: While the main goals (“where you want to go”) may be well defined qualitatively, it is also important to set specific quantitative targets. This step is about how to quantify and track the targets. * Understand variability and work towards minimising it: One of the fundamental tenets of quality performance is achieving consistency and predictability. Variability is the opposite of predictability. This step is about how to track variability and what to do to minimise it. * Act on data and strive for continuous improvement: Measurements are useful only when we act on them and take conscious decisions based on them. This step discusses what kind of action is needed and how to achieve this action. * Consider the human angle: Unlike most other industries, the software industry . is very people-oriented. We cannot measure people's progress the same way we would measure a machine's performance. There are significant human issues that need to be addressed and this step covers some of those issues. The subsequent sections go into the details of each of the above steps. WHAT SHOULD YOU MEASURE? With the advent of tools and technology, it is very tempting to measure Y a lot of things! But it is important that we measure those attributes that are related to the project goals and/or organisational goals. The purpose of measurements (and the subsequent control if applicable) is that they should lead to achieving the goals faster, more economically , with superior quality output and better employee/customer satisfaction. To achieve this end, we present below some questions that one should ask to decide whether a particular factor is worth measuring: * Is the entity that we want to measure directly relevant to the goals of the project | organisation? Suppose you are developing a software that is intended to run only on one single platform, then any measure of portability is notManaging Global Soq 64 _— and, if the stated goal j “ sraject On the other hand, 01 is th; geneva tothe Pg platforms, Ore Ta the metrics could be the te Ri eho i ed in porting the prod tte le tea lation fon tg effort inven ty nparurally” measurable? Idea ly, there atthe ty , measurements. Ina Practica, ul tt » fe the entity. extra effort inv averheads incu ved ot the measurements should neg 1%, Ne ir he Wreasurement are done to identify problem tts. hs fie iho) action ifthe process of taking meastironsr Te (066 and a sade ‘wubetantally to the over! eads, then the benefits gained wil] a, et tantly i lew) thie} en . om ae) the cost and benefits of mE is important, Mey in order to ensure that the benefits exceed the costs incy, Ody f the major challenges in any metrics Tr - (discussed in more detail 5,8) is the perception that meas, Rtn «ve rise to bureaucracy: Quantifying the costs an ae, in the context of their relevance to the project/ organisation goals benef helps in combating such negative perceptions. ttn © Is the entity controllable? Ideally, we should measure only those 4}; which allow us direct or indirect control over their effects. For examp} things were to set up an office in the San Francisco Bay Area, it would not be a our while to measure the frequency of earthquakes in that area becay, know that the area is prone to earthquakes and that any past data Se We pert the time of ake (and anyway, we have decider the next earthqu: ve an office in that area for business reasons). © Can you afford NOT to measure the entity? No matter how easy it j measure an entity, one should prioritise and measure only those me ‘Sto absolutely have to be measured. There are several reasons for this. First, 7 every measurement comes with a price tag and adding new meas ofall, would just add to the cost. Secondly, there are just a few things that measured and controlled appropriately, would improve significanth When inal improvement. Hence the mea a Ober things, at best, would show mar; imp! program should be limited to ‘measuring only those things which when controlled, produce the best bang-for-the-buck. SET TARGETS AND TRACK THEM In the previous step, We identified [Link] want to m\ this measurement is related to the project/' ee 5 ng is to ensure that certain targets are set for these entities, For exam deliveries, y coat ofa where the criteria for success is time-boune : a metric may be the “deviation from the scheduled time to the time taken”. This is called schedule variance in the project management patl6 Metrics However, itis not enough to say “we will track schedule variance as a metric”. It is also important to set some targets whic 1h reflect the degree of success of the project. These targets should then be understood by all the constituents who are concerned with achieving these targets While setting these targets, it is important to ensure that the targets satisfy the SMART criteria. SMART is an actonym for Specific, Measurable, Aggressive yet Achievable, Results-oriented and Time-bound 1 should be specific in terms of numbers. E.g., “the project shall Specific: The target ”” In particular, it is be delivered no later than within two days of the scheduled date important to avoid targets like “this project should be completed as soon as possi- ble” or “we should have as few defects as possible”. Measurable: The value of the target should be measurable unambiguously and in- terpreted consistently. There should be no disputes on the value or how itis arrived at. An example of a measurable target is “keeping the number of bugs in the prod- uct in single digits”. An example of a target that is not measurable is “what the customer feels about our product”. Aggressive yet Achievable: The target that one i utterly impossible to achieve (this will only discoura; should it be too easy (this will not be challenging an people enough). An example of a target that is aggressive and yet achievable could be “achieving a 20% speed-up in the delivery time for each cycle without compromising on qual- ity”. A goal that is not aggressive is, “competition is delivering this feature in three months — we will take six months!” Results-oriented: The measurement you are taking reflects a symptom. What you need to convey is how this measurement of the symptom has an impact on the final business outcome or project outcome. For example, when you measure time taken to fix a bug and the number of bugs over time, and find that one or both of these are decreasing, you can show how this measurement results in lower re-work costs and thus make the constituents realise the eventual benefits of what they are doing. This puts ownership of the results in the hands of the constituents. Time-Bound: The targets should be achieved within a given time-frame. In the era of the “Internet Time”, this time-frame is becoming ever shorter. Targets that do not have a time-frame associated with them do not serve any purpose at all. An exam- ple of a time-bound target is “Reduce the bug count to 50% of the current values in the next three months.” In order to ensure that the goals and targets are time-bound, it is important to avoid phrases like “as soon as possible” or “at the earliest oppor- tunity” in the statement of targets. is trying to achieve should not be ge and demotivate people), nor 'd hence will not motivate the> Managing Gloty Global Sofia 7 roy ey smart Goal A Great Example of a to cet SMART goals Was President y key xamples of how ji 1e best examp man’s conquest of the moon, "ty, f th vei that set the agenda for inspiring, speech hhoose to go to the moon in this We choose to goto the moon. We ¢ deca the other things, not because they are easy, but because they are hara, fa ty that goal will serve ize and measure the best of our energies ang Casg ne that we are willing to accept, one we are ski because that challenge is 0 kil oF papone, and one which we intend to win, and the others, too... ‘ning We have felt the ground shake and the air shattered by the testing of a Satur in mes as powerful asthe Atlas which launched John G ent to 10,000 automobiles with their acceleraton es 19 months at least 45 satellites have circled the he booster rocket, many ti generating power equival floor... Within these last \ Some 40 of them were ‘made in the United States of America’ ... earth, During the next 5 years the National Aeronautics and Space Administration & pects to double the number of scientists and engineers in this area, to in ‘outlays for salaries and expenses to $60 million a year; to invest some ‘cae Jion in plant and laboratory facilities; and to direct or contract for n 00 mit. efits over $ 1 billion from this Center in this City”. eW Space Look at how beautifully each criteria of SMART has been subtly but fi dressed: The target is very specific—he does not say “we will meander i nl a oF ty landing somewhere”, he say, “Wee choose fo goto the moon” nto spacer clear and specific. It is also clearly measurable: Either man has tended moon or not—as clear as black and white, with no uncertainties. He oul that the goal is aggressive—with the phrase “not because it is eas ra = is hard”. He also encourages people by telling them that the geal e ie 5 even though aggressive by citing examples of the frontiers con cae te far. The results orientation and the link-up to the ifjasleatinal nationa) eae vision is brought out by the urgency (“...not willing to postp it Pan tas io st treline= alana anne peaouan | ‘ . What is even | more interesting is that he has also out increase in investments on space nae y and Sr naa sident Kennedy launched rouael goal and the charismatic personal one of the greatest exploration: an1 TIES scat nas inhale UNDERSTANDING AND TRYING TO MINIMISE VARIABILITY ns Zz a When we discussed the process models, we discussed that one of the most important aspects of successful project management is achieving, | consistency and predictability. When there is consistency and predict- ability, the management would be able to bid for new projects with con- fidence knowing that they would be able to meet the requirements and estimates. Also, with such a track record, the Prospects and customers would feel more com- fortable doing business with the organisation. Much as we would like to achieve consistency and predictability, there are al- ways variations. These variations are a result of the inherent human nature (which is so prominent in software), unclear understanding of the requirements, changing technology and any other unforeseen reasons, Variability refers to how the key metrics vary over time or over different projects. As we discussed in the previous Paragraph, certain amount of variability is inevita- ble, but it is important to ensure that 1. The variability is within “reasonable limits” (called Lower Control Limitor LCL and Upper Control Limit or UCL) and hovers round an expected mean value. 2. When the variability exceeds these limits, we should understand exactly why | this happens and how we can minimise such a wide variation in future. How do you arrive at the specific values of UCL /LCL and the mean value of a metric? * You can benchmark yourself against the industry norms. As the ultimate objective of metrics is to ensure you stay competitive, it is necessary that you do not fall short of the prevailing industry norms. * You can examine your track record in similar projects (if you have got this data). This can give you some guidelines as to your potential, capabilities and limitations. * Your management may simply state the required value of metrics as targets to shoot for (after all, how did President Kennedy choose end of the decade as the time-frame to land on the moon!). Let us analyse each of the above concepts in detail with an example: Suppose you are in a software maintenance project fixing bugs in a product. In addition, also suppose that on an average, you expect to fix (or produce a responsecsusissuauiias Maneging Glo Hor no bug should the response ex ould be some customer lasattafae a ins esaistaction i ere W iid take a minimum of ime taken to set up the bug, Hunton, ree to fix tts 4 distribute the fix. 1 Feprodicg tt any bug-fix time should ideal} dan Upper Control Limit (UCL) Wat 5d, above assumptions, Limit (LCL) 1 day an Under the Lorwer Control mean of 3 days: ‘The meaning of these limits is that any variati within the range of 1 day to 5 days can be construed it nota time a wig? or bugfix times follow the *fistribution in Fig. 5.3, we can see hae example thy fall been te a and ic uch a distribution of bug-fix hte Of they, "8g “acceptable distril ution” as the bug-fix time: iia eae able limits. 2 i St oe opens (ie, Numbérs of days t fe abogMetrics £9 o ° Bucs ° é 2 x § . x x 5 Mean =3 x x § x x Reon ¥ ¥ E 2 tcl=1 . * EE Fig. 5.4.4 small percentage of values lying outside UCLILCL Even though the distribution still tends to be skewed towards values within ac+ ceptable ranges, it is very important to look at the instances where the value falls outside the UCL/LCL limits and understand the root cause of why such a deviation took place. For example, if you view the cases where the bug-fix time took more than 5 days, you may find that in those instances, any one of the possibilities listed in the first column of Table’5.1 could have been the cause: Table 5.1] Possible Root Cause Analysis for Observations in Fig. 5.4 ‘Machine outage happened during the time that these bugs were being fixed © Allocating more reliable machines to critical bugs © Changing the supplier of the machines * Get into a Service Level Agreement ‘The bugs required some information from the customer and the customer with the customer to provide all the did not get the information on time. information ¢ Design a template or check-list that elicits all the information needed © Change the metrics from absolute number of days to number of days after the complete information is available : (Contd.)Munnging Ctobat Softinare Projects the effecti Tere cause These bups were the fi by anew recrisit into the 1 if aout to be muich more The Bugs turne: complex than the “Fi Artive at a mechaniam of f aniarn of elaeal ermal” BURS the buga into categories and Mone target metrics separately for ah et the categories rather than provid ned universal target (oF all categart sting one of bugs. an analysis such as above is that the corrective acti needed to remove the variability and to smooth out the curve could be very ditter depending on the root cause. AS Cm roveen from the second (“Corrective ri tions”) column, action needed to minimise the recurrence of a similar varlanent future is very different. Such analysis and corrective action constitutes defect ein vention and leads to continuous improvement. Pre. Let us consider two more scenarios of behaviour ple: Let us suppose that the 20 observations you ™ the distribution pattern of Fig. 5.5. The purpose of doing with respect to the abov, ade for the bug-fix ine Satta Ow 3 2 s é 2 g 3 Ei a & z ofRFig. 5.5 Large percentage of values above the UCL Ae f you can notice from the graph, most of the bugs took more than 10 d lays to: fix whereas your UCL target was 5 days.a Tf you have achieved the tatget of 8 days consistently in the past) you should examine what has changed so suddenly to take this number to ten days and above? Has there been a greater attrition resulting in a larger number of new (and not filly trained) employees? Has there been a sudden increase in complexity of the product? Was there a new Version released to the market that caused a sudden increase in the inflow rate of bugs that caused the fall in service levels? Ifthis is the first time that you measured the targets, perhaps, your initial expecta- tion of what UCL/LCL should be is Not correct. If you have not received adverse customer comments, perhaps, striving for 5-day turnarounds is an overkill and you can make do with, Say, a seven day turnaround, with no adverse customer impact (youcan then divert the extra manpower gained by increasing the mean fix time by 2 days into activities that do have a more serious business impact). Whatever be the case, as the number of values that fall outside the UCL/LCL has increased, the cor- rective action should be more urgent and is likely to result in more significant over- haul of the targets or the environment, Metrics Ina final scenario, what if most of the values for bug-fix times are under half a day? It probably means that by setting a target of 1-5 days, you are not setting an aggressive enough target for the organisation to shoot for. Thus, over the long term, this could result in a decrease in motivation and productivity of the organisation. One of the main purposes behind gathering and analysing metrics is to ensure that the organisation stays ahead of competition through continuous improvement. By identifying the root causes of those instances of apparently out-of-range values and employing corrective measures for the same (as listed in Table 5.1), the organi- sation can remove most of the impediments and thus set the scene for continuous improvement. As the variability of metrics gets minimised, you will find three pat- terns emerging in the graphs of a continuously improving organisation: * The number of data points that fall outside the LCL-UCL range is reduced. This implies that more of the data points are within acceptable ranges. For those exceptional cases that do fall outside the UCL-LCL range, there is a clear understanding of the root cause for their falling outside the range. * The gap between the UCL and LCL is reduced and this means the metric lies within a much narrower band. Hence the predictability of the metric increases. * The acceptable level of performance or the mean value of the metric is continuously improving, The reason for this is that as you identify and remove2 Managing Globat so rt Softy, root causes of problems, you are weeding out the problem hives ~— Prien aind hence are providing » more conducive environment fro by come, improvement On Mien, Woe conclude this section by summarising the above discussions j sin salient points the fy r ™ ing Minimising variability in the chosen metrics is critical for busin, Fach metric is characterised by its mean value, upper control nt SUeceg, control limit Mand toy, For those values that fall outside the UCL/LCL range, it is im understand the root causes and take corrective action to reduce the ortant +, of such out of range data points in future CcUtrencg » As the next step in improving predictability, one should try to UCL-LCL gap (ie,, make the points fall under narrower ranges), » Finally, in continuous improvement, one should seek to brin, improvement in the base numbers itself, while keeping Variability 4 out Nder control. ce ty ACT ON DATA! We have already covered this indirectly in the previous section, But step is so important and easy to miss, that we would like to draw ong attention to it once again. It is very tempting to collect a lot of data mn nice, colourful graphs, but not take corrective action. The job of ie does not end with metrics collection and analysis. It begins there. The real culiting. tion of the process is when the data is acted upon and corrective action taken, a ing in continuous improvement, is seen. The importance of this step emanates fom three different (but related) factors: 1. Metrics collection and analysis is an expensive and skilled-labour intensive process. The benefits accrue only when corrective actions are taken. Hence, if we do not act on the data, metrics activity becomes just a costly exercise with no benefits. Continuous improvement is absolutely essential even if we want to stay where we are (let alone improve our position) in the market. If the findings from metrics analysis are not acted upon, the organisation is unlikely to achieve continuous improvement. NB 3. All the constituents do end UP putting in an extra effort into gathering these metrics. If they do not see any action ot get any feedback, their faith and willingness to gather and supply meaningful data will diminish. We will cover more of these human angles in the next section. Metrics When we say “Act on Data” what we mean is, “take conscious business decisions pased on the data and the analysis of metrics”, Status quo (or not taking any action) is acceptable so long as it is a conscious decision and is justified by what you see in the metrics. Also, we would like to emphasise that the metrics data should be used for making business decisions and not just some fringe decisions that are not core to the success of the business. At times, making business decisions based on metrics could be tough, but just for the sake of acting on the data, one should avoid the mistake of doing some superficial fixes on Non-core issues. In the long run, such an action (of superficial fixes) would erode the credibility of the metrics initiative. The specific actions we would recommend are: » Ask questions like “What if”, “Why not”, “So what?”, etc. For example, “What if the mean time to respond to a bug oes up from 5 days to 7 days? What is the business impact (preferably quantified in $ terms)’. « Arrive at root causes; don’t be satisfied with te-stating the symptoms. This point was discussed at length in the previous section. 5 + Decide whether the resultant action is a process change (e.g., how to route a - bug to ensure that it gets fixed in the fastest time) or a product change (perhaps a feature is bug prone and one may consider modifying the product with respect to this feature) or no change (i.e., status quo). « Analyse whether, on the basis of the changes, the current metric still makes sense. For example, if you modified the Process of submission of a bug to specify a minimum set of information that must accompany a bug, then the metric of time taken to get the minimum sei becomes irrelevant, * Complete the feedback loop to the people who supplied the data for the metrics. It is important for them to know that their data was acted upon and decisions were taken on the basis of such supplied data. PEOPLE AND ORGANISATIONAL ISSUES IN METRICS PROGRAMS F Implementing a successful metrics program does require whole-hearted Z co-operation from all levels in the organisation. Hence people issues play a very significant role, In this section, we will address some of the com-— Software Configuration Management change is the ‘only constant.” m Anonymous What is Software Configuration Management (SCM)?—Typical problems related to Change Control—Basic terminology and concepts in SCM—Processes and activities in SCM—Status Accounting—Configuration Audit—Impact of Distributed teams on ‘SCM—Metrics for SCM—SCM tools and automation. INTRODUCTION <=> We begin this chapter with a review of some of the characteristics of the YW software development environments that motivate Software Configura- tion Management. In section 6.1, we introduce some basic terminology used in SCM; sections 6.2, 6.3 and 6.4 covers the basic processes of SCM; in section 6.5., we address some issues in SCM that arise from geographically dis- tributed development teams. In section 6.6, we discuss the metrics that apply to SCM and in section 6.7 give an overview of the requirements of the SCM tools. Let us take a look’ at some of the characteristics of software development in to- day's product development organisations: Break-up of products into components and inter-component dependency: Today's software products are broken down into multiple inter-related and inter-dependent compo- nents. A change in one component of a product may have a ripple effect on other components of the product or even on other products (even though this is not a desirable situation). Consider the example in Fig. 6.1 where there are three prod- ucts: X, Y and Z. Product X is made up of two components—X1 and X2; product Y ismade up of three components—Y1, Y2 and Y3; product Z is again made up of two components—Z1 and Z2. Let us also assume that the component Y1 has a depend-jpoint of view of the end user, he has to get the right configuration (or rion) of each of the above, else the system may not be fully functional. fone were to buy a machine that runs on the Linux operating system the shrink-wrapped software version for the NT operating system, K Alternatively, if one gets all correct versions of the software but gets versions ‘of the documentation, the system may not be usable! ssure that the end-user gets the right configuration, i.e., the correct ‘of the hardware, system software, shrink-wrapped software and ‘the manufacturer must have a proper configuration management sres that the right components — be it hardware, software or docu- oo AND ACTIVITIES OF SOFTWARE MANAGEMENT like metrics, software configuration management (SCM) is an um- tivity that applies through all the phases of a software develop- life cycle. Again, just like in metrics, where even though what one es would be different in each phase, the basic principles and phi- ‘the same; so also, what one subjects to software configuration man- ld be different in each phase, while the basic principles and activities / would remain the same. What we will discuss in this section are / steps of SCM. At the end of this chapter, in Table 6.1, we ssible items in each phase that need be subject to software configu- of workspaces on status accounting n audit f h of the above points in the following sections and sub-sections.88 Managing Cota) Wns 18 Dy & 6.2.1 Initial Working i Almost all life cycle activities generally starts off with an initial Ad-hocigg, a small set of peaple (sometimes even just one) who do the initial researey "hg, munication, as well as the spade work that is required. During this time, ANd ~ files are held privately by the individuals. Sharing is need-based and fe Often ofthe plished by across the hallway shouts. There is absence of certain formatig, om, communication and sharing of files and Tesources. This informality helps I thy responsive to quick changes that are so characteristic of this phase of the j la being Also, during this phase, very little of the work done by this group is fddes Jee The product or the component being developed is put to use only internap example, if the product being developed is a set of APIs (Application Frogte For Interfaces) that enable other products to call the services of this product, wean are not yet exposed fully to other products. Any testing is done by local simula using internally developed test harness rather than by exposing this prody, cttg ion groups. Othe, Eventually, the team that is working on this phase of the product Comes to stage where they (in conjunction with other deciding authorities) decide that work product and all the items under development are in some reasonable = stability and that others can start using this work product under development of the other groups (or products) might reach a situation where they absolutely 4 access to the services provided by this group (or product). It is at this point ofting that the work product passes on to the next stage of SCM activities called basetin, \\ 6.2.2 Baselining Baselining is the process by which a given set of items formally become public) available in a standard location to the people who are authorised to use it, erase or modify this file at this local separate files with different version on, even such additions/changessnare Configuration Management 89 eas of baselining also establishes the appropriate authorities who can review and / or approve any further changes. As can be seen from the above discussion, baselining involves the following ac- tivities: » “Freezing” a current version of the product and its constituent elements (like source files, makefiles, etc.) » Allocating a configuration id to the entire configuration + Allocating version numbers and standard locations to the constituent elements of the configuration * Storing the approval authority information * Finally, broadcasting the above information to the concerned people. The vehicle used for these storage and communication purposes is the Configura- tion Management Repository (or Repository for short), The repository is the central place that contains all the configurations that have been made public along with their related items, the authority levels, etc. In other words, it contains information about all the baselined items. One can view this as a database of all configurations. The repository also records all the Change History information (as discussed in the next sections). The process of baselining (or re-baselining for a change) is also re- ferred to as check-in. It is interesting to note that by the process of baselining, the original owners of the configuration items are in a sense relinquishing some of the control and flexibil- ity they had to make any changes as and when they felt like and are, instead, volun- tarily imposing some structure and process in their functioning and that of the oth- ers to ensure that further changes are controlled. This extra care is required because the items could be used by other people and in ways not fully visualised by the original owners, the impact of the changes could be more significant than what the original owners could have imagined. Itis not necessary that all the items being worked on by the original owners get baselined. For instance, there may be some temporary files, internal test harnesses or email correspondence that might have been used and would therefore serve no purpose if shared by other users. Hence these items do not get baselined.Mérioging Cay, : fh Pig, Some Criteria for Choice of Items for Bavelning Th goneral, hacelite thoee Herne that are: © needed to recreate the configuration (6.8, source (1 compilers andl Finkere) *S: $Pecite % © needed for cretion of ject in the next phase of lite eye a ‘specs ate an input to the design phase) eg, © needed to establish the correctness or acceptability of the phase (e.p,, test logs from the testing phase or in general, any nt ty A popular misconception is that onl the component that ae ity oy in the configuration need to be baselined. For example, during tent phase, only the source files are baselined. tis equally importang deat hg environment around the source files ~ this includes the OS, the *® to the Com, the util; phase utili What are the items that usually need not be baselined? © Anything that is transient only to the builder/develope, r eg, correspondence) © Anything that is initially designed as a place holder for fy ‘harnesses) to be later filled up with the appropriate content © Any intermediate files (e.g., the myriad versions with the “[Link]” of the temporary variables!) sh nf SR oe "ure fillings \ 6.2.3 Change Management After any configuration item gets checked-in and is baselined, changes are take place. These changes can come about because of several re cha requirements, changes in design or architecture, changes in the platform ronment or simply changes caused by the initial opti 4 programming fraternity! ee The objective of change control (and indeed of SCM) well thought out and done consciously. One should91 «and authorisation jn a workspace : after baselining and when the product is in use, someone syganced tO make a change in some of the items. Such a need can be sae reasons mentioned in the previous section. When the need for 2d, the person who realises such a need (possibly the person who ge) raises a Change Request. The change request documents what the change is needed, what items would be affected by the jon of what changes would be made on each of the items and ion) what the impact of this change would be on other items as ge request may originate from someone who knows what the from a business perspective but may not be aware of what the sct of the change is (i.e., what files would need to be changed, which s get impacted, etc.). In such a case, it is important that the are also brought to light by someone before a decision is made t gets evaluated by the Review/ Approval Authority in the wiew/Approval juest, with its justifications, is raised it goes to the review/ap- this discussion, we have assumed that there are différent re- 3 ies for different products/configurations. Another varia- el is that all the change requests in an organisation go to a | Board, which performs exactly the same functions as described oval authority reviews the change request from three dimen- ropriateness and Impact. sion establishes whether a particular change is required at all; hnology change warrants this change request; does this change genuine requirements change or is it one of those “cool things” do with no business reason to back it up? In essence, the neces- that the change request is not a “feature creep”.: —Manesing Glog Glo 2 ann qacortained that the change request jg ~ ints , hat the change is carri Niece apron” a porics ensures une tried oy , the ppp : what” and “Why’ of the change, ve tig ¥ iy ay cuss ascertain 4 spropriateness, information aber ons tn files nee be changed) must be made availaty, OW (ho hy. +e the best Way to implement the changer” e. ‘lle q ; the chan 7", “Does the implementation trul Ps th ke tne hange is i y catty... the to ensure that the change 'S implemented cop, Cc i Tecth inal imension of an analysis Belore approval is the y. rons such & ag this change 'S made in the manner Proposed tan vould it have OF components?”, How much effort would », hay iy this change # what should be compromised to ensure that the re into pa vnade avail able”, etc. Impact analysis also considers en fe ky on this change request and vice versa. The ek etect ofits hidden surprises oT side effects in imy ct dimeng: er plementing this ion ey. rises the ke uestions y q! that the reviewer m ie ust pending there are no sures that The checklist below summa before approving 4 change request. Checklist for Revi ewing Change Requests Necessity ements change that triggered this ch; ange request 2 fed the requir nts) request originated from an appropriate authority? ed evidence for the requirements change? x uirements that conflict with the ci nth current it c e equest objective and clear (e.g., eee ‘changing to improve performance” one should use a more specifi ot seg “removing the performance overhead caused by condition XYZ")? cet [Is this a reactive change (i.e., correcting an existing anomaly to meet an expe functionality) or an enhancement (ie., adding a new feature)? sa = Have you (or the one who requested the chan; " \ge) been able to establis Fp the revenue / productivity or profitability that would arise from a aa especially if it were an enhancement type of [Link]? ee = Can you afford not to make the change? = Have you identi ig Has the (requireme ix [s there a document is Have there been any other req [Is the statement of change ftSoftware Configuration Management Appropriateness © Has the requestor of the change also outlined how he is planning to implement the change? Ate alternative strategies outlined and evaluated for relative merits and demerits? Have you, as the reviewer, got all the information needed to validate the implementation? Does the current implementation method appear to be the most cost-effective? Does the implementation method seem feasible? Impact What are the resources involved in implementing this change? Have you got an estimate for the manpower and machine requirements for implementing this change? Do you have bandwidth available to carry out the change? If not, where are you stealing the bandwidth from? What is the impact of such a move? Have you got the buy-in from the implementation team to ensure that they will make the change and under the given resource constraints? Will this change upset the apple-cart for any other requirements or change in progress? What is the impact of not making this change? Have you considered the run-time ramifications of making this change (e.g., if you are making a change to an algorithm, have you considered the memory-performance tradeoffs)? Once the approving authority is comfortable with all these three dimensions, the change request gets approved and this enables the actual change to be carried out. [Link] Making the Change: Workspace—Check-out and Check-in Once the change request has been approved, the next step is to make the actual change. As discussed earlier, it is likely that more than one developer feels there is a need to make changes to the same item (possibly to fix different problems). In order to ensure that no changes get lost or over-ridden due to other changes, there is a need to serialise the changes. The situation is not very different from that of a multi-user database management system when two users are concurrently trying to update the same record. Such updates must preserve the ACID property. ACID stands for Atomicity, Consistency, Isolation and Durability. It is common that a change request modifies multiple files. In such a case the changes to all the files must take place as one unit. This is referred to as ‘Atomicity’. The other side of the atomicity is consistency - the updates start with a consistent state of the repository