Performance Measurement

Lesson Structure
1. Improvement
1.1 ‘Continuous’ vs ‘Continual’ Improvement
2. Measuring Performance
2.1 Why Measure Performance?
3. Performance measurement framework
3.1 Establish Key Goals:
3.2 Establish Metrics (indicators):
3.3 Understand Performance:
3.4 Initiate Improvement
4. Elements for successful performance measurement system
Appendix-I: Examples of Performance Indicators (Metrics) in Manufacturing
Appendix-II: Measures of Service Quality

5. Improvement

Continuous improvement is an ongoing effort to improve products, services or

processes. These efforts can seek “incremental” improvement over time or
“breakthrough” improvement all at once.

As per ISO 9000:2005 the term ‘improvement’ has been defined as follows:

“part of quality management focused

“part of quality management focused
on increasing the ability to fulfil
on increasing the ability to fulfil
quality requirements”
quality requirements”
“NOTE: The requirements can be related to any
“NOTE: The requirements can be related to any
aspect such as effectiveness, efficiency or
aspect such as effectiveness, efficiency or
traceability”
traceability”

Form the above definition; it is clear that for making improvements, we need to
improve the ‘effectiveness’, ‘efficiency’ or ‘traceability’ of some product or service
requirements.

1.1 ‘Continuous’ vs ‘Continual’ Improvement

The terms continuous improvement and continual improvement are frequently used
interchangeably. But some quality practitioners make the following distinction:

Continual improvement: a broader term preferred by W. Edwards Deming to refer

to general processes of improvement and encompassing “discontinuous”
improvements—that is, through many different approaches, covering different areas.
Improvement in business strategy, business results, and customer, employee, and
supplier business relationships can be subject to continual improvement. Putting it
simply, it means ‘getting better all the time'. Continual improvement should focus on
enablers such as leadership, communication, resources, organization structure, people,
and processes - in other words, everything in the organization, in all functions at all
levels. Continual improvement should also lead to better results, such as, price, cost,
productivity, time to market, delivery, responsiveness, profit, and customer and
employee satisfaction. There has been a tendency in total quality management
programs to focus on departmental improvements which do not improve business
results overall. Departmental improvements may merely move the constraints or
problem somewhere else in the process chain.

Continuous improvement: a subset of continual improvement, with a more specific

focus on linear, incremental improvement within an existing process. Some
practitioners also associate continuous improvement more closely with techniques of
statistical process control.

Continuous improvement is gradual never-ending change, whereas continual

improvement is incremental change. Continuous improvements are linear, incremental
improvements to an existing process (Kaizen). Continual improvement includes this, as
well as discontinuous/innovative improvement.

Continual improvement can be considered as a set of improvement efforts (or

projects) over time in different areas through different quality initiatives/programs.
(e.g. I-1, I-2, I-3, etc. as shown in diagram below). These improvement efforts may
be spread over different times or may be carried out simultaneously or in a
combination of both. Any single improvement effort may consist of:

(a) Continuous improvement of the process (starting from a known performance

level and reaching a targeted level in a specified time).
(b) Stabilizing the gains (consolidating the newly acquired level of performance).
2. Measuring Performance

We have seen in the above discussions that in order to make quality improvements,
we need to improve performance in terms of either effectiveness or efficiency. For this
purpose, we need to know:

a) Our current level of performance (in terms of effectiveness or efficiency), and

b) The desired level of performance (again in terms of same measures of
effectiveness or efficiency).

The above can be known only if we measure the performance through some indicator
or metric.

2.1 Why Measure Performance?

It is well known that:

 If you can’t measure something, you can’t understand it
 If you can’t understand it, you can’t control it,
 If you can’t control it, you can’t improve it.

In the cycle of never-ending improvement, performance measurement plays an

important role in:
• Identifying and tracking progress against organizational goals
• Identifying opportunities for improvement
• Comparing performance against both internal and external standards

Reviewing the performance of an organization is also an important step when

formulating the direction of the strategic activities. It is important to know where the
strengths and weaknesses of the organization lie, and as part of the ‘Plan –Do – Check
– Act’ cycle, measurement plays a key role in quality and productivity improvement
activities. The main reasons it is needed are:
• To ensure customer requirements have been met
• To be able to set sensible objectives and comply with them
• To provide standards for establishing comparisons
 • To provide visibility and a “scoreboard” for people to monitor their own
performance level
• To highlight quality problems and determine areas for priority attention
• To provide feedback for driving the improvement effort

It is also important to understand the impact of TQM on improvements in business

performance, on sustaining current performance and reducing any possible decline in
performance.

3. Performance measurement framework

A good performance measurement framework will focus on the ‘customer’ as well as

‘stakeholder’ requirements (i.e. doing the right things rightly).

It will contain performance measures (indicators/metrics) on ‘effectiveness’ as well as

‘efficiency’.

Effectiveness: measures that impact customer requirements (doing the right things –
measurement of results)
Efficiency: measures that impact organizational (stakeholder) requirements (doing
things rightly – measurement of costs vis-à-vis results)

Performance measures must be:

• Meaningful, unambiguous and widely understood
• Owned and managed by the teams within the organization
• Based on a high level of data integrity
• Such that data collection is embedded within the normal procedures
• Able to drive improvement
• Linked to critical goals and key drivers of the organization

There are four key steps in a performance measurement framework:

 Establish Key Goals: the strategic objectives of the organization are converted
into desired standards of performance,
 Establish Metrics (indicators): metrics are developed to compare the desired
performance with the actual achieved standards,
 Understand Performance: gaps are identified, and
 Initiate Improvement: improvement actions are initiated.

These steps are continuously implemented and reviewed:

3.1 Establish Key Goals:

Initially, focus on a few key goals that are critical to the success of the organization or
business, and ensure that they are SMART, i.e.:

Specific
Measurable
Achievable
Relevant
Timely
Quality
To assist in the development of these goals, consider the use of a balanced scorecard,
as discussed in the Appendix-I of this lesson.

3.2 Establish Metrics (indicators):

Once the goals have been defined, the next step in developing a performance
measurement framework is to define the outcome metrics - what has to be
measured to determine if these goals are being achieved. If it is difficult to define
outcome metrics for a particular goal, it is possible that the goal is either not “SMART”
or critical to the success of the business.

For each outcome metric, brainstorm candidate drivers by answering the question,
“What measurable factors influence this outcome?” Once the list is complete, select
those with greatest impact, and these, the most important drivers, should have driver
metrics, and be put in place first. Driver metrics at one level will be outcome metrics
at the next level down.

An organization needs to evolve its own set of metrics, using any existing metrics as a
starting point in understanding current performance. To ensure they trigger the
improvement cycle, they should be in two main areas:

Effectiveness = Actual output x 100%

Expected output

This is about the process output, and doing what you said you would do. The
effectiveness metrics should reflect whether the desired results are being achieved,
the right things being accomplished. Metrics could include:
 quality, e.g., grade of product or level of service,
 quantity, e.g., tonnes, or
 timeliness, e.g., speed of response, and cost/price, e.g., unit cost.

Efficiency = Output x 100%

Cost

This is about the process input, e.g., labour, staff, equipment, materials, and it
measures the performance of the process system management. It is possible to use
resources efficiently, but ineffectively.

Simple ratios, e.g., tonnes per person-hour, computer output per operator day, a can
be used.
3.3 Understand Performance:

Next, design a data collection/reporting process using the following steps:

• Set up a system for collecting and reporting data
• Write clear definitions
• Agree method for establishing current performance (if not already determined)
• List resources required to support the design
• Agree data formats and classifications for aggregation and consolidation
• Identify possible sources of benchmark data
• Set reporting calendar
• Establish roles and responsibilities
• Detail training requirements
• Validate with process stakeholders

The gap between current and desired performance now has to be measured. Some of
the metrics already exist and their current performance data must be collected, as well
as data for new metrics.

Once all the data has been collected to identify the current performance, the target
level of performance for the medium- and long-term must be decided. These
performance levels must be achievable, and should be broken down into targets for
discrete short-term intervals, e.g, the next three quarters.

3.4 Initiate Improvement

To implement the performance measurement framework, a plan with timescales and

designated responsibilities is needed. Once the plan has been implemented and data
collected, new baselines can be set, comparisons made and new standards/targets set.

The metrics, targets and improvement activities must be cascaded down through the
organization, involving people and teamwork in the development of new metrics, data
collection and improvement activities.

Improvement can be initiated by examining the gaps between current and target
performance of the driver metrics at each level. A minimum, achievable set of actions
is determined, with plans, assigned responsibilities and owners.

4. Elements for successful performance measurement system

There are a few key points to take note of if you wish to develop a truly effective
performance management system, and use Performance Measures to drive
improvement. These are:
• Ensure that there is a high level of ownership of the performance measures used
by those that can actually influence performance
• Make sure that the measures that you select are:
o Few in number
o Balanced, and focused on the key areas that are important to your business
o Congruent with your business goals
o Controllable by those whom you are holding accountable for performance
• Ensure that targets (and more importantly target ranges) are established for all
performance measures used
• Make sure that you effectively close the control loop – ensure that the performance
management process is action-oriented, and that improvement actions are actually
put in place.
Appendix-I

Examples of Performance Indicators (Metrics) in Manufacturing

A. Quality Related Measures

Incoming Material Quality and Delivery Measures

 Percent of vendors that are certified
 Percent of lots sampled lots that are rejected
 Frequency of times production process problems or quality can be traced to
incoming product quality
 Percent of deliveries that arrive within the prescribed incoming material time frame
specifications

In-process Quality Measures

 Process capability (e.g. Cpk)
 Percent scrap
 Percent rework
 Percent of labor dollars spent on inspection
 Percent of deliveries that have the correct amount of the correct material

Outgoing Product Quality Measures

A whole variety of measures exist for final products, some of which are made when
prior to the product being shipped; some of which are customer reported. These
include:
 Percent that are perfect when shipped
 Percent that works right the first time [out of the box]
 Percent of warranty claims
 Percent requiring service
 Expenditures on engineering changes
 Average time between service calls

B. Quantity-Related Measures

Efficiency Related Measures

Simple measures that measure the efficiency of a process or a system include
 measures related to the number process steps (fewer the better).
 the percent of labor hours spend on value-added operations, and
 the ratio of direct labor to total labor for the business.

Many other measures could be created that are variations on these themes.

Process Time Related Measures

Cycle time is one of the most important variables in manufacturing. In typical long
cycle times, non-value added activities form a high percentage of the cycle. Long cycle
times create tremendous problems for manufacturers including the following:
 High work-in-process inventory - materials are sitting on the floor waiting to be
processed
 Make finished products inventory - when a company cannot make product quickly
enough, it must create finished products inventory so customer demand can be
quickly satisfied. This creates potential inventory holding costs and obsolescence
costs.
 Production changes - when goods take a long time to make, it is much more likely
that changes will occur, including production amount, engineering changes, and
actual changes to the process. All of these are expensive.
 Complex systems - when parts are sitting out on the floor, complex inventory
systems are needed to track them. These systems are expensive to build and
maintain.
 Uneven loading of work centers - when large batch sizes with long lead times are
made, it is very difficult to balance the flow of materials through the plant system.
World class manufacturers reduce the size of the production lot to improve the
flow.
 Inflexibility - long cycle times cause difficulties in responding to changing customer
needs

Today's world class manufacturers shorten cycle times by reducing lot sizes and by
employing synchronized production planning and control. Just-in-time (JIT) is the key
to this effort. Visual in-plant inventory systems such as Kanban are primarily reactive
and are no substitute to an effective JIT system. Measurements that are used to
monitor process times include:
 Manufacturing cycle time - this is typically done by continual recording actual cycle
data into a tracking system or manually, by sampling the system periodically, and
by using pragmatic methods (for example dividing the number of products
produced by the work in process remaining). Simply tracking the average lot size
can often be a key indicator.
 D/P ratio - the ratio D/P = [Delivery Lead Time/Production Lead Time] in which
delivery lead time is the amount of time is the lead time offered to customers and
the production lead time is the amount of time required to make the product. When
D is greater than P the product can be made to order. When D is less than P, the
product must be made to stock. The lower the ratio, the more likely the customer
will be happy because he has more flexibility. The trick with this ratio, is figuring
out how to accurately measure D & P.
 Setup times - with small lot sizes, setup time becomes critical. Typically work
teams work together over a period of time to reduce setup times. The most typical
measure of setup time is the standard deviation. Ideally, one would want the
average setup time to be small, with a very low standard deviation. This would
indicate that the production team can perform setups quickly with great precision,
thus creating little disturbance to smooth product flow in the system.
 Material availability - percent of time all parts needed are available; percent of job
schedules that can be released for production are two reasonable measures
 Distance of material movement during production - short distances; measured
from engineering drawings; fork-lift mileage
 Machine up-time - percentage of time machine is producing parts; percentage of
time machine is down for repair
 Customer service time - time it takes from the time the order is received until the
customer receives the product

Production Flexibility Related Measures

Production flexibility is highly desirable in today's customer driven marketplace.

Companies with short production cycles are much more flexible than those with long
cycle times. The commonality of components and subassemblies, the degree of
process commonality between products, and the number of subassemblies (fewer =
better) are all important characteristics of flexible processes.
In addition, production flexibility has another major component--workers who are able
to deliver multiple skills, solve problems, learn new skills continuously, and work
independently. These measures will be addressed in the Organizational Learning
measurements section.

Potential measures that indicate the flexibility of a company's manufacturing facility

include:

 Number of different parts in a bill of materials (smaller = better)

 Percentage of standard, common, and unique parts in the bill of materials
(higher = better)
 The number of production processes (fewer = better)
 The number of new products introduced each year and the time taken to launch
them (more new products: more = better; time to launch average: smaller =
better)
 Cross-training of production personnel (more cross-trained = better)
 Comparison of production output and production capacity (running close to
100% of capacity across all processes = better)

Appendix-II

Measures of Service Quality

Service processes can be so individualized that they seem to defy systemization and
measurement.

But some common ideas and themes do exist.

There are 10 common dimensions in service quality that can be measured thro’
performance indicators or metrics:

 Tangibles—physical appearance
 Reliability—perform as promised constantly
 Responsiveness
 Competence
 Courtesy
 Credibility
 Security/Safety
 Access—easy to do business
 Communication—keeping customer informed
 Understanding customer needs