Core Tools

MSA

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 1
 Welcome to training programme on
Measurement System Analysis
(MSA Reference Manual
Fourth Edition, June 2010)

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 2
Course Content and Agenda

 Basic Concepts and Definitions

o Measurement, Measurement System Analysis
o Purpose
o Basic concepts of Bias, Linearity, Stability, GRR,
o Accuracy, Precision, Consistency, Uniformity,
o Location Variation / Width Variation
o Measurement System Capability & Performance
 Standard Requirements
 Stability Study
 Bias Study
 Linearity Study
 Gauge R&R Study
 Attribute MSA – Kappa Method
 Q&A
This MSA course covers only limited example of Stability Study, GRR Study and Kappa Study without
opportunity of practical exercises. For comprehensive course on MSA, participant may opt for 2-day course
with exercises.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 3
Introduction,
Introduction,Purpose andand
Purpose Terminology
Terminology

Decision to adjust a manufacturing process or not is now commonly based on measurement data.

Another use of measurement data is to determine if a significant relationship exists between two or
more variables.

An analytic study is one that increases knowledge about the system of cause that affect the
process.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 4
Quality
QualityofofMeasurement
MeasurementDataData

The statistical properties most commonly used to characterize the quality of data are the
bias and variance of the measurement system.

The property called bias refers to the location of the data relative to a reference (master)
value, and the property called variance refers to the spread of the data.

Large amount of variation may not be appropriate for use in analyzing a manufacturing
process because the measurement system’s variation may mask the variation in the
manufacturing process.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 5
Purpose
Purpose

Customer approval is required for measurement systems analysis methods not

covered in this manual

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 6
Terminology
Terminology

 Measurement is defined as “the assignment of numbers [or values] to material things

to represent the relations among them with respect to particular properties.” This
definition was first given by C. Eisenhart (1963). The Process of assigning the
numbers is defined as the measurement process, and the value assigned is defined
as the measurement value.

 Gage is any device used to obtain measurements; frequently used to refer

specifically to the devices use on the shop floor; includes go/no-go devices.

 Measurement system is the collection of instruments or gages, standards,

operations, methods, fixtures, software, personnel, environment and assumptions
used to quantify a unit of measure or fix assessment to the feature characteristic being
measured; the complete process used to obtain measurements.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 7
Standard
Standard

 Accepted basis for comparison

 Criteria for acceptance

 Known value, within stated limits of uncertainty, accepted as a true value

 Reference value

A standard should be an operational definition: a definition which will yield

the same results when applied by the supplier or customer, with the same
meaning yesterday, today, and tomorrow

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 8
Basic
BasicEquipment
equipment

 Discrimination, readability, resolution

 Alias: smallest readable unit, measurement resolution, scale limit, or detection limit
 An inherent property fixed by design
 Smallest scale unit of measure or output for an instrument
 Always reported as a unit of measure
 10 to 1 rule of thumb

 Effective resolution
 The sensitivity of a measurement system to process variation for a particular
application
 Smallest input that results in a usable output signal of measurement
 Always reported as a unit of measure

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 9
Basic
BasicEquipment
Equipment

 Reference value

 Accepted value of an artifact

 Requires an operational definition
 Used as the surrogate for the true value

 True Value

 Actual value of an artifact

 Unknown and unknowable

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 10
Location
LocationVariation
Variation

 Accuracy
 “Closeness” to the true value, or to an accepted reference value
 ASTM includes the effect of location and width errors

 Bias
 Difference between the observed average of measurements and the reference value
 A systematic error component of the measurement system

BIAS

Measurement System’s Average Reference Value

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 11
Location
LocationVariation
Variation

 Stability
 The change in bias over time
 A stable measurement process is in statistical control with respect to location

Time

Reference Value

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 12
Location
LocationVariation
Variation

 Linearity
 The change in bias over the normal operating range
 The correlation of multiple and independent bias errors over the operating range
 A systematic error component of the measurement system

BIAS

BIAS

Size 1 Size N

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 13
Width
WidthVariation
Variation

 Precision

 “Closeness” of repeated readings to each other

 A random error component of the measurement system

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 14
Width
WidthVariation
Variation

 Repeatability
 Variation in measurements obtained with one measuring instrument
when used several times by an appraiser while measuring the
identical characteristic on the same part
 The variation in successive (short term) trials under fixed and defined
conditions of measurement
 Commonly referred to as E.V.-Equipment Variation
 Instrument (gage) capability or potential
 Within-system variation

Repeatability

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 15
Width
WidthVariation
Variation

 Reproducibility

 Variation in the average of the measurements made by different

appraisers using the same gage when measuring a characteristic
on one part
 For product and process qualification, error may be appraiser,
environment (time), or method
 Commonly referred to as A.V. – Appraiser Variation
 Between-system (conditions) variation

Reproducibility

Appraiser
A C B

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 16
Width
WidthVariation
Variation

 GRR or Gage R&R

 Gage repeatability and reproducibility: the combined estimate of
measurement system repeatability and reproducibility
 Measurement system capability; depending on the method used,
may or may not include the effects of time

Reference Value

A C B
GRR

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 17
  Measurement System Capability

 Short-term estimate of measurement system variation

(e.g., “GRR” including graphics)

 Measurement System Performance

 Long-term estimate of measurement system variation

(e.g., long-term Control Chart Method)

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 18
  Sensitivity

 Smallest input that results in a detectable output signal

 Responsiveness of the measurement system to changes in measured
feature

 Always reported as a unit of measure

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 19
  Consistency

 The degree of change of repeatability over time

 A consistent measurement process is in statistical control with respect to
width (variability)

UCL

Average Range

LCL

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 20
  Uniformity

 The change in repeatability over the normal operating range

 Homogeneity of repeatability

Size 1 Size N

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 21
System
SystemVariation
Variation

Measurement system variation can be characterized as:

 Capability
 Variability in-readings taken over a short period of time
 Performance
 Variability in readings taken over a long period of time
 Based on total variation
 Uncertainty
 An estimated range of values about the measured value in which the true
value is believed to be contained

All characterizations of the total variation of the measurement system assume

that the system is stable and consistent.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 22
Traceability
Traceability

Traceability is defined by the ISO International Vocabulary of Basic and

General Terms in Metrology (VIM) as:

“The property of a measurement or the value of standard whereby it can be

related to stated references, usually national or international standards,
through an unbroken chain of comparisons all having stated uncertainties.”

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 23
Statistical
Statistical Properties of Measurement
Properties Systems
of Measurement Systems

There are certain fundamental properties that define a “good” measurement system.
These include:

1. Adequate Discrimination:
Adequate discrimination and sensitivity. The increments of measure should be small
relative to the process variation or specification limits for the purpose of measurement.
The commonly known Rule of Tens, or 10-to-1 Rule, states that instrument discrimination
should divide the tolerance (or process variation) into ten parts of more.
This rule-of-thumb was intended as a practical minimum starting point for gage selection.

2. Statistical Stability (Only presence of Common Causes)

The measurement system ought to be in statistical control. This means that under
repeatable conditions, the variation in the measurement system is due to common causes
only and not due to special causes. This can be referred to as statistical stability and is
best evaluated by graphical methods.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 24
Statistical
Statistical Properties of Measurement
Properties Systems
of Measurement Systems

3. Contribution of Measurement System Variation compare to Specification limits

For Product control, variability of the measurement system must be
small compared to the specification limits. Assess the measurement
system to the feature tolerance.

4. Contribution of Measurement System Variation compare to Process Variation

For process control, the variability of the measurement system ought to demonstrate
effective resolution and be small compared to manufacturing process variation.
Assess the measurement system to the 6-sigma Process variation and/or
Total Variation from the MSA study.
Effect on Process Decisions
(Cp)2obs = (Cp)2actual + (Cp)2msa

Therefore the observed process capability is a combination of the actual process capability plus
the variation due to the measurement process. To reach a specific process capability goal
would require factoring in the measurement variation.
TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 25
IATF 16949:2016 Standard requirements

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 26
Preparation
Preparation forfor
a Measurement System
a Measurement StudyStudy
System

1) Since the approach to be used should be planned. For instance, determine by

using engineering judgment, visual observations, or a gage study, if there is an
appraiser influence in calibrating or using the instrument.

2) The number of appraisers, number of sample parts, and number of repeat readings
should be determined in advance.

3) Since the purpose is to evaluate the total measurement system, the appraisers
chosen should be selected from those who normally operate the instrument.

4) Selection of the sample parts is critical for proper analysis and depends entirely
upon the design of the MSA study, purpose of the measurement system, and
availability of part samples that represent the production process.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 27
Preparation
Preparation forfor
a Measurement System
a Measurement StudyStudy
System

For Product Control situations where the measurement result and decision criteria
determine, “conformance or nonconformance to the feature specification” (i.e., 100%
inspection of sampling), samples (or standards) must be selected, but need not cover
the entire process range. The assessment of the measurement system is based on the
feature tolerance (i.e., %GRR to TOLERANCE).

For Process Control situations where the measurement result and decision criteria
determine, “process stability, direction and compliance with the natural process
variation” (i.e., SPC, process monitoring, capability, and process improvement), the
availability of samples over the entire operating range becomes very important. An
independent estimate of process variation (process capability study) is recommended
when assessing the adequacy of the measurement system for process control (i.e.,
%GRR to process variation).

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 28
Preparation
Preparation forfor
a Measurement System
a Measurement StudyStudy
System

When an independent estimate of process variation is not available, OR to determine

process direction and continued suitability of the measurement system for process
control, the sample parts must be selected from the process and represent the entire
production operating range.

The variation in sample parts (PV) selected for MSA study is used to calculate the Total
Variation (TV) of the study. The TV index (i.e., %GRR to TV) is an indicator of process
direction and continued suitability of the measurement system for process control. If the
sample parts DO NOT represent the production process, TV must be ignored in the
assessment. Ignoring TV does not affect assessments using tolerance (product control)
or an independent estimate of process variation (process control).

Samples can be selected by taking one sample per day for several days. Again, this is
necessary because the parts will be treated in the analysis as if they represent the range
of production variation in the process. Since each part will be measured several times,
each part must be numbered for identification.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 29
Preparation
Preparation forfor
a Measurement System
a Measurement StudyStudy
System

5) The instrument should have a discrimination that allows at least one-tenth of the
expected process variation of the characteristic to be read directly. For example, if
the characteristic’s variation is 0.001, the equipment should be able to “read” a
change of 0.0001.

6) Assure that the measuring method (i.e., appraiser and instrument) is measuring
the dimension of the characteristic and is following the defined measurement
procedure.

The manner in which a study is conducted is very important. The measurements should
be made in a random order to ensure that any drift or changes that could occur will be
spread randomly throughout the study. The appraisers should be unaware of which
numbered part is being checked in order to avoid any possible knowledge bias.
However, the person conducting the study should know which numbered part is being
checked and record the data accordingly, that is Appraiser A, Part 1, first trial; Appraiser
B, Part 4, second trial, etc.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 30
 Stability Study

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 31
Guidelines
Guidelines forfor
Determining Stability
Determining Stability

Conducting the Study

1) Obtain a sample and establish its reference value(s) relative to a traceable

standard. If one is not available, select a production part that falls in the mid-
range of the production measurements and designate it as the master sample
for stability analysis. The known reference value is not required for tracking
measurement system stability.

It may be desirable to have master samples for the low end, the high end, and
the mid-range of the expected measurements. Separate measurements and
control charts are recommended for each.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 32
Guidelines
Guidelines forfor
Determining Stability
Determining Stability

2) On a periodic basis (daily, weekly); measure the master sample three to five
times. The sample size and frequency should be based on knowledge of the
measurement system. Factors could include how often recalibration or repair
has been required, how frequently the measurement system is used, and how
stressful the operating conditions are. The readings needs to be taken at
differing times to represent when the measurement system is actually being
used. This will account for warm-up, ambient or other factors that may change
during the day.

3) Plot the data on an X & R or X & S control chart in time order.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 33
Guidelines
Guidelines forfor
Determining Stability
Determining Stability

Analysis of Results - Graphical

4) Establish control limits and evaluate for out of control or unstable conditions
using standard control chart analysis.

Analysis of Results - Numerical

Other than normal control chart analyses, there is no specific numerical analysis or index
for stability.

If the measurement process is stable, the data can be used to determine the bias of the
measurement system.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 34
Guidelines
Guidelines forfor
Determining Stability
Determining Stability

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 35
Example
Example- Stability
- Stability

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 36
Example
Example- Stability
- Stability

Calculation Guideline for finding out

X bar bar/ UCLx bar & LCLx bar ; R bar / UCLR & LCLR

6.021 6.297 5.746

0.4779 1.010 0

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 37
 BIAS Study

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 38
Guidelines forDetermining
Guidelines for DeterminingBias--
BiasIndependent
– Independent Sample
Sample Method
Method

Conducting the Study

1) Obtain a sample and establish its reference value relative to a traceable
standard. If one is not available, select a production part that falls in the mid-
range of the production measurements and designate it as the master
sample for bias analysis. Measure the part n ≥ 10 times in the tool room, and
compute the average of the n readings. Use this average as the “reference
value.”

It may be desirable to have master samples for the low end of the expected
measurements, the high end, and the mid-range. If this is done, analyze the
data using a linearity study.

2) Have a single appraiser measure the sample n ≥ 10 times in the normal

manner.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 39
Guidelines forDetermining
Guidelines for DeterminingBias
Bias
35 --–Independent
Independent Sample
Sample Method
Method

Analysis of Results - Graphical

3) Plot the data as a histogram relative to the reference value. Review the
histogram, using subject matter knowledge, to determine if any special caused or
anomalies are present. If not, continue with the analysis. Special caution ought to
be exercised for any interpretation or analysis when n < 30.

Analysis of Results - Graphical

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 40
Guidelines forDetermining
Guidelines for DeterminingBias
Bias
35 --–Independent
Independent Sample
Sample Method
Method

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 41
Guidelines forDetermining
Guidelines for DeterminingBias
Bias
35 --–Independent
Independent Sample
Sample Method
Method

7) Bias is acceptable at the α level if zero falls within the 1- α confidence bounds around
the bias value:

< Bias +

The α level which is used depends on the level of sensitivity which is needed to
evaluate/control the process and is associated with the loss function (sensitivity curve)
of the product/process. Customer agreement should be obtained if an α level other
than the default value of .05 (95% confidence is used).

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 42
Guidelines forDetermining
Guidelines for DeterminingBias
Bias
35 --–Independent
Independent Sample
Sample Method
Method

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 43
 LINEARITY Study

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 44
Guidelines
Guidelines forfor
Determining Linearity
Determining Linearity

Conducting the Study

Linearity can be evaluated using the following guidelines:

1) Select g ≥ 5 parts whose measurement, due to process variation, cover the

operating range of the gage.

2) Have each part measured by layout inspection to determine its reference value
and to confirm that the operating range of the subject gage is encompassed.

3) Have each part measured m ≥ 10 times on the subject gage by one of the
operators who normally use the gage.
 Select the parts at random to minimize appraiser “recall” bias in the
measurements.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 45
Guidelines
Guidelines forfor
Determining Linearity
Determining Linearity39

Analysis of Results - Graphical

4) Calculate the part bias for each measurement and the bias average for each part.

5) Plot the individual biases and the bias averages with respect to the reference values on a
linear graph.

6) Calculate and plot the best fit line and the confidence band of the line using the following
equations.

7) Plot the “bias = 0” line and review the graph for indications of special causes and the
acceptability of the linearity.
For the measurement system linearity to be acceptable, the “bias = 0” line must lie entirely
within the confidence bands of the fitted line.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 46
Guidelines
Guidelines forfor
Determining Linearity
Determining Linearity39

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 47
 R& R Study
(Average and Range Method)

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 48
Guidelines forDetermining
Guidelines for DeterminingRepeatability
Repeatabilityandand Reproducibility
Reproducibility 43

Analysis of Results - Numerical

Although the number of appraisers, trials and parts may be varied, the
subsequent discussion represents the optimum conditions for conducting the study.

1) Obtain a sample of n ≥ 5 parts that represent the actual or expected range of

process variation.

2) Refer to the appraisers as A,B,C, etc. and number the parts 1 through n so that the
numbers are not visible to the appraisers.

3) Calibrate the gage if this is part of the normal measurement system procedures.
Let appraiser A measure n parts in a random order and enter the results in row 1.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 49
Guidelines forDetermining
Guidelines for DeterminingRepeatability
Repeatabilityandand Reproducibility
Reproducibility 43

4) Let appraisers B and C measure the same n parts without seeing each other’s
readings; then enter the results in rows 6 and 11, respectively.

5) Repeat the cycle using a different random order of measurement. Enter data in
rows 2,7 and 12. Record the data in the appropriate column. For example if the
first piece measured is part 7 then record the result in the column labeled part 7.
If there trials are needed, repeat the cycle and enter data in rows 3, 8 and 13.

6) Steps 4 and 5 may be changed to the following when large part size of
simultaneous unavailability of parts makes it necessary:

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 50
Guidelines forDetermining
Guidelines for DeterminingRepeatability
Repeatabilityandand Reproducibility
Reproducibility 43

 Let appraiser A measure the first part and record the reading in row 1. Let
appraiser B measure the first part and record the reading in row 6. Let
appraiser C measure the first part and record the reading in row 11.

 Let appraiser A repeat reading on the first part and record the reading in row
2, appraiser B record the repeat reading in row 7, and appraiser C record the
repeat reading in row 12. Repeat this cycle and enter the results in rows 3, 8,
and 13, if three trials are to be used.

7) An alternative method may be used if the appraisers are on different shifts. Let
appraiser A measure all 10 parts and enter the reading in row 1. Then have
appraiser A repeat the reading in a different order and enter the results in rows 2
and 3. Do the same with appraisers B and C.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 51
GRR
GRRAcceptance Guidelines
Acceptance Guidelines

Acceptance criteria based on estimated value of R&R (%R&R) of system

• If % GRR < 10%, system is acceptable

• If 10% < % GRR < 30%, system may be acceptable based on application, cost of gage,
cost of repair, etc.
• If % GRR > 30%, system needs improvement or corrective action

+ In addition, the ndc to be greater than or equal to 5.

Another statistic of the measurement system variability is the number distinct categories (ndc).
This statistic indicates the number of categories into which the measurement process can be
divided. This value should be greater than or equal to 5

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 52
GRR
GRRAcceptance Guidelines
Acceptance Guidelines
Significance of ndc value in GRR study

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 53
GRR
GRRAcceptance Guidelines
Acceptance Guidelines

Possible causes for poor

Repeatability & Reproducibility

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 54
Example of GRR Study
GRR Acceptance Guidelines

Page 1

Data
Collection
sheet with
Basic
calculation of
X Bar &
Range….
Continue
in Next Slide
too..

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 55
Example of GRR Study
GRR Acceptance Guidelines

Page 1
Data Collection sheet with Basic calculation of
X Bar & Range, X bar diff, Part Average, R double bar
…. Continued from Previous slide

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 56
Example of GRR Study
GRR Acceptance Guidelines

Conclusion

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 57
Attribute
AttributeMeasurement
MeasurementSystems StudyStudy
Systems

Hypothesis Test Analyses

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 58
 Part A-1 A-2 A-3 B-1 B-2 B-3 C-1 C-2 C-3 Reference
1 1 1 1 1 1 1 1 1 1 1
2 1 1 1 1 1 1 1 1 1 1
3 0 0 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0 0 0
5 0 0 0 0 0 0 0 0 0 0
6 1 1 0 1 1 0 1 0 0 1
7 1 1 1 1 1 1 1 0 1 1
8 1 1 1 1 1 1 1 1 1 1
9 0 0 0 0 0 0 0 0 0 0
10 1 1 1 1 1 1 1 1 1 1
11 1 1 1 1 1 1 1 1 1 1
12 0 0 0 0 0 0 0 0 0 0
13 1 1 1 1 1 1 1 1 1 1
14 1 1 0 1 1 1 1 0 0 1
15 1 1 1 1 1 1 1 1 1 1
16 1 1 1 1 1 1 1 1 1 1
17 1 1 1 1 1 1 1 1 1 1
18 1 1 1 1 1 1 1 1 1 1
19 1 1 1 1 1 1 1 1 1 1
20 1 1 1 1 1 1 1 1 1 1
21 1 1 0 1 0 1 0 1 0 1
22 0 0 1 0 1 0 1 1 0 0
23 1 1 1 1 1 1 1 1 1 1
24 1 1 1 1 1 1 1 1 1 1
25 0 0 0 0 0 0 0 0 0 0

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 59
Hypothesis Test Analyses – Cross Tab Method
Part A-1 A-2 A-3 B-1 B-2 B-3 C-1 C-2 C-3 Reference
26 0 1 0 0 0 0 0 0 1 0
27 1 1 1 1 1 1 1 1 1 1
28 1 1 1 1 1 1 1 1 1 1
29 1 1 1 1 1 1 1 1 1 1
30 0 0 0 0 0 1 0 0 0 0
31 1 1 1 1 1 1 1 1 1 1
32 1 1 1 1 1 1 1 1 1 1
33 1 1 1 1 1 1 1 1 1 1
34 0 0 1 0 0 1 0 1 1 0
35 1 1 1 1 1 1 1 1 1 1
36 1 1 0 1 1 1 1 0 1 1
37 0 0 0 0 0 0 0 0 0 0
38 1 1 1 1 1 1 1 1 1 1
39 0 0 0 0 0 0 0 0 0 0
40 1 1 1 1 1 1 1 1 1 1
41 1 1 1 1 1 1 1 1 1 1
42 0 0 0 0 0 0 0 0 0 0
43 1 0 1 1 1 1 1 1 0 1
44 1 1 1 1 1 1 1 1 1 1
45 0 0 0 0 0 0 0 0 0 0
46 1 1 1 1 1 1 1 1 1 1
47 1 1 1 1 1 1 1 1 1 1
48 0 0 0 0 0 0 0 0 0 0
49 1 1 1 1 1 1 1 1 1 1
50 0 0 0 0 0 0 0 0 0 0

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 60
Since the team did not know the reference decisions for the parts, they developed
cross-tabulations comparing each of the appraisers to the other

Total Where A-x = 1 & B-x = 1 – 97, Total Where A-x = 0 & B-x = 0 – 44,
Total where A-x = 0 & B-x = 1 – 6, Total Where A-x = 1 & B-x = 0 – 3.
Expected count = Column Total x [Row Total / Grand Total]
So for A = 0 & B = 0 Expected count = 47 x [50/150] = 15.67

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 61
Hypothesis Test Analyses – Cross Tab Method

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 62
Hypothesis
HypothesisTest Analyses
Test – Cross
Analyses Tab Method
– Cross-Tab Method

Kappa is a measure of interrater agreement that tests if the counts in the

diagonal cells (the parts that receive the same rating) differ from those expected by
chance alone.

Let po = the sum of the observed proportions in the diagonal cells

pe = the sum of the expected proportion in the diagonal cells
then
kappa = po - pe
1 - pe
Kappa is measure rather than a test. Its size is judged by using an asymptotic
standard error to construct a t statistic. A general rule of thumb is that values of kappa
greater than 0.75 indicate good to excellent agreement (with a maximum kappa = 1);
values less than 0.40 indicate poor agreement.
So for A * B cross tabulation po = 44 + 97 / 150 = 0.94 &
pe = 15.7 + 68.7 / 150 = 0.56
Kappa = (0.94 – 0.56) / (1-0.56) = 0.86

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 63
Hypothesis
HypothesisTest Analyses
Test – Cross
Analyses Tab Method
– Cross-Tab Method

Kappa takes no account of the size of disagreement between the raters, but only
whether they agree or not.

Upon calculating the kappa measure for the appraisers, then team came up with the
following:

kappa A B C
A - .86 .78
B .86 - .79
C .78 .79 -

This analysis indicates that all the appraisers show good agreement
between each other.

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 64
 Question & Answer

TÜV SÜD South Asia 23 July 2018 Core Tools - MSA - Ed 2018 Rev 0 Slide 65