See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/322115082

Methods for identifying out-of-trend data in analysis of stability

measurements-Part I: Regression control chart

Article in Pharmaceutical Technology Europe · November 2017

CITATIONS READS
0 1,814

2 authors:

Mate Mihalovits Sandor Kemeny

Budapest University of Technology and Economics Budapest University of Technology and Economics
12 PUBLICATIONS 38 CITATIONS 73 PUBLICATIONS 764 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Applied statistics in pharmaceutical stability studies View project

All content following this page was uploaded by Mate Mihalovits on 08 April 2020.

The user has requested enhancement of the downloaded file.

 Peer-Reviewed

Methods for Identifying

Out-of-Trend Data in Analysis of
Stability Measurements—
Part I: Regression Control Chart
Máté Mihalovits and Sándor Kemény

I
The regulation of pharmaceutical stability n the course of stability studies performed in the phar-
studies still lacks universally accepted techniques maceutical industry, stability profiles are obtained for
regarding the identification of out-of-trend batches by measuring the change of certain attributes of
data. Three methods have been suggested for the pharmaceutical preparations over time. The samples
identifying out-of-trend data in pharmaceutical from the batches are kept at certain conditions (e.g., tem-
perature, humidity) for the time of the study and measured
stability studies: the regression control chart, the
at certain time points. The purpose of stability studies is
by-time-point method, and the slope control to establish shelf-life of products. At the registration of the
chart. In Part I of this article series, the regression medicine, the method of recording the stability profile is
control chart method is investigated, and an regulated by the International Council for Harmonization
improved approach is suggested. The method of Technical Requirements for Pharmaceuticals for Human
is illustrated using realistic data. In Part II, the Use (ICH) (1). The same method may be used to monitor
ongoing production processes.
by-time-point method and the multivariate
Certain variation in data may well be explained by uncer-
control chart method will be discussed. tainty. Larger deviations are considered out-of-trend (OOT)
results. OOT is to be distinguished from out-of-specifica-
tion (OOS); the latter means that the result is outside of the
allowed range given by the specification. As noted by the
Pharmaceutical Research and Manufacturers of America
Chemistry, Manufacturing, and Control (PhRMA CMC)
Statistics and Stability Expert Teams (2), “The procedures
described for detecting OOT results can be viewed as an
alarm or alert system, showing that some kind of action is
needed. In other words, at each stability time point when a
new result is collected, one should determine whether the
result is in agreement with what is expected and if not, take
the appropriate action.”
A stability result at a single time-point may not follow
the expected trend in two ways: either in comparison
Submitted: 22 March 2017 with earlier batches or with respect to previous results
kentoh/shutterstock.com

Accepted: 27 June 2017 collected from the observed batch. The OOT phenom-
enon is to be detected as soon as possible, because these
CITATION: When referring to this article, please cite it as M. results indicate errors either in the process of production
Mihalovits and S. Kemény, “Methods for Identifying Out-of- or in the analytical measurement. If an OOT point is
Trend Data in Analysis of Stability Measurements—Part I: detected, one should find the source of error and fix it.
Regression Control Chart,” Pharmaceutical Technology 40 Further actions may be required if the problem is found
(11) 46–53 (2017).
to be with the production process.
32
32 Pharmaceutical Technology
Pharmaceutical Technology Europe2017
November P h a r mTe
November c hPharmTech.com
2017 .com
 The PhRMA CMC Statistics and Stability Expert Teams This latter probability is not explicitly stated, however. In
began a dialogue toward building up a system, supported the pharmaceutical industry (and also from a regulatory
by statistical calculations, to identify OOT results in stabil- viewpoint), type II error (not realizing that the data is actu-
ity measurements (2). The methods suggested in this 2003 ally OOT)) presents a more serious threat, but such errors
seminal paper were supported by example calculations in can be avoided if the confidence level is decreased. The 95%
an article by Torbovska and Trajkovic-Jolevska (3). Also, confidence level is used in this paper.
statistically less efficient, but more user-friendly methods
were suggested by the PhRMA CMC Statistics and Stability Identifying OOT data by regression control chart
Expert Teams in a later paper published in 2005 (4). In this The peculiarity of identifying OOT phenomenon with re-
article, the authors reflect on the regression control chart spect to stability studies is that there is a trend in data any-
method, described in reference 2 and used in reference 3, way, because the attributes change with time due to chemi-
and attempt to refine the method. The amount of active cal and physical changes, and OOT is to be stated in relation
substance (assay) of the pharmaceutical product was used as to this obvious trend. In this method, a straight line model is
an example, but the method presented may be used for any used, however, one should justify the assumption of linearity
other attributes that follow a linear trend over time. of a stability profile.
Control charts. Control charts (7) are widely used in the in-
Detecting OOT results dustry to monitor the production process. The regular usage
Three methods are outlined and illustrated by the PhRMA of the charts can be separated into two phases (8). In the
CMC Statistics and Stability Expert Teams (2): first phase (Phase I), parameters of the distribution (typi-
• regression control chart method cally mean and standard deviation) of the reference data are
• by-time-point method estimated. In the second phase (Phase II), certain statistics
• slope control chart method. (e.g., sample mean, range, etc.) of the observed sample are
In the regression control chart method, data are com- plotted and examined. In the example used by the authors,
pared within a batch, while in the by-time-point method, the observed sample contains only a single datapoint, thus
data are compared to historical data of earlier batches at the statistic to be plotted is the individual value (individu-
the same time point. In the slope control chart method, a als control chart). The distribution parameters obtained in
feature (the slope) of the stability profile of the observed Phase I are used to construct lower and upper control limits
batch is compared to that of earlier batches. (i.e., lower and upper quantiles) for the statistic (i.e., the in-
The technique of decision is the comparison of a value dividual value) and added to the chart. The interval between
to a statistical interval. Three kinds of statistical intervals the control limits of the individuals control chart contains
are used: confidence, prediction, and tolerance. Users are the allowable value of the new data of the observed process
sometimes confused about which interval should be used with a certain probability, when the process is in control.
in certain situations (5). Typically, this probability is 0.9973, which conforms to the
The confidence interval is the range in which a parameter ±3σ range (3 is taken from z-table). This range is valid only
(e.g., the expected value) occurs with a certain probability. if the reference sample size in Phase I is theoretically infi-
The prediction interval is the range in which a future observa- nite, because the method (calculation of quantiles) assumes
tion is found with some specified probability. The tolerance the perfect knowledge of the distribution. If the distribu-
range contains a specified part of future observations (con- tion is not perfectly known (e.g., the parameters are esti-
tent) at a certain probability (confidence). There is a connec- mated only), the 0.99865 and 0.00135 quantiles are at best
tion between the latter two intervals: a prediction interval to approximate only. This method is proposed by Shewhart
contain a single future observation with 1-alpha probability for industrial processes and proved to be useful there for
is equivalent to a tolerance interval to contain, on average, distinguishing assignable and common causes, thus they
1-alpha proportion of the population (6). Calculating the con- serve as statistical process monitoring (9). The typical ap-
fidence and prediction range is straightforward; it is found in plication requires at least 25 samples in the reference set. For
standard textbooks and taught in basic statistics courses. The the authors’ study, this method is valid only if the sample
tolerance range is more difficult to calculate, and sometimes, size in the reference set of Phase I is large enough, then the
only approximate methods are available. 0.9973 probability for the ±3σ range is reached as a limit.
The interval is characterized by its acceptance probability. This approach will be referred to as the Shewhart method in
For quality engineering, usually a 99.73% confidence level is this paper. In the current situation, however, the Shewhart
used in control charts. This may not be the proper choice in method is not considered reliable because the reference data
OOT analysis of stability studies. The higher the confidence set is small.
level, the lower the chance of identifying data as OOT while In the case where the sample size is small, one should con-
it is actually not the case (i.e., type I error, “false positive sider the uncertainty of expected value and variance. The
alarm”), while the chance of accepting an OOT data as non- earlier is taken into account by including the term
OOT is increased (i.e., type II error, “false negative case”). into the standard deviation, while the latter is taken into
Pharmaceutical
Pharmaceutical TechnologyTechnology
EuropeNovember
November2017
2017 33
33
Peer-Reviewed
account by using t-distribution instead of z. Thus, the width expected value and the variance are unknown and estimated
of the valid control limits in the authors’ typical case (small only from a small sample. The method requires points in
reference set) are , which is the prediction interval. the observed batch as reference data to construct the first
The prediction limits are used in Phase II of the individu- regression line. One should decide how many points from
als control chart to check whether the stability is further the beginning of the observed batch should be considered
maintained. If a point is found to be OOT, one may choose as the reference data set. Moreover, one should assume that
from two routes, depending on whether data analysis is in those reference data are non-OOT. Here, the first three
real time or not. If the data analysis is performed in real points of the observed batch are considered to be the refer-
time, the OOT suspicion, if arisen, is followed by the repeti- ence data, and the fourth point is the first to be analyzed
tion of analytical measurement. If the non-OOT nature is by the method.
justified (by re-measurement), the stability study is contin- The prediction interval for the measured variable (y) at
ued, neglecting the point. If the OOT nature is justified, a the new time point (x*) is calculated in Equation 1:
corrective action follows the detection, and thus, the origi-
nal process is ended.
The other route is followed if the analysis of data is per- (Eq. 1)
formed not in real time but upon collecting more data. Here,
the original process is continued, and the point that is later where is the predicted value of y at , is the one-
found to be OOT cannot be re-measured and should be re- sided upper critical t value at α/2 one-sided level (with (n-2)
moved from the data set. If the OOT point is not removed degrees of freedom, where n is the number of points used
from further analysis, the acceptance range will be wid- to construct the regression line), y* is the new measured
ened, hindering detection of further OOT points. If a point value at x* time point, and is the sample standard
is found to be non-OOT, the point is added to the historical deviation calculated for the variable. The right side of
data set, new control limits are calculated, and a new control the inequality is the upper prediction limit (UPL), while the
chart is made for the next observed data. left side is the lower prediction limit (LPL). If the inequal-
Regression control chart. The regression control chart ity is satisfied, that is the y* is within the limits, the data is
is used to monitor processes where there is a systematic accepted, otherwise it is OOT.
change over time (10). The “parameters” of the chart are a is calculated through the extrapolation of the regression
regression line (i.e., an expected value changing with time) line to the x* new time point by Equation 2:
constructed by the previous data from Phase I and the
variance of residuals, and both are considered as known (Eq. 2)
in the Shewhart method. If the new point is found to be
within the control limits, the point is accepted; if not, the where b0 and b1 are the intercept and slope of the regres-
point is rejected and declared as OOT. sion line (created with reference data), respectively.

Use of regression control chart method, original proposal is calculated by Equation 3:

The PhRMA CMC Statistics and Stability Expert Teams (2)
suggest fitting a regression line to historical data of the ob-
served batch and extrapolating the line to the time point of
the new data. Control limits are calculated for the expected
value by expected value±ks at the new time point. The k is (Eq. 3)
taken from a table of normal distribution at a desired sig-
nificance level. The residual standard error (s) is calculated where sr is the residual standard deviation, n is the num-
either by only the regression line of the observed batch, or ber of points used to construct the regression line, and
by regression lines of historical batches. In the latter case, a is the mean of the xj values (time points) of data (without
common slope is assumed, and different intercepts are al- x*) used to estimate the regression line. The third term in
lowed for the historical batches. If the observed point is out the bracket is the reason of constriction of the band in the
of the interval, it is identified as OOT. It is also mentioned middle.
in the paper (2) that using prediction or tolerance interval As the PhRMA CMC Statistics and Stability Expert Teams
is a better but more complex method. The limits calculated highlights (2), the residual standard error may be calculated
there (and in reference 2) using the expected value±ks for- using earlier batches as well, not only from the recent one in
mula are called the Shewhart limits (SL) here. study. The use of earlier batches may follow several different
routes. The authors of reference 2 used parallel lines (i.e.,
Suggested use of regression control chart method the common slope for all historical batches). The authors of
The prediction interval with t-distribution is to be used if this article are not convinced about the usefulness of this
one asks about a single new observation, because both the assumption, however. If common slope is forced for histori-
34
34 Pharmaceutical Technology
Pharmaceutical Technology Europe2017
November P h a r mTe
November c hPharmTech.com
2017 .com
Peer-Reviewed
cal batches, the fit is deteriorated. This situation would lead For the calculations with pooled standard deviation, sr is
to greater residual error than would be achieved using the substituted with in Equation 6 and degrees of freedom
best linear fits (letting different slopes). On the other hand, of is used to obtain the t-score.
if separate lines are fitted to historical batches, the residual Tolerance limits can be calculated by Equation 7:
standard errors obtained may be pooled. The pooled error
has higher degrees of freedom than that obtained from a (Eq. 7)
single batch, thus, it is a better estimate to the variance, lead-
ing to better estimated control limits. If every group has Where k1 is calculated by Equation 8 (11):
the same number of data, the pooled sample variance is
the arithmetic average of the individual sample variances:

(Eq. 4) (Eq. 8)

where n’ is the effective number of observations, ν is the

where is the pooled sample variance, is the degrees of freedom of sr2 (n-2 actually), is the criti-
squared standard deviation of residuals for the ith histori- cal value of the chi-square distribution at a one-sided 1-γ
cal batch, and pi is the number of historical batches involved. level with ν degrees of freedom, is the critical value
For this purpose, regression lines were fit to each historical of non-central-chi-square distribution at one-sided P level
batch and residual standard errors (si) were calculated. The with ν degrees of freedom, and δ is the argument in the non-
pooling is justified only if the variation of data of different central-chi-square function. Actually, its value here is 1/n’.
batches has the same variance, which can be tested by the n’ can be calculated by Equation 9 (12):
Bartlett and Levene tests for example. For the authors’ data,
the hypothesis of homogeneity of variances is accepted (de-
tails of the tests are not shown here).
For calculation of the prediction interval with pooled (Eq. 9)
residual standard error, in Equation 3, is substituted for sr.
The degrees of freedom of is p(n-2), thus a different This equation is simplified to Equation 10:
t α/2 should be taken from t-table when calculating the
control limits.
For illustration purposes along with prediction limits, the
Shewhart limits (α=0.05), confidence limits (α=0.05), and (Eq. 10)
tolerance limits (P=0.99 as content, γ=0.95 as confidence
level) are calculated as well. None of these are appropri- Where
ate to use in the current situation (where only a single new
data point [y*] is observed), except for the prediction limits.
These prediction limits are discussed in detail below.
Calculation of the Shewhart limits:
(Eq. 11)

For the calculations using pooled standard deviation, sr is

(Eq. 5)
substituted with in Equation 10 and Equation 11 when-
where zα/2 is the critical value of standard normal distri- ever a new effective number (n’) of observations is to be
bution at two-sided α/2 level, and σ r is the square root of obtained. The proper substitution is performed for degrees
variance of residuals, equal to sr. As the known variance of freedom as well in Equation 8. Also, in Equation 8, a new
is assumed, there is no room for pooling. One may use the is calculated with the new n’, which is calculated
square root of the pooled standard deviation, however, as a by Equation 10.
substitute of the σr. The latter is falsely assumed to be known
because of the small sample size as explained earlier. When Example: Regression control chart
the square root of the pooled estimated variance is substi- The data in Table I are from a realistic stability data set
tuted, σr is taken to be equal to . from a pharmaceutical manufacturing process (details
Confidence limits are calculated in Equation 6: are not given due to proprietary reason). The amount
of active substance of the medicine was recorded over
(Eq. 6) 36 months.
36
36 Pharmaceutical Technology
Pharmaceutical Technology Europe2017
November P h a r mTe
November c hPharmTech.com
2017 .com
Peer-Reviewed
Table I: Stability data set.
Amount of active substance (%)
Time
Batch I Batch II Batch III Batch IV Batch V Batch VI Batch VII Batch VIII Batch IX
(month)
0 97.6 98.4 100.9 98.7 98.8 100.5 100.3 101.5 100.9
3 97.7 99.4 98.2 95.8 97.5 96.5 99.7 100.1 97.3
6 97.7 96.2 98.5 96.7 97.5 96 98.6 99.5 97.7
9 96.9 97.3 94.6 97.5 98.9 96.3 98.3 99.6 98.4
12 94 95.3 96.9 94.7 97.5 98.3 96.8 98.3 96.5
18 96.5 94.9 96.3 93.7 96.5 94.1 96.7 95.2 99.5
24 96 97.5 95.8 93.1 96 92.5 96.3 97.1 96
36 92.1 92.7 92.3 91.3 92 89.5 93.9 93.8 93.7

Table II: Prediction limits, Shewhart limits, confidence limit, tolerance limits, regression control chart method
using pooled standard deviation sp(res). LPL is lower prediction limit, UPL is upper prediction limit, LSL is lower
Shewhart limit, USL is upper Shewhart limit, LCL is lower confidence limit, UCL is upper confidence limit, LTL is
lower tolerance limit, UTL is upper tolerance limit.
Time Observed
Sp(res) LPL UPL LSL USL LCL UCL LTL UTL
(months) data
0 - 100.9 - - - - - - - -
3 - 97.3 - - - - - - - -
6 - 97.7 - - - - - - - -
9 1.2 98.4 91.0 99.8 93.1 97.8 91.8 99.1 89.9 101.0
12 1.2 96.5 93.0 100.6 91.5 96.2 93.8 99.8 91.7 101.9
18 1.2 99.5 91.0 99.1 88.3 93.0 91.8 98.3 89.8 100.4
24 1.2 96.0 88.3 98.8 85.1 89.8 88.8 98.2 92.6 102.7
36 1.2 93.7 89.3 97.9 78.7 83.4 90.1 97.2 90.3 100.7

Batch IX is the object of these calculations; the rest of the

Figure 1: Statistical intervals of regression control chart
columns refer to historical batches. The regression line was
created from the first three points of the observed batch;
110 it was then extrapolated to the new point (i.e., the ninth
Historical data New data month), and the prediction interval (LPL, UPL) is calculated
Shewhart Confidence
Prediction Tolerance along with the other intervals that should not be used to
identify OOT points (see Table II).
Amount of active substance (%)

105
The limits calculated with pooled standard deviation are
given in Table II. Using Equation 4, the calculated s_(p(res))^2
is 1.438. Figure 1 shows the fitted line, the data points up to
100 the 9-month time point, the 95% Shewhart interval (SI), the
95% confidence interval (CI), the 95% prediction interval
(PI), and the 95% tolerance interval (TI) containing 99% of
Figure 1 is courtesy of the authors.

future observations.
95
The confidence interval is very narrow in the range of
historical data (i.e., for the first 3 points); it allows an even
tighter interval than the Shewhart limits. The uncertainty
90 of the parameters of the regression line is the reason behind
the bend in the limits. The farther the new point is from
the historical data, the wider the interval is given. As the
0 3 6 9 Shewhart method assumes known parameters of the regres-
Time (month) sion line, the band of that is not bent; the limits are parallel
to the regression line.
38
38 Pharmaceutical Technology
Pharmaceutical Technology Europe2017
November P h a r mTe
November c hPharmTech.com
2017 .com
 Having accepted the nine-month time point, the calcu- References
lations are repeated, but now the line is fitted to both the 1. ICH, Q1A(R2) Stability Testing of New Drug Substances and Prod-
initial three data points and the accepted data (nine-month ucts, Step 4 version (2003).
2. PhRMA CMC Statistics and Stability Expert Teams, Pharm. Tech.,
time point) while the new data at the12-month time point 27 (4) 38–52 (2003).
is observed. Having continued the calculation practice, the 3. A. Torbovska and S. Trajkovic-Jolevska, Pharm. Tech. Eur, 37 (6)
18-month time point is found to be out of the prediction (2013).
interval, as 99.5 > UPL, therefore, it is declared as OOT. The 4. PhRMA CMC Statistics and Stability Expert Teams, Pharm. Tech.,
outlier point is then left out from the examination of later 29 (10) (2005).
5. S. De Gryze, I. Langhans, and M. Vandebroek, Chemometrics and
points if the study is to be continued, but intervention is Intelligent Laboratory Systems, 87 (2) 147–154 (2007).
clearly required. The grey colouring indicates that the ob- 6. F. Proschan, Journal of American Statistical Association, 48 (263)
served data in the row of the grey pair of boxes is OOT con- 550–564 (1953).
sidering the limits specified in the columns. The mistakenly 7. W.A. Jensen et al., Journal of Quality Technology, 38 (4) 349–364
used confidence interval would give the same qualitative (2006)
8. T.P. Ryan, Statistical Methods For Quality Improvement (John
result in the current situation, while with tolerance interval Wiley & Sons, NY, 3rd ed., 1989), pp. 89-95.
approach, all the points would be accepted, and with the 9. W.A. Shewhart, Economic control of quality of manufactured prod-
Shewhart interval method, every point would be OOT. uct. (ASQ Quality Press, Milwauke, Wisconsin, 1931)
10. B.J. Mandel, Journal of Quality Technology, 1 (1) 1–9 (1969).
Conclusion 11. K. Krishnamoorthy, T. Mathew, Statistical Tolerance Regions:
Theory, Applications, and Computation (John Wiley & Sons, New
Having realized that the regulation of pharmaceutical Jersey, 2009), pp. 70.
stability studies is still lacking universally accepted tech- 12. W.A. Wallis, Proceedings of the Second Berkeley Symposium on
niques for the identification of OOT data, three methods Mathematical Statistics and Probability (University of California
were suggested earlier by PhRMA CMC group. The re- Press, Berkeley, Calif., 1951), pp. 43–51. PTE
gression control chart method discussed in this paper is
a possible way to identify OOT data within a batch. To be Máté Mihalovits is a PhD student, [email protected];
and *Sándor Kemény is an emeritus professor,
statistically more rigorous, the earlier suggested approach
Tel.: +36 309936307, [email protected], both at
is improved. As only a small sample is available, the use of Budapest University of Technology and Economics,
prediction interval with t-distribution is justified. Incor- Faculty of Chemical Technology and Biotechnology,
rectly using the tolerance interval instead would unneces- Department of Chemical and Environmental Process
sarily widen the acceptance interval, while the unjustified Engineering Hungary, 1111. Budapest, Műegyetem rakpart 3.
use of the Shewhart procedure would result in too narrow
* To whom all correspondence should be addressed.
an interval.

♦ Call for Papers ♦ Call for Papers ♦ Call for Papers ♦ Call for Papers ♦

Pharmaceutical Technology Europe and Pharmaceutical Technology cover all aspects of phar-
maceutical drug development and manufacturing, including formulation development, process
development and manufacturing of active pharmaceutical ingredients (both small molecule
and large molecules) and finished drug-products (solid dosage, semisolid, liquids, parenteral
drugs and topical drugs), drug-delivery technologies, analytical methods development, ana-
lytical testing, quality assurance/quality control, validation and advances in pharmaceutical
equipment, machinery, instrumentation, facility design, and plant operations.

We are currently seeking novel research articles for our peer-reviewed journal as well as manuscripts for our special issues.
For peer-reviewed papers, members of the Editorial Advisory Board of Pharmaceutical Technology Europe and Pharmaceuti-
cal Technology and other industry experts review manuscripts on technical and regulatory topics. The review process is
double-blind. Manuscripts are reviewed on a rolling basis.

Our single-themed issues, which include literature reviews and tutorials, address excipients and ingredients, analytical testing,
outsourcing, solid dosage and more.

Please visit our website, www.PharmTech.com/pharmtech-author-guidelines-and-editorial-calendars, to view our full Author

Guidelines. Manuscripts may be sent to Editorial Director Rita Peters at [email protected].

We look forward to hearing from you.

Pharmaceutical
Pharmaceutical TechnologyTechnology
EuropeNovember
November2017
2017 39
39

View publication stats